What does the young man mean by “great honey-coloued /Ramparts at your ear?” Why does he say that young men are “thrown into despair” by them?

The young man refers to the beautiful yellow hair of the woman. He calls them ramparts because they act like a protective fort framing her face. Her beautiful hair is so attractive that many young men fall for her. However, though everyone wants her, they can’t get her. So, they are thrown into despair by her rejection.