What does the young man mean by “great honey-coloured/Ramparts at your ear?” Why does he say that young men are “thrown into despair” by them?

The “great honey-coloured/Ramparts at your ear” refers to the beautiful yellow coloured hair that falls at the woman’s ear and cover it like a wall around a fort. He says that the young men are “thrown into despair” by them because they look so beautiful on the woman that her beauty gets thoroughly enhanced. The young men fall in love with her and feel despair. He says that it is not possible that someone would love her alone and not her yellow hair.