I have many of the same misgivings that you do. At first blush, it seems that we are punishing people for their thoughts. Of course, it's not quite that simple. They are being punished for the act of assaulting someone. So it's not
just for their thoughts. Thus, the question becomes whether there is a legitimate justification for enhancing the punishment because of their thoughts. (I'm assuming for present purposes that enhancing the punishment just because the criminal had immoral thoughts is not legitimate. Others can argue that point if they wish.)
In different circumstances, punishment is enhanced when there is some element of the crime that makes the criminal more dangerous. For example, just breaking into a house is a crime, but it is a more serious crime, and punished more severely, if the purpose in breaking into the house is to commit a felony. Robbing someone is a crime, but it is a more serious crime, and punished more severely, to use a dangerous weapon while robbing someone. In each of the second situations, because the criminal is more dangerous, the punishment is more severe.
It is at least arguable that someone who assaults others just because of their race, religion, sexual orientation, etc, is more dangerous than the average assailant because of that motive. To the extent that this is what is actually motivating legislatures when they enact hate crime legislation, it is defensible on that basis. Whether a particular legislature is actually relying on this reasoning, or instead is simply enhancing punishment because they wish to punishment the criminal for having hateful thoughts, must be examined on a case by case basis.
Ridicule is the only weapon which can be used against unintelligible propositions. Ideas must be distinct before reason can act upon them; and no man ever had a distinct idea of the trinity. It is the mere Abracadabra of the mountebanks calling themselves the priests of Jesus. -- Thomas Jefferson
For we know that our patchwork heritage is a strength, not a weakness. We are a nation of Christians and Muslims, Jews and Hindus -- and non-believers. -- Barack Obama
We see monsters where science shows us windmills. -- Phat