If it needs to be said, it needs to be said, and if you're concerned about being approached and accused of personally hating the town, point out that the accuser needs to read more carefully. I used to be afraid of repercussions in real life about what I'm writing due to the religious beliefs I was articulating, but since I stopped worrying, the writing has taken off and become easier.
I, too have a character who dislikes where he is. Some of his motives/thoughts/comments stem from the fact that he was born somewhere else and thinks the place he's currently in is inferior. I just had to be careful because he was starting to become rather two-dimensional what with all his complaining and griping. The only reactions to his dislike that I'm concerned about are other characters'

Although my setting is quite different because it's a fictional place!