I was going to say similar. It's not your fault, guy. We've grown up in a country that subtly, and sometimes not subtly, sends out images of fear. Stereotyping the black race with the news, the tv shows like COPS, movies, and a handful of rap artists... To me, your statement reinforces this belief I have. You seem to have resisted the hatespeak, but they got a foot in the door on the 'black issue'.
It is racism, sort of. It's fearmongering, quietly. You've seen and met many normal, good-natured African-American people and you've seen obnoxious, loud, and potentially destructive people of all walks of life. Black male. Scary. Right? Wrong.
I work at a gas station in the Sacramento area. It's about 25 minutes out from Sac. It's a predominantly Hispanic and white town. When a ghetto-dressed black man comes in to the store, swaggerin, grill in mouth, mean muggin... I get that feeling that you get. I know better. My best friend is black my cousin married a black man and they're raising a family together. Something has been engrained in us. I've never felt that way from an Asian person. Hispanic, maybe. White, yea. Everybody is roughly the same. We are all people. What do I do? I let the adrenaline subside, the thoughts of how to defend myself, and approach these folks as I would any other friend, little old lady, or anybody. Polite, crack a joke, be sincere. You can't fake sincerity well, either. If you don't notice a difference between when they came in and when they left, maybe they're unhelpable. Or maybe they didn't believe you truly were real with them. Even if you are! I see it both ways. If I was him, I'd be thinking "Why's he being so mellow? Most white people won't even look me in the eyes."
It's an act. Most people who hurt act all day. The world's a stage, indeed. Wise words.
Break the mental slavery. Some people have it a whole lot worse. Some people are so afraid of colored people, they've never had a sincere moment to connect with anyone different. So they never fix their broken stereotype. At a certain point, you won't change your mind without an epiphany...
But real quick... You'd imagine if you were deathly hateful towards colored people and a colored person saved your life. Would you or would you not change the way you felt? I've seen/heard stories where after the fact, the helper gets shat on. And the stereotype lives on. It's absurd.

This is representative of many minds in America^