It's not like people are waiting for you to take the clothes of your back.
I've went to some concerts in Detroit when my cousin lived there, I also went out a bunch of times around the city to see the clubs and nightlife.
I'm not saying it's Disneyland or anything, but nothing ever happened to me.