I wouldn't actually 'call' someone a brother or a sister, but to describe a certain relationship, I've used that term before. As in 'he's like a brother to me' --- that sort of stuff.

What exactly freaks you out about it? I mean, surely somebody calling me sister - out of the blue would freak me a little, but then if a male friend would say it to clarify the sort of relationship we have, I wouldn't have a problem with it.