Most harm would be done to his personal life & Family, normal fans wouldn't really be that bothered as long as the music keeps coming.
Then again I guess the obsessed fans would be making a fuss!