A French saying would say :" I see you coming with your shoes as big as a house."
Anyway...
I've hardly ever been attacked but avoided quite a few conflicts by just staying calm, showing some serenity and that I haven't got a sudden rush to leave, keeping talking and eye contact, even joking.
I was punched one time by a guy in Buenos Aires I didn't wan't to give my watch to. I didn't hit back. I just walked away and he stood there just waiting for some reaction and never chased after me. I'm not a fighting guy but even if I was I don't like to give bastards what they want, and I couldn't care less if someone I don't know, and would most certainly despise anyway, thinks I'm a coward.
That was in 1998 and I still have my watch, and it's got this extra sentimental value now

.
Unless your (or anyone's) life is really in danger, and there's no other choice to end the incident, there's no need to figh back, imo, it will just make things worse, short and long run.
Ponrauil