You could also look into the outer boroughs of NYC, particularly Brooklyn and Queens (of which I am now an expert since I just moved here). Much cheaper than Manhattan but still part of the city. It can also be the best of both worlds since some areas are quiet and mostly residential with private houses but still right on a subway line. And subways are good because they run 24 hours a day unlike the trains to and from New Jersey..