Jersey City is expensive near the waterfront. Parts of Journal Square are still viable. The next 5 years might change that (Journal Square), but its still cheap in the sense. Now is the time to rent. I talk to people who wanted and/or are planning to move to JC, but they all think waterfront. There's a stigma against Journal Square and the surrounding areas. They think ghetto, but I tell them thats not the case.
The area around Hamilton Park is viable, likewise some homes around Van Vorst Park. Location is key, but the city is far from expensive. Now if you want a ripoff, and enjoy living in a 400-500 sq ft shoebox, than Hoboken is right there. More housing in JC, without getting raped in rent. Plus, drinks aren't $14 for a Whisky sour thats weak as hell.
Friend of mine makes 55k a year, lives in Hoboken (not even in a exclusive area either), and spends 50% on housing. Its nuts. I told him Journal Square, but he enjoys paying $1,800 a month on a 450 sq foot shack. Then he wonders why he's poor after taxes, and other living expenses/maintenance fees.