This has more to do with income inequality than housing. The middle-class has largely evaporated (tech workers are probably the closest thing to it now). And, even when there was a thriving middle-class in this country, they didn't really live in places like Manhattan anyway -- they commuted in from suburbs. Building low-income housing is fraught with difficulty. Even if you were to pour money into it, you'd still risk manufacturing slums anyway.