It's not just population, it's the age of the population.
Currently, we have a higher percentage of retired folk than we had traditionally, and fewer babies being born. The proportion of retired is growing every year, and will until the boomer generation passes over. So the population is skewed towards the older age groups. Immigration partly redressed this. The only other 'solution' is to make people work longer. But many jobs are unsuited to those over 60. Builders, especially roofers and the like. Police. Fire. Arguably doctors and nurses. You want a surgeon with a shaky hand? You want a bloke of 74 driving your train at 125 mph? Or your bus at 30, for that matter?
With a restructure of society, we could put over-60s on 'light duties' and ask them to work longer. But such jobs are nowhere near as common as they used to be. Routine clerical jobs, for example, are a dying institution. Most have already gone. Bear in mind also that in a free market state it's very hard to restructure in this way, No mechanism exists.
Housing is squeezed because objectively we haven't built enough for decades. Thatcher smashed public sector housing as she imagined it would free up resources for the private sector. So it did, and that's why you get profitable 5 and 6-bedroom executive houses being built instead of what young families and single people need.
There are more single households because that's how society has evolved, like it or not. There are more divorces, and even many 'couples' live in separate houses because they don't want a closer relationship with the financial entanglement that involves. Again, a more aged society means more widows/widowers. It's OK saying downsize. If you can put all your possessions in one box it's easy. When you have a lifetime's accumulated possessions and you're on your tod, it isn't. It's also costly to move. I estimate 15k and upwards. I wouldn't do that just so I could live in a shoebox. I'd live on blind hash and baked beans first.