Sure there's a hard core of Labour support, either through circumstance - people who feel Labour policies will do more for them in the situation they are in; or through family tradition perhaps.
But the point is the numbers of such people, as time passes and the nation as a whole becomes more and more affluent, is decreasing. There's a clear correlation I am sure you would agree, between the levels of prosperity in a given area and the propensity for that ward to return either a Labour or Tory MP. Many more Labour MPs in the relatively less well off North, relatively less in the more affluent south and what support there is, is concentrated in inner city areas with higher levels of poverty. So it logically follows that greater affluence favours the Tories, and the country for the past 50 years has been getting more and more affluent.
I found this data quite interesting from the Guardian, showing the rise of union power over the past decades, with membership peaking on the post war years, and declining away again nowadays. Correlated with the amount of unrest and days lost due to strikes.