Most of the welfare state is state pensions.
Another massive chunk goes to working people to subsidise inadequate wages and excessive rents. (I'm quite up for not subsidising bad employers and greedy landlords, but are the Tories?)
Another chunk goes on disability benefits, including those paid to disabled people in work. (You can claim PIP and work if you qualify, and as it is (rightly) not means-tested you could be the MD of a FTSE 100 company in theory.)
What people go on about is the money given to long-term unemployed, and that is petty cash in the scheme of things. If it was stopped tomorrow, I doubt you'd get 10p off your tax. But you would get a whole lot of extra crime in exchange.
(EDIT) Forgot to point out that the 'economically inactive' includes full-time students and all those people over 55 who have retired on private pensions, and don't need to work anymore as they have enough to carry them through to state pension age and beyond. These people aren't going to work unless you seize their assets.