Since most people do not pay it, it's quite bizarre. It's about as relevant to most of us as a special tax on houses with more than 25 bedrooms.
Taxes are by their nature unpopular. There is an alternative (as I think I mentioned the other day.) No taxes at all, but the government pays for its spending by printing/creating money. You would be looking at very high inflation, but hey, never mind. No tax! The UK pound belongs to the government, wich has a monopoly of creating it, so the government could, in theory, do this.
Taxes are simply a way of keeping inflation (relatively) low. This is highly desirable for 1001 reasons. They enable a portion of national resources to be allocated to public services without at the same time creating massive inflation.
Want to abolish IHT? No problem. Increase Income Tax to 25p (or whatever the necessary level), higher rate to 45p, top rate to 50p. Who would that benefit, and who would gain? Those inheriting large estates would benefit, and everyone else would lose. Try putting that to the punters, see what they say.