Certainly imaginable with the blends. Would you not be able to tell with the 3 year olds since they shouldn't be all that coloured? Doing that to anything older than 3 would be daft imo unless that's the flavour they're going for.
I was reading on a whisky forum that some of high end single malts, stuff that costs over £100 a bottle add it too. One of my favourite whiskies uses it, Glenmorangie. If it says contains E150 on the side of the bottle, then it uses caramel to colour it.