When the story broke back in 2012, I remember me and some friends all saying this, and my parents just not seeing it.
I think to the older generation it was a bit of the boiling frog effect where he just always dressed like that and when they were kids they got used to it, whereas the younger generation saw him at 70+ and immediately thought it was creepy.
I also wonder how much a caricature of a pedophile in this country from comedy sketches etc. would be informed and defined by the non-stop rumours about one of the most famous people in the country being one.