Think one of the biggest points is like you say most people were either Nadal or Federer fans, then he came along too late and fans stuck with their favourite with not many making the switch to supporting him - especially as when you look at how he acted when he first started, he was impressively unlikeable.
He's manufactured himself to try and be more likeable but it has never really worked as I think most people can see that it's a show and people love genuineness. For what it's worth if you read what other players say about him it seems he genuinely is a nice bloke off the court.
The other big factor is his style of play is pretty dull. He's an absolute wall with remarkable movement - but that's not going to get people off their feet often unlike Federer/Nadal/Alcaraz/Sinner can do (or did).
