It probably stems from the notion that he was meant to be a special talent. When the reality is he's not. He's a good technical premier league winger with no truely outstanding attributes.
He doesn't score a lot
He doesn't assist a lot
He doesn't dribble past his fullback constantly and make things happen
He doesn't run his socks off for the team
He can't dictate the tempo of a game
He's not rapid
He doesn't have the vision to pick out passes others don't see
None of the above is drasitcally going to change either. He's likely one of those players that have reached their peak early and people are frustrated with what they believe he could be rather than accepting this is simply just what he's actually capable of.