It's not necessarily the direct effects of the referee/VAR official, but the second and third order effects such as influencing the decision making process of our players/manager or emboldening opposition players.
If one set of players knows full well that any hard & fair challenge is going to be punished as a foul they will think twice. Same the other way round. If an opposition team know they can go in hard with relative impunity our players might consider backing out of a tackle.
That can 100% effect the outcome of a game.