Referees’ Performances | 2024/2025

[halfcenturyup]

Are you able to provide links to the analysis of officiating and VAR that you are referring to?

I am aware of PGMOL’s analysis of its own officiating and VAR outcomes that found high rates of correct decisions, and have spoken—as a data scientist with two decades in financial analytics, economic analysis, and statistical fraud detection and prevention—why that shouldn’t be used to form any real conclusions, given the source of the analysis. PGMOL will always find a body of evidence that their officials get the vast majority of decisions correct, and the way in which they defined the analysis universe was highly problematic, as is often the case with attempting to creative quantitative analysis of qualitative assessments. This is without getting in to the flaws of how the actual decisions are recorded and assessed (many decisions aren’t actually recorded because the officials decided not to act in a particular moment, and thus do not should up at all in officiating analysis).

Interestingly enough, prior to the introduction of VAR, PGMOL claimed that officials got 98% of decisions correct. Then, after VAR was implemented, they claimed that it had significantly improved the rate of correct decisions compared to pre-VAR seasons, which would, of course, be statistically highly improbable, even ignoring our ability to analyse officiating and see there is not now a greater than 98% correct decision rate. 98% itself is highly improbable given humans are making the decisions in real time (98% would make PL match officials the single most accurate group on human beings on the planet).

I am also aware of the Sky Sports and ESPN VAR analyses that are also widely accepted is quite dubious by football fans that also happen to have some statistical expertise and understanding of the methodology being used.

I am genuinely curious to dig in to the analyses you have read.

Not sure that's necessary, tbh.

I think it's clear (at least that's my feeling, which I trust more than statistics either side could produce) that referees get the vast majority of decisions "right". Whether it's 85% or 95% doesn't concern me because 80% or 90% of decisions don't really affected the game's outcome.

I think it's also clear that processes aren't robust enough to provide satisfaction to the public that the remaining decisions, that do affect the game's outcomes, are made as fairly and as consistently as possible. I am thinking quality and training of referees, the choice and use of game management techniques, increasingly complex IFAB rules and guidelines, VAR protocols and guidelines, "independent" performance reviews, communication of laws, guidelines and decisions, and the like).

This is a sport based on subjective decision making whether by officials or fans. You will never be able to convince fans of a particular club that decisions that go against them are fair and consistent. That's normal human nature. But fans should be able to watch games as a neutral and think, yes, that decision was made fairly and consistently. Are we getting closer to that with all these new rules, guidelines and technologies? I am not so sure.

Anyway, the guy at the weekend was good enough for me :)

 

[SebastianBlue]

Not sure that's necessary, tbh.

I think it's clear (at least that's my feeling, which I trust more than statistics either side could produce) that referees get the vast majority of decisions "right". Whether it's 85% or 95% doesn't concern me because 80% or 90% of decisions don't really affected the game's outcome.

I think it's also clear that processes aren't robust enough to provide satisfaction to the public that the remaining decisions, that do affect the game's outcomes, are made as fairly and as consistently as possible. I am thinking quality and training of referees, the choice and use of game management techniques, increasingly complex IFAB rules and guidelines, VAR protocols and guidelines, "independent" performance reviews, communication of laws, guidelines and decisions, and the like).

This is a sport based on subjective decision making whether by officials or fans. You will never be able to convince fans of a particular club that decisions that go against them are fair and consistent. That's normal human nature. But fans should be able to watch games as a neutral and think, yes, that decision was made fairly and consistently. Are we getting closer to that with all these new rules, guidelines and technologies? I am not so sure.

Anyway, the guy at the weekend was good enough for me :)

I think it does matter when very dubious claims of exceedingly unlikely correct decision rates of 98% prior to VAR—and even higher afterward—are made, as it undermines the credibility of PGMOL. Anyone with a basic understanding of statistics and human behaviour understand such claims are exceedingly likely to be false (or, at least based on faulty data and analysis), for many reasons, including some you have outlined.

And that plays a part in the lack of fan confidence and trust in officiating, which I agree has not been improved with the poor implementing of VAR, which I believe has had a net negative impact on the quality of officiating on the pitch.

Reasonable observers understand that humans carry with them inherent biases and cognitive limits that mean referees will not get every decision right. I don’t think any rational football expects that.

But what they do expect is that those inherent biases and cognitive limits will be mitigated to the greatest extend possible and that when supplement systems intend to aid officials on the pitch are designed and implemented, that mitigation will be further expanded and enhanced.

I don’t personally think that has taken place. And the manipulated correct decision rates (pre- and post-VAR) claimed by PGMOL are both laughable from a professional analyst point-of-view, and a hinderance to efforts of actually trying to improve match officiating.

 

[halfcenturyup]

I think it does matter when very dubious claims of exceedingly unlikely correct decision rates of 98% prior to VAR—and even higher afterward—are made, as it undermines the credibility of PGMOL. Anyone with a basic understanding of statistics and human behaviour understand such claims are exceedingly likely to be false (or, at least based on faulty data and analysis), for many reasons, including some you have outlined.

And that plays a part in the lack of fan confidence and trust in officiating, which I agree has not been improved with the poor implementing of VAR, which I believe has had a net negative impact on the quality of officiating on the pitch.

Reasonable observers understand that humans carry with them inherent biases and cognitive limits that mean referees will not get every decision right. I don’t think any rational football expects that.

But what they do expect is that those inherent biases and cognitive limits will be mitigated to the greatest extend possible and that when supplement systems intend to aid officials on the pitch are designed and implemented, that mitigation will be further expanded and enhanced.

I don’t personally think that has taken place. And the manipulated correct decision rates (pre- and post-VAR) claimed by PGMOL are both laughable from a professional analyst point-of-view, and a hinderance to efforts of actually trying to improve match officiating.

Fair enough.

Edit: I still think the guy at the weekend was fine, though (all that agreeing with each other was getting nauseating).

 

[Blue Pete]

Given the subjective nature of a lot of decisions, anyone claiming 98% correct is questionable. 98% in whose opinion?

I think Haaland was pulled back in the first minute, the ref and VAR didn't. That doesnt make them, or me right, it's an opinion.

 

[Baggins.mcfc]

Given the subjective nature of a lot of decisions, anyone claiming 98% correct is questionable. 98% in whose opinion?

I think Haaland was pulled back in the first minute, the ref and VAR didn't. That doesnt make them, or me right, it's an opinion.

For what it's worth, i agree with you and so does Haaland. So its now 3.2 in our favour. Does that make us right ?

 

[BringBackSwales]

Not sure how he thought the penalty wasn’t a foul, I would have thought the only issue was whether it was inside the box or not. To miss the foul was really poor, if he thought it was a dive then why no yellow?

 

[BringBackSwales]

Fucking scouse **** Warnock, it’s a penalty but VAR shouldn’t have got involved, it’s re-refereeing the game. Clueless

 

[citydad]

The BBC used to list the referee after the line ups on their Scores 'n Fixtures page. Then they stopped doing it but today I noticed that they had No 1 Ass Ref, No 2 Ass ref, Fourth Official, 'n VAR Wallah, but no ref's name!

Sam Allison! The first black ref since Uriah Rennie. If he's twice as good as Uriah he'll still be shit!

Yes they stopped putting the officials names on the line up page.
I sent an email in on the BBC complaints portal and they said they’d take it on board and look into it, so I was generally shocked when they reinstated it ref and Var at top and assistants at the bottom of page. When you hear about games that have gone mental or even not I like to see who’s officiating as my lad is a young ref.

 

[CrystalBayBlue]

Every set of fans in the country think their club is hard done to by the officials. Every single one. A few of those fanbases even have songs about it. Every forum I know of has this kind of thread concentrating on everything that goes wrong against them.

Most of these fans just select half a dozen examples, from the 250 decisions refs make in every game, and only look at what they think has gone against them and rarely if ever looking at what the ref’s got wrong in their team’s favour or all the things the ref’s got right… all just done to cement their confirmation biases.

Analysis on wrong decisions was done about five years ago and they found that 98% of all decisions made by refs are correct. It was done again last season and it found a similar figure of 96% correct. On average they get 5 decisions wrong a game. It happens, they’re human.

Last season they also looked at VAR and who got the rough and the smooth. It showed that Liverpool got the most number of wrong decisions go against them, yet you’d listen to City, Everton and United fans’ opinions on refs and they’d all tell you that Liverpool are favoured compared to everyone else.

Like you say, yesterday Ipswich should have had a penalty, also Lewis dived three times and wasn’t booked for any of them. Those are the sort of things Ipswich fans will pick up on while also ignoring other things that went on in their favour because their fanbase is the same as our fanbase who do the same, because all fanbases do it.

Over the years, Fulham fans have the hump about City getting all the decisions go our way: from the Sterling joke of a penalty, de Bruyne’s joke of a penalty that won us the game in injury time, Aké’s goal when Akanji was offside… but I bet if you analyse all decisions between the two teams, it won’t be as one-sided as Fulham fans think.

Amen to that. 100 percent correct. But they’re will be thousands on here you’ll never convince

 

[Trevor Morley's Tache]

Are you able to provide links to the analysis of officiating and VAR that you are referring to?

I am aware of PGMOL’s analysis of its own officiating and VAR outcomes that found very high rates of correct decisions, and have spoken—as a data scientist with two decades in financial analytics, economic analysis, and statistical fraud detection and prevention—to why that shouldn’t be used to form any real conclusions, given the source of the analysis.

PGMOL will always find a body of evidence that their officials get the vast majority of decisions correct, and the way in which they defined the analysis universe was highly problematic, as is often the case with attempting to create quantitative analysis of qualitative (subjective) assessments. This is without getting in to the flaws of how the actual decisions are recorded and assessed (many decisions aren’t actually represented in the data because the officials decided not to act in a particular moment, and thus do not should up at all in officiating analysis).

Interestingly enough, prior to the introduction of VAR, PGMOL claimed that officials got 98% of decisions correct. Then, after VAR was implemented, they claimed that it had significantly improved the rate of correct decisions compared to pre-VAR seasons, which would, of course, be statistically highly improbable (bordering impossible), even ignoring our ability to analyse officiating and see there is not now a greater than 98% correct decision rate. 98% itself is highly improbable given humans are making the decisions in real time (a 98% rate would make PL match officials the single most accurate group of human beings on the planet).

I am also aware of the Sky Sports and ESPN VAR analyses that are widely accepted as quite dubious by football fans that also happen to have some statistical expertise and understanding of the methodology being used.

I am genuinely curious to dig in to the analyses you have read.

Regarding Lewis diving three times, I don’t recall that happening. Could you provide the minutes in which you believe he dived? I would like to watch those passages to get a sense if I missed it.

As far as statistics that could help support that the referee may not have been balanced in his officiating, I think the basic possession, fouls, and cards count is helpful in providing some context (of course, it still is plagued by the aforementioned issue of non-decisions not being captured and multiple fouls in a single sequence not being fully reflected).

Ipswich: 24% possession, 15 fouls, 3 yellows
City: 76% possession, 4 fouls, 2 yellows

It tells a story, even without noting that he let a fair few challenges from Ipswich players go that probably should have been given as fouls, including the early pull back on Haaland that should have been a penalty, and only gave one foul (no card) for a 5 second sequence that saw Delap elbow Akanji’s throat, swing his arm around to forearm Manu’s chest to finish the job, and then go studs up in to Gvardiol’s shin, just below his knee. And that was only three of about 8 vet questionable challenges Delap put in on our players before he was subbed off without a yellow. Also worth noting Dias got booked for putting his arm across a player (his first foul) literally seconds after one of theirs did it and merely got a foul, and Grealish got booked for being fouled in the Ipswich box.

I personally think—having more time to ruminate on his overall performance—that he was drafted in unready (due to the injury to the initial referee in the warm up), the occasion was too much for him, and he leaned too far in to the well-known “home team officiating effect” (which sees officials often be more lenient on away teams to avoid the appearance of favouritism to the home team) rather than any conspiracies.

Only 20% of Ipswich's fouls resulted in a booking, compared to 50% of City's fouls.