No, because a ref can be shit and bottle every decision under the sun with the safety of knowing some prick by a monitor in a white van will buzz down to him and make everything OK.
So manage it.
Monitor it. If one particular ref is abusing the system then fuck him off.