There is an assumption that having the European Court would never be to our advantage. If we had a dispute with another EU state, the EC could adjudicate. If we had a dispute with Iceland or Norway, it would be the EU versus Iceland or Norway. Now we will be on our own, parties to some international fishing agreements, and trying to be admitted to other local area agreements (all of which expect cooperation between coastal states over fish stocks but none of which have any mechanism of enforcement).
And while we get exclusive access to our waters, it's still not sure how we'd expand our fleet to exploit it (given the involvement of foreign owners and foreign crews in the existing fleet).