It was instigated by the Tories, but I agree it is not just a Tory thing. The problem is the portrayal of it as a Tory thing and the total lack of coverage of the LEXIT position. Therefore it became a Tory thing by default rather than by design .
I would categorise it more like this now
Left<--------------------Labour--------------------------------Centre-----------------------------Tory----------------------------------->Right
LEAVE< I--------------I===============================REMAIN =========I---------------------------------------------------I>LEAVE
I like your diagram. But it's important to differentiate between what the public think, what the party members think, and what the MPs think.
In terms of the parliamentary parties, I'd broadly agree with you. I'd have said:
Left<--------------------Labour--------------------------------Centre-----------------------------Tory----------------------------------->Right
LEAVE< I--I======================================REMAIN ==============================I-----------------------I>LEAVE
There's a huge Remain majority in the HoC. All but 10 Labour MPs are Remainers, and from memory, only something like 100 Tories voted Leave in the Referendum.
But if you look at the party members, it's more like:
Left<--------------------Labour--------------------------------Centre-----------------------------Tory----------------------------------->Right
LEAVE< I---------------------------|====================REMAIN ======|-------------------------------------------------------------I>LEAVE
Which very clearly demonstrates the problem both Labour and (especially) the Tories have. The Tories are powerless to elect a moderate leader, since a huge proportion of their membership are geriatrics shouting "OUT, OUT, OUT". Their leadership election was and always was going to be won by whoever had the most strident anti-EU messaging.