There is a political vacuum caused by the fact that anyone who is opposed to brexit is currently homeless politically - ignoring the Lib Dems as a credible political force. And nature abhors a vacuum.
There are undoubtedly differences between those who have left labour and those who have left the tories, but I think the differences are not as extreme as the differences between the right and left wings of both labour and the conservatives. I see little difference between a party that contains a range of opinion from Dominic Grieve to William rees Mogg, and one that contains a range of opinion from Anna Soubry to Bill Cash, and I see much less of an issue with Anna Soubry making common cause with Chukka Umuna than I do with Jeremy Corbyn making common cause with William Rees Mogg.