It's a complex situation and i can agree with a lot of what you say.
As Peterson says later in that video you reference, a major thing is the perception of fairness.
Interestingly not that long ago, i think it was one of the Australian councils or government offices that introduced a 'blind' recruitment & selection policy believing it would help redress a perceived diversity imbalance. I don't know the exact mechanics that were used, but it was abandoned soon after its introduction as it was producing the exact opposite results they wanted or expected.
It shows that there isn't any easy solutions, but to get a 'fair' solution all sides need to have some input and be listened to.
One of the reasons the SJW types get a bad rep is that they show no intention of considering or listening to counterpoints, and worse will actively try to shut down any argument against themselves.