The reason it happened is because a group or groups of people demonised and persecuted another group or groups of people, something that has happened many times in history. Often it has a religious or ethnic basis - non-Jews v Jews, Catholic v Protestant, Sunni v Shia, tribe against tribe - but the common thread is that involves people who aren't part of what we could call the "mainstream", who are different in some way. Who weren't "one of us".
You're right that it wasn't a simplistic event but it started with the need for a scapegoat and the use of increasingly virulent language, via the media, to start to dehumanise that scapegoat. The US Holocaust Museum is particularly good on that part of it. People just assume it started with places like Auschwitz and Ken Livingstone perpetuated that with his ridiculous and insulting comments about Hitler suddenly 'going mad'. I'd recommend the late David Cesarani's brilliantly researched and written book 'Final Solution'.
And it's equally facile (but quite convenient for you) to blame it solely on 'fascists'. There are people on the left who deny it ever happened and who think British Jews are just Israeli propaganda shills. You & me have had this discussion and I've no wish to go over old ground but we had the former leader of the Labour Party - not someone you'd describe as a 'fascist' - describe British Jews (except he used the term 'Zionists') as having no sense of English irony, thereby making the 'not one of us' point quite openly. There are people who called for one of the oldest affiliated groups in the Labour to be kicked out because "they're not one of us" and "they're just a fifth column for a foreign power".
And that's how it starts, with people applauding and defending it. It's the pattern every single time it happens, with people turning their backs on, then actively turning on, their former friends & neighbours. The wilful blindness of some on here to this is empirical proof of how these things start, along with the more extreme view that would see Israel as a wholly malevolent presence that needs to be wiped out. And that's just on this, supposedly well-moderated forum.
And while your desire for a Holocaust Memorial is undoubtedly sincere & well meant, I think such a thing is the height of hypocrisy, as the UK (along with most other countries) did everything it could to turn a blind eye to the situation and kept the borders of the Palestine Mandate firmly shut even after Kristallnacht in 1938. The USA was little better, refusing to accept the few hundred passengers on the ship St. Louis, who had applied for US visas before they sailed from Hamburg. They were forced to return to Europe and meant were subsequently murdered.
Even today, we saw Brexit fuelled by the anti-immigrant sentiment expressed by people like Farage and the Leave campaign openly taking this line before the referendum. Rwanda, Bosnia, the situation of the Rohingya in Burma, the Kurds in Iraq & Turkey, and the Uighurs in China, show that if the lessons of the Holocaust have been learned, it's probably not the lessons we thought ought to be.