Your problem is a lack of comprehension not racism. You can’t even grasp that it seems.
Holding a passport from a European country in itself confers no extra rights. A Russian passport is a European passport and your average Russian would have more difficulty entering the UK than Wayne. From next year a British passport will confer no extra rights as they will have been removed. The key is not European. Russia and the UK are European. They key is contracted rights and reciprocated rights.
So being European means nothing in itself. We are European and have opted out of the contract that gave us those extra rights. Your average Frenchman still retains these extra contractual rights. The only difference is the number of countries he can exercise these rights in is now 29 and not 30. We on the other hand went from 30 to 1, being the RoI. And Wayne, poor bastard, is still stuck in customs and it is no easier for him to get in the UK than it was before. It seems our sacrifice was in vain.
Countries that are close tend to have more liberal immigration between them because it is practical to do so, especially if it is a land border. FoM is simply one contractual manifestation of this. Another is more relaxed VISA requirements. If we negotiate relaxed or no VISA requirements with Spain is this racist if we do not do the same with an African or Asian country? No. It is simply putting in place a contractual and reciprocal agreement between two countries in close proximity. Australia and New Zealand have FoM between them because of location and cultural ties. African countries are establishing their version of the EU with FoM, Single Market and common currency. Are they being racist by not including the UK? No. Again shared borders, regionalisation, pooling resources including labour are the driving forces.
The elimination of borders has to start somewhere and it will only start with countries in proximity to each other. We ourselves, a union of nations, are an example of that. Establish enough blocs with FoM then you are on the way to making it more worldwide. Regressing back to borders is just that. Regression. Trying to justify this regression as ‘fighting racism’ is profoundly ignorant and has the opposite effect. It feeds racism. It promotes differences.