I don’t think that’s racist that mate.
Some people, with no malice intended, just genuinely don’t like immigration, or even something like an influx of student renters nearby if a university has grown in size. Anything that changes the demographic that someone is used to and liked will always be annoying to some people.
There might be multiple non-personal reasons why they don’t like it. It that doesn’t mean they don’t like the individuals, they just might not like the overall change in culture to their area that they’ve seen.
It might be a bit of an old fashioned way of thinking to most of us, but who are we to tell them their idea of what they want their neighbourhood to be like is wrong?
As long as they don’t make an issue of it, aren’t bell ends about it, aren’t using racist slurs towards them or acting upon their dislike; and if all they’ve done about it is move elsewhere, there’s no harm in that for me.
It’s only the same as me moving to Cornwall if I wanted to live in a quieter part of the country, and the local Cornish people having a problem with me adding to the influx of out of towners. It wouldn’t bother me that they didn’t want me there, in fact, I’d perfectly understand it.
It’s just old fashioned and that sort of thing will eventually disappear after a few more generations.