There are a few in the FL, but only a very few. The head of diversity at the FA has previously seemed to suggest this is mainly becvause of religious reasons. I don't really know enough about religion to see why this would be the case. Some have suggested a lot of BAME people may be put off due to the fear of racism.
When I was starting out I used to be assistant in the Wearside league. The best ref I ever worked with got promoted to the FL. However, the next season the best referee was a Korean lady who worked at Durham Uni. Every ref you spoke to had a high opinion of her. Come the end of the season she was in the bottom band for assessors, even though she was top of the clubs markings. So the old boys in blazers basically stopped her progress. Same happened the next season and she wrapped in reffing in that league. 2012 and watching the Olympics and lo and behold who was reffing the semi final of the womens tournament. Now I have never been 100% sure if it was her sex or nationality that the assessors took a dislike to but I can understand if others have had similar experiences why there is not more.