Ali R.Rizvi's The Atheist Muslim: A Journey from Religion to Reason considers the use of the term 'Islamaphobia' in a similar manner.
In the relevant section of the book, the author - commenting on a TV debate between Ben Affleck and Sam Harris hosted by Bill Maher - notes that Harris was attempting to point out that there is a difference between criticism of Islam (an idea) and bigotry against Muslims (a people).
Rizvi clearly approves of Harris's stance and continues:
'Anti-Muslim hate is a real thing, and is no longer limited to just white supremacists, nationalists, or far-right bigots. On November 19, 2015, a fourteen-year-old schoolgirl in New York City was placed in a headlock by three boys in her class who punched her repeatedly and called her 'ISIS' while trying to yank off her hijab. Just imagining something like this happening to my own beloved niece is heartbreaking to me. As a man with a similar name, heritage, and look as the Muslims I grew up with, I am as much a potential target for these thugs as my Muslim family and friends. So let me again be clear: anti-Muslim bigotry is real, it exists and it is wrong.
This is all the more reason why umbrella terms like 'Islamaphobia' - which conflate criticism of Islam with anti-Muslim bigotry - are so sinister. Semantics matter. When legitimately criticizing illiberal elements of Islam - as we might do with any other religion or political ideology - elicits accusations of bigotry and racism, it abruptly ends an important conversation that needs to be had.'
From the wider reading that I have done, I would tend to agree with Rizvi. The term ‘Islamaphobia’ can potentially be deployed as a smokescreen to prevent a closer examination of forms of Islam that are not violent but that are still deserving of close scrutiny and criticism (e.g. for their misogyny and homophobia) and the agendas of those who subscribe to them.