Whilst I agree that the whole thing does at times get out of hand, you're not accounting for the people whose minds are configured towards one gender and their bodies the other. In other words, their bodies might be born as one gender but their minds are born another - they have no choice in the matter. It's hard for most of us to imagine what that must be like - to feel like your brain doesn't match your body - but I reckon it must be torture.
Yes, they are very much a minority as cases are rare, but for the relatively few people afflicted by it the rate of depression and suicide is awfully high. I think that the small inconvenience that the majority of us have to go through in trying to navigate things like pronouns and toilets is a small price to pay if it saves lives and makes those people happier; as long as they understand that the rest of us won't get it get right all the time.
People who object to gender on political or philosophical grounds are not the same as those who have this innate conflict that they can't help.
