Can't we just take a John Stuart Mill approach to it and let people do what they want unless it affects us? What was he said, something like you have the freedom to swing your fist till you hit my nose...
Still works, doesn't bother me if women want to wear burkhas, hijabs anything, up to them...