Could be trying to hide anything from a weed farm to a bdsm club, they may be getting a big dog who knows, but 6 ft is ok although he should have asked your opinion at the very least, if I were in your position I'd pretty it up on my side (paint, plants,whatever) but tell him if anything like this happens again you will take action.
Hand on heart I'd be very suspicious, I'd be having a polite conversation and just mention that the police were asking about the fence, don't make a big deal it of it just throw it into a chat, it will make them rethink if they are upto something lol.