When they're looking at your house and you see them doing so, go up to your window with binoculars and make direct eye contact with them.
Then while they're looking at you, take a painfully obvious flash photograph directed towards their house.
see their reaction.