Politely ask him why he's following you. If you're uncomfortable with the answer, ask him to unfollow you.