If I don't care, I say sure. If I don't want to, I say no. Here's something to remember, if its in public, there is a good chance you will never see that person ever again (unless you both go to school together) so if you are rude, they will brush it off and forget it. Its not like they are going to be all pissed off the rest of the day because they didn't get candy from a stranger.