best thing to do here is just remain calm and rational, if you start acting nervous and paranoid then people may actually think you're going to be all stabby stabby.
treat it like a joke, because that's what it is, next time it's brought up just laugh it off and call it bullstuff, because that's what it is, bullstuff.