So is cracker but that doesn't mean most whites are offended by it
It also wasn't a wartime slogan for slaughtering white people. White people aren't a minority group either, and I'd imagine it's easier to stomach slurs when you're on top. Not that it makes it okay to call people that.
Then where are you getting the "Most Japanese are offended by Jap" line
Again, it's common knowledge. I haven't read any studies, but I'd bet that my odds of pissing someone off by calling them a 'spic' or a 'monday' are pretty high.
If you can find me a large, outspoken group of Japanese people who think that the word is harmless, then I'll agree to be a bit more skeptical. You won't though because, surprise, it's still a slur!