It's all stereotyping.
An idiot comes on and commences general bullstuff. Turns out, his ID is in the 4000-9000 area.
Another idiot comes on and heckles the hell out of everyone. What do you know? His ID's high too.
After 20 of these incidents, someone starts to think "all high IDs are idiots" and spreads his opinions to others. Gullible people agree with his immature idea, and spread it to others, who also agree.
The reason a lot (not all) of the high IDs are handicaps is that it's mostly children who've just found out about the game, get on, and want to do everything at once. They get in people's way (or, in general, just troll) which results in anger from other players.
TL;DR: Stereotyping is bad.
