While i agree...
just think of all the ways we could abuse the "impeach early" thing in the future. The Media is a powerful tool and we have some scarily powerful people in currently overseeing things.
Hell it could get to the point where "If your world view doesn't line up with mine then you're automatically not qualified for anything at all."
Yea, you basically have to check everything for abuse.
A solution to this would be that it's decided by Supreme Court judges rather than the public. Technically the public could do exactly what you're talking about, except without the actual impeachment.
Though another way to fix this would just for people to express how they feel on media rather than cuss and critisize in front of their computer. Mpst Annoying Orange supporters really want him on there, so they express their feelings online, therefore making it look like there's a higher ratio between Annoying Orange and other presidential candidates.