Night Fox, I think you're thinking that NOT doing affirmative action is giving minorities the upper hand. It's the opposite and the articles even say this.
Affirmative action is when colleges give minorities a higher chance than majorities.
A ban on affirmative action requires that colleges DON'T discriminate based on race. Just because racism is in the opposite direction as it was does not mean it's not racism.
If it helps, whenever you read "affirmative action," pretend it says "reverse racism." Or, maybe you've already realized you're wrong, but continuing to argue so you don't injure your pride. If this is the case, stop. I've done this before and it never ends good for anyone on either side of the argument. It wastes their time, it wastes your time, and you feel like even more of an idiot when you fight for even longer.