"He started it!"
Even if he did you don't need to continue it, should've explained the situation and if he still said something rude afterward, just walk away
But because you had to make a retort i suppose you don't like feeling defeated, but just consider he has to use the disabled bathroom in the first place