It may have been that one SMF bug I used to have when I had DSL.
It would say that I was banned for "Trolling/handicap" but when I went on a proxy (it wasn't an account ban, it was an IP ban apparently), and PMed him about it, this was his response:
The ip 65.95.166.127 does not appear to be banned.
So yeah. It was a bug. Happened multiple times. He's the only person that can ban as well, so it's not like someone else was adding and removing it.
It would also happen when he's not even online.