I only ban when they start acting in a derogatory manner and keep pestering me when I am building.
Unfortunately that happens 9/10 times, so I ban them for a day with the reason: "Bye!"
Barnabas was feeling nice one day and let a guy load something, it ended up crashing his server. Never again we vowed. : /