I drum on desks constantly, but I don't like drum kits. I tried playing bodhran but my hands swell up every time I do, even though I'm pretty good at it.
Anyway, go ahead. Just the fact that you drum on desks doesn't mean that you're more or less suited to a certain instrument, percussive or not.