No, it's not a sport. Yes, it's demanding and it's competitive. But you're not marching and blasting music into each other's faces as opposed to football, basketball, or soccer where there is direct contact involved. It's not a sport on account of the fact that you don't directly contact opposing "players". But it's just not a sport as we see "sports" in comparison to football, soccer, etc.
Before you fire off going "omgf u retad", I did say that YES, it is PHYSICALLY DEMANDING. My school's marching band is pretty great and they're very competitive, and lots of guys in it tell me that it's no walk in the park.