It's not the fact that it has ponies, it's that you all are obsessed with it. Clydesdale's are one of my favorite animals. You don't see me going "AWW, I LOVE CLYDESDALES. THEY ARE SO CUTEEEEEE." It gives the wrong impression.
Indeed, but those who are obsessed with the show are only a minority of the fandom, and they are the loudest. You are generalizing all of us upon a small part of a fandom, the part that makes us all look bad.
In reality, the large part of us actually aren't obsessed with it, we enjoy it, that's all, it's not our life.