I rarely use my speakers, but even when I do, I always leave my headphones on, even though they're not playing anything. This is because nothing plays through the speakers when my headphones are plugged in. But even so, I always have to urge to wear my headphones, even if nothing is playing, simply because it helps me focus and keeps my ears warm.
Perhaps you might have some of these reasons...