The Olympics are watched worldwide, even hockey. In fact, that was probably one of the most watched events at that time because it was the Soviets facing off against the Americans in hockey, the ultimate team sport.
To add onto this, for a lot of people, even nowadays, hockey is the only event they watch in the winter Olympics. There isn't really another event in the winter Olympics that a large group of people focuses on. You don't see tons of people tuning in to the Olympics to just watch the luge.