Why would it be like this if notch started the company himself? He is the CEO, that's either someone playing a prank at mojang or something forgeted up.
Being the CEO of a business isn't a small amount of responsibility, but moreover it's not being the leader of the business either. The CEO still reports to the Board of Directors in a standard company hierarchy and I would assume Notch would be the head of that board; the CEO is the guy who takes care of the business' assets (be they in the form of employees, money or products).
At least, that's how I remember it off the top of my head. Regardless, it's not a position that gives you total and absolute power over the company.