Sunni believe that the caliph should be chosen by consensus (as Abu Bakr, Muhammad's father-in-law, was), while Shi'ah believe that only the family of Muhammad should be caliph.
There's several other differences between the sects, but that's the major one, and the reason the two even developed.