Keep in mind the CR system is set up for a party of 4. (A CR 3 monster is a decent challenge for a party of 4 level 3's.) Anyways, my point is that you don't need a lot of people. In fact, having over 5 slows down gameplay a lot... My advice would be to keep on trying to get more and that if 3 is really the best you can do, just roll with it. If there's a class that you're team really needs (Say you don't have a cleric...) then perhaps the DM can make one and play it himself as an NPC accompanying the party.