It would be overly abusable. Say, host is afk for a full day, or dedi or something. Sets default minigame. So some admin comes, gets bored, and makes a minigame. Then everyone joins it and leaves that minigame. Now the host has a stuffton of people running around building crap.
Unless I'm misunderstanding you, this sounds like a really bad idea.