Time isn't relevant in the Zelda universe, the moons destruction would wipe the entire population out no matter the location. The moon is technically supposed to be the size of a real moon hence why on the final hours it starts burning up in the atmosphere except you can't exactly go into that scale/detail on the N64. Mind you the Termina universe has been scale right the forget down, Clock Town is supposed to be huge as well as the field which it sits in not to mention the mountains etc.
Plus the moon is created by dark magic etc, it explodes on impact suggesting its doing a little more then landing on the ground. All in all your "plot hole" is WROOOOONNNNGGGG.