This is exactly what the game does. It only ghosts bricks that are within your draw distance.
except that in a majority of servers, the 250,000 bricks are condensed in one region IE a city or something. if it would ghost the area you spawn in first (ie: self-centered ghosting), you wouldn't flicker up and down in the ground and you could at least move around to pick up a weapon or select a class or something while the ghosting is going on.
if ghosting works on a row based algorithm, maybe it could start on the X/Y by you and work its work up then loop back around to finish.