It's a huge feature that applies to only rare occasions of the game.
And I don't just mean brick damage, because you also have to consider events.
Opening doors. Chests. Things going from transparent to non trans and vice versa. Bricks disappearing. The entire fact that bricks also have to ghost as the client approaches them, which also causes bricks to appear.
To get what you want, you'd have to disable any brick that changes datablocks (default doors and chest), a bunch of events and default functions and such, change the brick ghosting network structure, and so on, just to have baked shadows.
I'm not saying they would be bad to have, but it's no petty cost.