Your client only supports a limited amount of objects. Once this limit is reached, no more new objects can be received/created until previous ones are removed. This includes all mission objects, datablocks, and GUIs. (remember, I'm talking client side)
Bricks and objects that you've loaded (by being in viewing range), but then moved out of viewing range from will still remain active and thus contribute to this limit.
The reason behind infinite mining servers going above the limit is that it does not reduce the brick count when a brick is mined. Yes, the brick count needs to be semi-manually adjusted. Planting any brick through any means will automatically increase the count, however, deleting is only through certain methods. Deleting the brick will not automatically reduce the count, but doing killbrick will, along with causing chainkills.