I do have 8GB of ram and I heard bricks depend on ram.
Yes, and GPU for rendering, CPU for physics and other random calculations. The game relies on everything like all games do. From what you're telling me though, its not just bricks, it could be anything. You had shaders enabled, it could be those. You were on a multiplayer server, it could be that. You said there were vehicles, it could be that. You have mods installed, it could be those. There are literally tons of things and Im just trying to widdle down to the root cause.
If you can't figure out what's making you lag, there's really no way I try and fix it.
I can't recreate the stuff I get in single player, I should note that I have 8x AA forced and 8x anisotropic.
Then try disabling those and see if it makes a difference. Turning off anything that has changed the game from a default installations helps identify the cause.
Even so, a clean install in another location is even better because the above part doesn't diagnose things modified in the game itself. This includes anything in your cache.db, your config folders, your base folders, etc
EDIT: If you cannot recreate it in single player, figure out how to recreate it in multiplayer and start disabling stuff till the lag goes away. From what it sounds like though, its just some servers, and if other people are complaining, maybe its a problem with the server and nothing you can fix?