This is exactly what I did.
I had a pre-built stuffty HP Compaq, bought a new video card + psu
Next thing I know I'm bottlenecking the forget out of the GPU with my CPU, end up grabbing a new CPU only to find the incompatibility with my mobo, so I buy a new mobo.
New mobo isn't compatible with my old RAM, go buy some DDR3, put it all together in my stuffty tower, the damn thing overheats.
Buy a new case, and viola, a new computer, just because I wanted a new GPU