If you want a cheap start and plan to upgrade later it's going to cost more. An upgrade doesn't just have to be a graphics and CPU, it can probably cost you up to more than you can by a boat load.
My advice: Do what I'm doing, upgrade things that would be necessary for an upgrade. Motherboard, CPU, Graphics, RAM and power supply. Those five are necessary for upgrades. Things like the case and the DVD drive as well as the monitor and audio device can be re-used obviously unless you want to upgrade those as well.
I spent roughly $500 and got a nice motherboard with energy saving stuff, a dual core which is over clocked to 3 GHz, a Geforce 9800GT, 4 GB of RAM and a 650 watt power supply which also saves energy.