If you put DDR3 RAM with a DDR2 mobo it likely wouldn't boot at all. I don't think it would work at all if you had the wrong DDR tech.
I don't even think it would fit, haha.
I know DDR2 doesn't fit in DDR slots.
Also, ontopic, fairly sure that your PSU isn't the problem. You've probably muddled up the layout of the channels (some motherboards, especially prebuilts, are very brown town about what order you install the chips in and balance etc). Or something like that.