If you use a Windows VPS it's pretty straightforward. You get a remote desktop tool, and then you go ahead and just download and install everything like you would on your normal computer. Drawback is mainly that Windows Server costs money, and most VPS hosts will charge you more to cover that for you.
Also if you want to be able to move files directly from your computer to the VPS and back, you'll want to set up Filezilla server on it.