You cannot boot your computer into the VPS OS as you would do with a thin client (head less workstation). This has to do with broadcast being blocked at the router level since the VPS is located on the Internet.

You can VNC into the VPS if it has Xorg installed and a desktop manager like Gnome (geekish). Or you can install Windows and RDP to the server. You'll effectively be able to use it as a workstation. You can use storage on the VPS up to the amount of disk quota you are assigned. But don't expect the disk speed to be reasonable.

Remember that with a VPS, there are many other VPS nodes sharing the same disk. So performance will drag if you intend to use it extensively. Get a dedicated server instead. Also, you'd be limited by the bandwidth speed and transfer amount. Don't ever sign up for a provider that offers "unlimited" resources. You then know you can get your VPS suspended for resource abuse and your data lost.

I know that with Linux you can encrypt your data at the filesystem level with EncFS. For example, let's say your home directory is /home/myhome. You could password-protect the mount directory so only your passphrase can let you in.

Providers have direct access to the VPS filesystem so chances are they would be able to read files and directories but with greater difficulty if they're encrypted. Keep in mind providers should respect the privacy of their customers.

As someone that helps run a VPS and dedicated server business, privacy is a must and is protected.

I hope this answered all of your question. Let me know if you have any question or comment.

Best