The most "inexpensive" VPS hosting provider I know is 4Goodhosting.com. It places high value on support and it shows. Their Customer support is responsive and helpful solving any of my self inflicted errors. The staff are really kind and everything is fine.