I always look for a few things: Reliability, Support/Knowledge, Price - you get what you pay for really.

Zappie host is good cheap wordpress hosting. Generally 100% uptime... Lots of nice features like SSH.
The support is good and they always seem prepared to go out of their way to get things done.