Hosting
Hosting will cost you the most when starting up, unless you go with already hosted platform like WordPress.com. For my very first sites I have been using free hosting providers and I would never ever do that again. Management of the blog, performance and uptime were horrible. I have been using WPWebHost Freedom plan for a long time and I can say they are pretty good for a blog up to 50,000 unique visitors a month. For this blog I have chosen a little bit more expensive and powerful MediaTemple. For the cheaper alternative I would suggest you to go with BlueHost.
Publishing Platform
The way to go from my experience is a self-hosted WordPress. Powerful, secure, tons of articles and tutorials available as well as themes and plugins. However there are more publishing platforms so browse through and choose your favorite one. The main difference between self-hosted and publicly available platform is that on the self-hosted platform you can do whatever you like, customize it, extend its functionality and brand it to be more unique where on free services you are limited to what service offers you and if you want to get something extra, you'll need to pay.
WordPress (no hosting needed)
Tumblr (no hosting needed)
Medium (no hosting needed)
SquareSpace (no hosting needed)