I use Visual Studio full time in work so it is an obvious choice as an IDE at home as well. As none of my hobby projects are revenue generating and are quite small, the Express (free) editions of Visual Studio are another obvious choice. I have been using the Web Developer, C#, SQL versions of Visual Studio Express and find them sufficient for my small to medium projects.
The express editions are very limiting when it comes to plugins and extensions so a lot of the tools and extras that I am used to are not available. However the only thing I ever really miss is source control. Every experienced developer knows how important it is to have their source code under some sort of source control. For now I am the only developer, but I still like to know what I have changed and be able to revert files back to a previous state. The other thing you take for granted when you have source control (in normal multi developer environment) is the automatic backup and duplication of code on to another machine (which is usually backed up off site).
In the past I have used Visual Source Safe, Perforce and Team Systems and first looked at these as a solution. Of the three, only Perforce offers a free license (for two users). I installed the Perforce server and the P4V client (which I had used before). So I added my source code directory and started coding. The P4V client was a bit awkward to use and it didn’t pick up file changes automatically. It was technically usable, but I ended up not using it properly, just doing an arbitrary weekly check-in.

Now I had some form of basic revision history and rollback facilities but I was not happy with my back up system, which was to backup every thing onto a CD tomorrow. The simple solution for this is a third party to host my source control database. There are a few websites that provided this service for free but most are only available for Open Source projects. A few commercial services do offer a limited service for free and I went with Unfuddle who provide free Subversion hosting (size limited). So I signed up, created a repository and tried a few clients.
The minute I started using the TortoiseSVN client I was very impressed. It has all the functions you would expect from a version control system (see screenshot), but as part of your normal file explorer. It picked up changes in files automatically and was a pleasure to use. Straight away I started doing more regular change related submits. And now I have peace of mind that my source code is backed up off site.