Below is a list of helpful resources that I put together based on what I think you, my readers, might find useful. Please note that some of these links are affiliate links in which I will receive a tiny commission should you decide to purchase the products mentioned, however none of these links are things I haven't personally used often and recommend.
Promotional Materials and Printing:
Vistaprint (get 250 free business cards with this link only) - My number one choice for affordable promotional materials and printing. Their site is so easy to use and allows you to upload your own custom artwork or choose from pre-designed items. They literally print everything from pens to postcards and business cards.
Moo - Another great place to get business cards printed if you are looking for a higher quality product. They can cut them in different shapes and print on a super heavy stock paper.
Website, Marketing and Design:
InMotion Hosting - I've used InMotion Hosting for years for all of my website hosting needs. As a web designer, I've been through tons of web hosts and InMotion has proven time and time again to be the most reliable and affordable choice.
GoDaddy - This is where I register all of my domain names. If you need a domain name, they make it super easy to obtain one. They also offer hosting and other features, however I personally prefer to register with GoDaddy and host with InMotion Hosting (above).
Photoshop Elements - The "minimalist" version of the professional Photoshop programs. This one is the cheapest and lets you do a bunch of the same things you can achieve in Photoshop. Great for Photoshop beginners!
Photoshop Lightroom - Editing software for Photographers and photo enthusiasts. Easily tweak your digital photos and do all of your post processing quickly.
Photoshop CS6 - Photoshop is what I use daily to create everything from illustrations to edited photos.
Wordpress.org - Where you need to go to get a self-hosted Wordpress site. You'll need web hosting and a domain name and some experience with website developing.
MailChimp - This is what I use for my newsletter/subscription services. They offer a great free package, or you can upgrade if your list requires more than the basics.
DaFont - Where I get all of my free fonts from.
Flickr Creative Commons - Here is where you can get free photos for your website. Be sure to play by the rules, however. There are different types of licenses that allow you to do different things with the photos. These are outlined in the sidebar of the page linked. Read the Creative Commons website for more info.
Dreamstime - They offer a "free images" section for royalty free photo downloads. Be sure to read the license information before using the images on your website or print material.
Stock.xchng - Another leading free stock photo website from the makers of iStockPhoto.
iStockPhoto - You have to purchase the rights to use any photo on this website, however they are much better quality and there is a much larger selection.
Crestock - Probably one of the cheapest paid photo sites with the most flexible terms. You don't need to buy "credits" like you do with iStock either, you can pay directly from Paypal per image. As low as $1 for a single image.
Fabric.com - Sells a wide range of fabric, and they also ship to Canada.
Etsy - I search the Supplies section of Etsy for lots of things, fabric in particular.
Hautelook - My secret to saving tons of money is Hautelook. By far my favourite place to shop! They are a members only private sale site with new deals on fashion, home decor, kids stuff, and more every day. I wrote about it here.
Gilt - Gilt is another member's only popup sale site that is very similar to Hautelook, but a little more high end, in my opinion. They have everything from clothing to housewares to travel deals daily. If you're not a member you really should be!
eBay - I shop eBay pretty much multiple times a week. Check eBay before shopping anywhere else, I bet you can find what you are looking for cheaper than retail!