It really depends on where you're located. For heaters and filters, I highly recommend Amazon, but don't cheap out on a heater as this will be the one thing where if it breaks/malfunctions, you're whole tank could be killed (frozen, cooked or even electrocuted!). For filters, I'm a fan of the Marina slim filters, for a 10gal, I'd recommend the S20, very quiet, works well & very cheap (I have the S10 in my 5 gal), oh! and the flow's completely adjustable so there's no need to baffle. Many people also like sponge filters as they can be cheap & effective, however, I don't like the loud hum that the air pumps seem to make (my tank's in my room, so this is important) and I found that the slims were actually cheaper than any air pumps I could find (at least the slims were cheaper on amazon than any I found on Amazon or in pet stores, about $15USD).
I'm sure you know about decor, maybe some plants (I'm a live plant fan) and food, but you want to get a water dechlorinator that removes chlorine, chloramine and heavy metals. I suggest Sechem Prime as it's cheap, it works FABULOUSLY and you get the best bang for your buck (steer clear of the Betta-specific stuff... it's just watered-down regular stuff so you use about 10 times more). You'll also want a water testing kit of some kind. The liquid ones are the most reliable, such as the API Master Test Kit (these are quite expensive, but they work out to less that $0.50USD/test and your water parameters are going to be the lifeblood of your tank).
For tanks, the best deal is when petco in the US has their $1/gal sale where you can snag a 10G for about $10. You can also try thrift stores, local classifieds or even craigslist to see if you can find a good cheap tank. Just be sure to test anything used with the 24hr leak test outdoors/in a bathtub before using it.
& that's my 29cents' worth lol