Take the time to read through this article a few times: http://www.bettafish.com/showthread.php?t=49160
It has a lot of basic information, but you will want to do as much research and reading around the forum as you can. The more you know about bettas, the less money you will waste by making poor choices and the less inconveniences you will experience down the road.
My advice to you is to not go cheap on the tank and accessories for your betta. You get what you pay for, and the smaller the container you choose the more maintenance you'll have to do and the less reliable it will be. An ideal setup would be 3 gallons or larger (the bigger the better--as long as the current isn't too strong from the filter, a 10G should be perfect), with a fluorescent light and a power filter or sponge filter. You should look for a 25 watt (50 watts for larger than 5 gallons) adjustable
heater to go along with it since non-adjustable heaters, mini heaters, and heater pads will underheat or overheat the water because they don't work on a real thermostat. You will also need to use a dechlorinating product with your tap water--I recommend Seachem Prime because it is more concentrated than similar products, so it will last longer and give you the most product for your money. When picking decorations, feel carefully for rough or sharp edges, and choose fabric plants instead of plastic. You will find that in each of the product categories I'm describing, there are cheaper alternatives, but don't be tempted by them--they are rip-offs.
You should also take the time to research the nitrogen cycle and how to cycle a filter using the fishless method. Doing this before you get your fish will save you a lot of time and energy.