First I would read all about the nitrogen cycle and betta care. Then I would go out and buy a 5.5 gal. tank for about $6, a heater for about $10, and an azoo palm filter or other small filter for about $8. Those are really the basics. You can put whatever kind of light that you want, any regular light from home depot will do, and a tight fitting lid (bettas jump!). As for decorations, all you really need is some smooth gravel, a few silk plants, and a smooth cave of some sort. I said that everything should be smooth because bettas are known for ripping their fins on things. You will also need a water conditioner, and an ammonia, nitrite, and nitrate liquid test kits (so you can monitor your cycle). Yes, its costly at first, but after that you won't really need anything for a while.
You can also check craigslist, there are usually some great deals there. :D