Some kind of fish store will have them. Kanaplex is a Seachem product and can be hard to find. Maracyn Plus is easier (it's not the same as Maracyn or Maracyn II) and comes in a bottle in liquid form. Both will need to be diluted to work with the exact gallons you have in your tank. You can treat without a heater. Actually it would be preferable to start treatment first because increasing head could speed a bacterial infection along faster. I would go to yellowpages.com or find a local phone book and start calling fish/aquarium supply stores to see if you can find either med. If you can't, Furan 2 and Erythrymycin (API meds) or Maracyn has some potential to work but no nearly as much as the first two meds listed. They're a lot more commonly found though.
You need an in tank thermometer like this: http://www.drsfostersmith.com/produc...fm?pcatid=3921
You need a lid. You can make your own or the 2.5g rectangular ones at peco have glass lids you can buy for them. There may also be some 2.5-3gallon minibow kits at petco and petsmart that are pretty popular. They are more expensive and come with some extra stuff. I would not use the filter or I would put a prefilter sponge on them because they tend to be too much for bettas and some get sucked into the filter.
You also need a little place for him to hide. You can buy a little cave like these: http://www.drsfostersmith.com/produc...m?pcatid=21395
Or you can even use a coffee mug. I'd get him a new one that's never seen soap/chems and rinse it really well. You'll also want a couple soft silk plants that are tall enough to reach the top of the tank because most bettas like to sleep in those. Mine all like broad flat leaf plants the best, like these these:http://www.petmountain.com/show_product/11442-571173 http://www.amazon.com/dp/B003QCILFU/...SIN=B003QCILFU
Bettas have especially sensitive fins that can snag on a lot of plastic plants and rough ornaments. You can use nylons/pantyhose to run across/though anything and if it snags the nylons they're too rough for betta fins.
This is my personal favorite 25w heater: http://www.drsfostersmith.com/produc...m?pcatid=21322
These are some good 50w ones:http://www.drsfostersmith.com/produc...6&pcatid=23726 http://www.drsfostersmith.com/produc...2&pcatid=23952
You can put gravel in, but you don't need to. Personally, I only use gravel in larger tanks I cycle.
You can just look around at your local shop to see what you like or you can choose to order online if you see something online that stands out to you.
ETA: You can get 5 gallon standard rectangular glass tanks for about $10 but the filter and heater etc would be more. I've heard good things about this one, but no personal experience: http://www.amazon.com/Hagen-A285-Mar.../dp/B0032G8TPW
You can look around for kits at walmart, petco, petsmart etc. Sponge fitlers like these aren't very expensive: http://www.amazon.com/Hagen-A900-Sin...+sponge+filter
If you use an HOB that you might find in a kit make sure to use a prefilter sponge on the intank tube. http://www.drsfostersmith.com/produc...fm?pcatid=4182
Here you can see 2.5 and 5 gallon kits but you'll see need a heater and decor: http://www.petco.com/product/10313/A...arium-Kit.aspx
These kits are really popular. Personally, I like getting a regular standard glass aquarium and getting a glass top and my own filter and stuff like the first few links.