I've had to order my pellets and a lot of my supplies online because I also only have a Walmart in my area an they obviously don't have the greatest when it comes to good care.
Petco had some good prices right now on tank kits. The 2.5 gallon mini bow kit is only $20, and a heater would only be $20 more (then add a $3 thermometer and the cost of decorations and gravel...). This would be the price for each and every new Betta you rescue. The initial cost of the other needed equipment can be quite high, but after that, they are pretty inexpensive to care for. A container of Omega One or New life Spectrum pellets would be about $5 or less and would last a while, along with a bottle of Prime conditioner for about $8 and maybe like $10 for aquarium salt and Epsom salt to keep on hand. AQ salt is important if you are rescuing fish with fin rot or something.
It's really easy to get addicted and to start spending hundreds on them in a few weeks xD I had one and after joining this site, I have two in their own five gallon tanks, and an empty 10 gallon and 4 gallon with three females currently on their way, haha. I've been here for a little over a month I think.