Welcome to the forum!
Walmart sells a couple of good kits that are 5 gallons each and $30 or under each. They come with the filter and a light. Great deal that you won't find anywhere else. Go to walmart.com and google fish tanks and you should be able to find them.
Also you will need a heater unless your house is at a CONSTANT 80*. Bettas are tropical fish that need constant temps of 76*-82* ish, never above 86*. Walmart sells OK heaters, but they are not adjustable (being adjustable will help A LOT should he ever get sick). I recommend this heater
(apparently it comes in a package now, but everything else can be used except the bubble wand). The siphon will come in handy for sure. EDIT: This same seller sells 2 gallon tanks made by Marineland (the best tank-maker) - highly recommend these as well and you might be able to get him to combine shipping!
Do you already have food and water conditioner?