That is a great setup for a single betta! Quite a few people here have it actually. Great price too.
A list for you:
Heater - Walmart sells 2-15 gallon heaters that automatically shut off at 78-79 degrees. I've had one in tanks from 2 to 10 gallons and mine has worked perfectly.
Gravel, decorations, plants
Water conditioner - I use Tetra AquaSafe sold at Walmart with no problems, people here also recommend Prime.
Net
Thermometer
Food - pellets are recommended for staple diet and freeze-dried bloodworms of brine shrimp for a weekly treat.
A gravel vacuum also comes in handy.
I think that's it.
As far as getting caves, that's great. My bettas really enjoy hiding sometimes! With plants, if you're going fake go with silk plants. Plastic plants can tear a bettas fins so you'll have to watch out for that. If you're not sure whether the plant will tear you can run pantyhose across the plant. If it snags, it will tear your bettas fins.
With getting a 5 gallon tank, I suggest you cycle the tank. It allows for good bacteria to grow in your tank to help with water quality and it will make water changes so much easier and you'll have to only do them once a week.
http://www.fishlore.com/NitrogenCycle.htm this link tell you how to do it. If you do cycle, you'll need to either get a test kit or take a water sample to a pet store every couple of days. Since you'd be cycling with the betta in it, when you get any readings of ammonia, you'll have to do a water change so it doesn't harm your fish. After 4-8 weeks of cycling, you'll only have to change 20-25% of the water a week.
*If you don't cycle, you'll have to do 25-50% changes every couple of days. See why cycling is better? lol
Where you buy your bettas from depends on you. Some peoples Walmarts are horrible with their fish, specifically their bettas, and if they do get their fish there it's only to rescue the poor things. If your Walmart takes good care of their fish, by all means go ahead and get one. It's ultimately up to you. :)
Just some tips when looking for a betta, because I remember asking this question myself, look for active ones who aren't pale, bubblenests are a great sign of a healthy betta (although harder to find them in petstore cups), and ones that don't have raggedy or torn fins. Mostly though go fo the one that you have a connection with! My first betta was an absolute ham and I loved everything about him!
Good luck and welcome!
Check out the eBook Betta Fish Care Made Easy
