Wal-Mart sells some really great 10 gallon kits comes with tank, hood, filter and net all for 27.00, even some have a 20 gallon kit also. Get the kit with the Flourscent hood, NOT the LED - led hood is not bright enough if you ever want to have live plants.
I would order a heater online as they tend to be a much better price and have it shipped to you, however Wal-Mart does sell them. I would also order fish food online New Life Spectrum betta food is my favorite, Wal-Mart has Tetra brand and really isn't that good of quality for an everyday food.
Tan (natural) or Black sand/gravel is my favorite. If you don't plan to have live plants or cories then small Pea gravel in the garden section is a fraction of the cost of aquarium gravel.
My only plastic piece of decor is a turtle basking platform that I have covered in Moss and java ferns. Our giant betta loves it but it does take up a lot of surface area in that tank. I really want a natural looking castle in a tank eventually and make a spooky planted aqua scape around it but I tend to pull all plastic decor out for more room for plants. When looking for plastic decor make sure the holes/openings are big enough for a grown betta to get through and not get stuck and they don't have any sharp edges on them.