I would get a filter, unless you want to be doing the water changes alot. Its a lot of lifting and carrying around buckets of water. With a filter, you'd cycle it to get beneficial bacteria, which in turn, reduces the amount of water changes you need to do. Here is a helpful sticky about cycling: http://www.bettafish.com/showthread.php?t=47838 If you cycle (and its reccomended) you should get a testing kit, that can test for ammonia, nitrites, and nitrates.
If you want gravel, you need about 1 pound of gravel per gallon, so 10 pounds of gravel. And decorations! Lots o' decorations; plants, hidey holes etc.
A siphon is very helpful, its like a hose that sucks up all the water. Makes things SO much easier, I love my siphon, and can't do a water change without it!
That's all I can think of for now!
You will need a heater, a filter (You may want to watch this for a while, because it may create a miniature current, even if it says it's for your size aquarium. If it does, you may want to baffle it.), a thermometer, decorations, substrate (if you wish), a test kit (Liquid test kits are more expensive, but they work well. =]), and a siphon is very helpful. In addition to this, a special aquarium stand may be something that you want to consider. A gallon of water weighs about ten pounds. So, a ten-gallon tank would weigh about one-hundred pounds. Shocking, right? So, be sure that whatever you put your tank on can handle that weight, and I'd suggest placing the tank where you want it before beginning to fill it up with water. Lol.
Good luck! =]
A 10 gal is surely an upgrade in fishkeeping experience. All the items listed above by the others is what you'll need. Very important aspect is the weight and placement of your 10 gal as Comet1993 mentioned! As PewPewPew stated, your current heater is too small. You'll need at least a 50 watt, adjustable. Again, as Comet1993 stated, you'll want to baffle your filter too. Bettas don't generally like the water current. You can read about this in the Betta Care Section under "How to Baffle Your Filter". You will definitely need to cycle the tank first too. You can learn about this in the Betta Care Section under "Basics of an Aquarium Cycle". Moving forward you'll then need to do water changes by including vaccuming the rocks. A siphon is essential. For one fish in a 10 gal, once the tank is cycled, you'll probably be able to do 25% water changes every 2 weeks. In the beginning, just check your water parameters every few days to decipher what is best for your set up. A water test kit that checks for ammonia, nitrites and nitrates will be important for the ongoing health of your tank. The API Master Test Kit works well. Larger tanks are great. I prefer them over the small tanks/bowls. Good Luck!!
I would stick with a 50 watt adjustable heater for a 10 gallon tank. Petco has a nice one Its about 27 dollars. I would just buy one of those 10 gallon starter kits from aqueon. Comes with everything you need. (besides gravel and decorations)