Hey there, a ten gallon for your Betta is great! He'll have plenty of room to swim in! It'll be a big upgrade from one to ten gallons, so he might take a few days to adjust. I'll recommend to you a heater (Elite Submersible 50w heater), and a filter (AquaClear 20) with an adjustable flow, Bettas don't like strong currents.
If you're wondering about companions for your Betta, you can visit this sub-forum
and make another thread to see what you can stock in that ten gallon. Also you can go ahead and use this app
to see what you can stock with your Betta~
Decorations you can put many things in it, if you're worried about a decoration/fake plant that might tear or rip your Betta's fin an easy test is to get some pantyhose, run it over the item and it snags or tears the pantyhose it'll probably snag/tear your Betta's fins too! For cave anything simple is fine, a simple pot or a coffee mug whichever you prefer.
If you choose to go the live plant routes, refer to this thread
, it has tons of information about live plants and how to care for them. Although since it seems you're on a budget you might go the alternative routes.
Remember to get water conditioner, I recommend a big jug of water conditioner not one of those small dinky Betta water treatment products, you'll need a lot of water conditioner. Also a variety of food, flakes, pellets, frozen bloodworms, freeze dried products. A diet with variety can make your Betta happier, it's boring to eat the same food every single day.
Be sure to carry some emergency item/medication if your Betta gets sick, it's handy so you don't have to run out of the house just to get some Aqaurium Salt, and Epsom Salt, also keep your small one gallon bowl. It can act as an quarantine tank.
That's pretty much it for caring for a Betta. Don't hesitate to ask us anymore questions, we're always happy to help. Also welcome to the forums~