I usually go with plastic plants. I am just pretty careful and make they aren't to pointy.
I have colored gravel in both my tanks, but if rocks going down the drain will be a problem I would go with something a bit bigger. My mom was getting annoyed at the rocks going down the drain lol
I have silk plants in one of my tanks, and they do look nice, but are a bit more expensive. And as soleilvie said a nice cave. I would go to petsmart as they have some nice bridges and small caves for not to badly priced. I found some nice bridges and a few caves for like $6 there.
Oh and pretty betta! I am sure he will love his new home!