You could try to find something inexpensive that matches your decor like a ceramic or glass mug, cup .. even some glass candle holders work, as long as they are smooth ones. You could find something cheap in Target or Walmart.
I have done the pvc pipe + aquarium silicone + matching smooth pebbles or gravel idea that people have posted about a lot. My bettas seemed to like one of the bigger "cave/tunnels" I made, one of my guys slept in that. But if you don't feel like getting all crafty, just buy a few plain pieces of pvc pipe from Home Depot in the Plumbing aisle. They are in boxes in the beginning of the aisle, usually, and are usually around $1 to a few dollars. I've gotten some for less than a dollar, actually. I'd get a few different ones but just make sure you don't buy the very smallest ones where you think a betta could maybe get stuck. Mine have liked the "elbow" shape and also the T shaped ones.
You have to look closely but if you can make out on both sides of the photo there are black pvc pipes that I covered with black pebbles, and my substrate is dark too because I wanted it all to blend. But my betta slept in one of these, he liked smaller ones too. I'm sure he wouldn't care if I didn't cover them with anything, but if you do, you can tie java fern or anubias or moss to them if you ever go the natural planted tank route in the future. The java fern and anubias will start "holding on" to the rocks.