Coconut caves are pretty popular and bettas seem to love them. I use terracotta pots myself. I cut them vertically down the middle with a wet saw and place them in the gravel, cut side down. Quinn loves his cave, although he never does more than just stick his tail in them when he sleeps.
I just had an old castle that I didn't like and put it on it's side as a cave for my girl ha. I have also seen tons of tanks with pvc pipe. My neighbor has a huge 90 gallon cichlid tank with pvc pipe. I have just a regular silk plant kind of tied to a filter that one of my boys likes to hide in.