Tennis balls are kind of fibrous and may have all kinds of weird dyes or chemicals attached to them. You can use a plastic or ceramic cup, like a coffee mug--just rinse with hot water and try to pick one that hasn't seen soap or at least hasn't seen soap in a long time. Terra cotta pots are also good caves--just make sure you plug the drainage hole in the bottom because many bettas have gotten trapped trying to swim through the small hole.
You should really upgrade to a larger tank soon, honestly, I would return that little deathtrap and get your money back so theat the pet industry won't profit from these unethical containers. Until you do upgrade, you should change the water daily.
Also, you should reconsider getting a heater. There will always be unpredictable cold snaps, and you never know, you might come home one day and really want to turn on the air conditioning. It's good to have an adjustable heater for these purposes, and also if your fish ever gets sick, you will want to have the ability to adjust the temperature up or down to accommodate whatever that may be. Heaters aren't very expensive, so I hope you consider it. You should try to get one with an adjustable temperature dial and not bother with the cheap heater pads or pre-set heaters. Here is the type I use:
http://www.fosterandsmithaquatics.co...8&pcatid=11368