Bettas are ok when they stop moving. They like to just hang out on large leaves without moving for a while, it's natural for them to do that. As long as you see his gills opening and closing he's fine. If he's lethargic for more than week and not eating, then maybe start worrying and find out if your water parameters are ok (temperature, pH, ammonia, nitrites, nitrate).
As he is your first fish, did you make sure to put conditioner in the water as well? Also make sure to use the betta food pellets, not the flakes.
Give us an update in a few days and let us know how he's doing.