If he was in one of those little cups and I am sure he was it just may take awhile till he is used to swimming. I would just keep and eye on him for awhile and see how he does after a couple of hours or so. He looks healthy to me. The bubbles are just from the new water I think and you did condition the water before adding him right ?
Also do you know the temp of the water ? Bettas like warm water between 76 and 80 degrees is good. Not sure if you have a heater but if not you may want to get one soon to be sure he is warm enough.