Some Betta's find a transfer to a large tank very stressful and somewhat of a shock to them.
Most over come this within a few days,
But there is the few who doesn't so easily, I have had some like that,
I ended up doing the downsizing and slowly acclimating them to larger sizes,
Start with a half gallon if you got an old ice cream container that's empty and clean, will work ok, Give him a few days to a week in that, once he seems settled and not stressed Then try the big tank again.. You should see a big difference in him, If not go back to a gallon size for a few days or so, then try the big tank again...