His problem is the food you are feeding and the lack of water changes.
Wardley betta pellets are one of the worst on the market because of fillers and wheat. Also blood worms should only be fed no more than one once a week as a treat in replace of a pellet meal. Anything freeze dried must be thoroughly soaked in a cup with some treated/tank water for at least 10 minutes to thoroughly soak it or this also leads to constipation issues. You should look for a different pellet. The first two or three ingredients should be
whole fish, not fish meal or wheat. New Life Spectrum Betta, or Omega One Betta Buffet (pellets, not flakes) are two good ones. Feed either 3-5 the Omega one split up into twice a day feedings with one fast day a week, or 5-8 of the NLS split up into twice a day feedings with one fast day a week.
Bumping up his temperature into the 78-80 range will also help with constipation. Right now he's on the chilly end of ok.
He was paling because of water quality issues, not lack of food. Pink/red around the gills is ammonia poisoning and cannot be caused by fasting. Paling and lethargy are also signs of water quality issues.
A tank of this size should be getting two water changes a week - one 50% and one 100% which should include a thorough rinsing of the gravel. Your water change routine is not frequent enough. Make sure when you do a water change you use your in tank thermometer to match the temperature of the new water to existing water, and premix conditioner ( you can use gallon water jugs from the grocery store, rinsed thoroughly in hot water but no chems) before using it in his tank. For the 50%, you can use a turkey baster to get as much poop out of the gravel as you can plus half the water. For the 100% he should be scooped up in a plastic solo type cup (not net) and set aside while you rinse out his tank and replace the water. Then he should be floated for an hour, while adding a couple tablespoons of the new tank water to his cup every 10 minutes.
I would use epsom salts for the constipation at this point - must be pure 100% magnesium sulfate with no dyes, perfumes or any additives of any kind. I use CVS Pharmacy brand. You should predissolve it at a rate of 1 tsp per gallon and add slowly over an hour to avoid shock. You only need to readd with a water change and then only add as much as water you change. Once he is used to it you don't need to add it slowly anymore.
Report back on what the poop looks like when you see it. Chronic constipation can lead to internal infection. Hopefully it has not gotten to this point yet.
-----------
*You need to do a series of 2 or 3 50% water changes followed by a full change at this time.
*Then you need to go find some different food.
*Add the epsom salts and fast at least 3-4 days, if not a full week with the salts. After you have seen a lot of poop and the bloat is all gone you can continue to feed but stay on the epsoms for 2 weeks.
*I would bump up his temp a couple of degrees too.