Well the pellets aren't good quality.. You should look for a good quality pellets whose first two or three ingredients are whole fish, not fish meal or wheat. He should be fed two small meals a day (how many depends on the pellet you pick up) and one fast day a week.
The blood worms I understand you are just trying to get your fish interested in eating, but they should generally be reserved for treats, as they can be compared to people eating potato chips - high fat and low nutrition. If you want to feed them regularly, you should pick one day a week and feed one worm in replace of a pellet meal. Anything freeze dried also needs to be thoroughly soaked in a separate cup/container with some tank/treated water for at least 10 minutes soften it before feeding.
Have you done a water change yet?
More info is needed: http://www.bettafish.com/showthread.php?t=49233