It sounds to me like they may be too big or too hard for him to eat. I suggest breaking them apart if possible, or finding smaller pellets. May I ask what brand you feed? Hikari makes very small ones that I've never had a problem with in the past, though I have had other brands where the pellets were larger, and I had to soak them first and then feed, so the betta could bite it apart (or I had to manually break them as you do).