That should be fine--just make sure the pygmies have enough places to hide from the betta and get out of the light. You should also do a fishless cycle on the ten gallon before putting your fish in, especially since you want to put a group of schooling fish in. A fishless cycle will give you a much larger bacterial colony, so you could put all of your pygmies in at once.
As for your diet concerns, corydoras are omnivorous and are scavengers more than algae eaters, I would not expect them to eat algae. They will eat blanched peas and zuchini and other vegetables along with tubifex worms and any betta pellets your betta happens to miss. You should research their dietary needs more thoroughly, should be easy with just a little googling.