If you want pygmy cories, swap out that gravel and replace it with sand. That's a good idea with any cories, but with pygmies it is pretty much essential because they have the tiniest little barbels you ever did see, and they can easily be damaged on gravel. They are super cute.
I'm not a fan of platys in less than a 15 gallon because they are quite large, active fish. I would go for the tetras, though I would choose ember tetras over neons (calmer, less nippy, very peaceful and pretty - mine get on really well with my girls). Another option would be male Endlers livebearers, which are also very small and non-nippy (females can be a bit grumpy sometimes).
My stocking suggestion would be your betta, 6-8 pygmy cories and 3-4 male Endlers. :)