Pushing with it's side is thought of as aggression, kind of like slapping it in the face.
Those three fish in a 5 gallon most likely are feeling confined - they can feel trapped if there isn't a whole lot of room with multiple fish. Usually 5 gallons is recommended for 1 or 2 bettas (divided), rather than a betta + multiple other fish. Make sure you do your weekly water changes and possibly a secondary one mid-week to help control the water quality.. 5 gallons can really only handle so much bio load at a time.
If the betta continues to harass - chase, bump, etc - then you definitely need to either upgrade to the proper size to house that many fish, or place him in a new tank where he won't have to worry about tank mates (some bettas do not like them) nor feeling closed in with multiple fish in a smaller tank.
If you are wanting babies then upgrade the tank to at least a 10 gallon where the platies would be more comfortable, divide a section off with the premade dividers as they have tiny holes, and let the female give birth in that side and remove her to the other once done.. let the babies grow up in the divided section until they are large enough to survive with mom and dad.
Edit: saw where you said the tank was cycled and could hold the bio load - possibly.. but also keep in mind the size of the tank.. there is no 1 inch per gallon rule here, and with bettas being territorial and generally solitary fish... a lot do not want to be with others, and not in a smaller tank with them. Just keep an eye on them, if he views the other fish as not listening to his "threats" about leaving, he may get rougher :(
I know you stated it's all you can afford.. keep in mind, Walmart sells 10 gallon complete kits for $30, for when you can. Good luck with them! They are cute and it's a pretty tank :)