Minimum suggested tank size for mollies is 25 gallons.(30 would be better)
This is not only due to the three inches that adult mollies can reach, (Sailfin's even larger), but to help relieve aggression by providing space ,whether it be male's or females who are being the aggressor's.
Fish are also fairly good jumper's, and if tank is too small for females to find relief,, i would expect them to possibly leap from the tank at first opportunity.
If the tank is large enough ,and females aren't too small for the male to breed with,,then adding another couple female's would help occupy the male's advances.