Moving to a larger tank can always be stressful, and moving to one with a filter for the first time can be even moreso.
Filters in small tanks such as a 2 gallon isn't necessary. In fact, I personally prefer not to use them in tanks below 4. They can be quite strong in the current, which isn't fun for long finned fish.
To keep his water healthy without a filter you are looking at 1 50% and 1 100% water change per week with water conditioner..
It can vary on how long he unclamps his fins.. if it is stress related it could take a day or a few before he adjusts and becomes less afraid. If he wasn't ill, and wasn't like that prior to the move then I would say it is stress..
Did you add in water conditioner?
What is the temp of the tank? And is there a heater?
How did you acclimate him to his new tank? (How did you go about putting him in there.)
If you wish to use the filter, you could get an aquarium sponge, and cut it to fit inside the outflow section of the filter and use a clean/new rubber band to hold it into place. You would still need 2 weekly water changes though- 1 50% water only, and 1 50% with substrate cleaning by either using a small hose or stir the gravel and dip in a cup to remove it.