Just change as much as you can, and let another jug to sit out and when it ready change it again.
And sorry i really not sure how the sponge filter is working, but for now as you having the problem ,and doing full water changes just take the filter out . Lets figure out what is bothering him first . You really don't need that filter with all those water changes.
Ok, so i did everything, i prepared a new water to sit up, i watched him for a while, he was swimming for some time, then lying at the bottom, then again swimming and now he is resting on the top...What should be my next step, if he doesn't get any better? I will watch him tonight....And what to do with a filter? Does my tank even need a filter?
I don't think your sponge filter is the issue, I think its water quality. A sponge filter has very little water movement or typically its very little movement. The filter is up to you, you've already started cycling the tank, leaving the filter off you said didn't make any difference in his behavior. I strongly recommend a water test kit to see what your ammonia, nitrite & nitrates levels are especially if you choose to continue with the filter.
I find that betta fish who are trying to avoid the current will close up their fins to avoid being dragged around like a flag in the wind. This does not necessarily mean they are unhappy, just some physical adaptations to the new conditions. Is he still eating well?
I can't say I like sponge filters very much... I tried a sponge filter in my 2.5 gallon before, it was nothing but violent XD I was using a whisper10 airpump with an adjustable flow valve
Yes he is eating still....but it seems he is getting worse....Lying on the bottom, when i call him, he reacts after few seconds, last time i thought he was dead..:( Then he swims really fast through the tank, and goes back again on the bottom...I think he is dying