Moon, my halfmoon betta I think is scared of the filter

It's a 10i Tetra whisper filter, with just the biobag then it starts, the filter was on the lowest setting and it was doing nothing! So I attempted to put it in the middle, and I got it in the perfect spot so it isn't so powerful, nor weak. The next day I been noticing my betta has been starting to go to the bottom of his tank and go under the plastic plant which is spikey, and purple. When I come to the tank I have to ether lead him out by making him follow my fingers so he doesn't hurt himself, or he pushes himself out to come to me. Is this normal, or do you think he's afraid of the filter? He is swimming normal, but then sometimes he just goes to the plant?
I do not have a heater, I am still waiting for my mom to go to America and pick up a filter under 10 watts for a 3 gallon :P