Could you perhaps fill out this form? The more information you can give the better and easier we can understand and help out. ;)
And +1 about the decor and the filter...its really not a good idea to use anything in your tank that isn't proven to be aquarium safe. Depending on the tape, it could in fact be leeching toxins into the water that could be harming your guy, and though I imagine paper wouldn't hold up in water at all if its colored paper it could be leeching dye and other chemicals used in its making. Best remove anything tape/paper in the tank and do a full 100% water change, rinse everything well, set everything back up and add your water conditioner and then slowly acclimate(float him in his cup or a container for a while, then every five minutes or so pour a little bit of his water in the container out into a sink or other container and add a little of the fresh water in. Repeat this several times, and then slowly let him go in the tank)your guy back into the tank. Try to get as little of the 'old' water from his cup into the tank as possible.
And about the filter....its best to either leave it on or take it out all together really. If your tank is smaller then 5 gallons, I'd suggest just taking it out all together as it can often be a little more difficult to establish and maintain a stable Cycle in such a small tank, but if its over 5 gallons I'd say go ahead and leave it on, baffle it(I can elaborate if needed), and do a little research on the Nitrogen cycle for fish tanks(I can provide some links if you like).