If the tank is less than 5 gallons, I would suggest taking out the filter as well. You can either replace it with a less dangerous, more useful filter, such as a sponge filter, or leave it out entirely and change your water change schedule accordingly.