You should be worried. It's a product designed to make people feel better about not taking care of their fish properly. Regular and frequent water changes with non-chlorinated water are the best thing you can do for preventing fish waste building up.
Just do a daily water change for a little while. In time, the nitrifying bacteria will catch up to the bioload, and you can move to a less frequent water changing schedule in relative peace. Keep monitoring it, but don't put anything into your water to cover it up.
A little extra work for a few weeks vs. turning your little gilled friend into a science experiment...in my opnion, it's kind of a no-brainer.
I'm not proud to say that I learned to distrust the chemical "miracle-cures" and "maintenance corner-cutters" through many a disheartening experience, just to learn that the best thing I could have done is a water change! And this is why I'm downright obsessive about changing water.