Metals don't evaporate. Gases do.
Chlorine is a gas, so that's what evaporates when you leave water in an open-topped container with lots of surface area, with an air stone in it.
However, a lot of tap water now contains chloramine, which is a sort of chlorine that bonds to the water more thoroughly so it doesn't evaporate at all -- great for us, not so great for the fish.
Metals don't evaporate however - so conditioner to neutralise them is important.
For an emergency, I'd simply purchase an extra bottle of conditioner...