Goldfish should NEVER be kept in a 2 gallon! Even as babies!
What kind of Goldfish do you have? Depending on the species, you may need 30 gallons once adult, or over 100 gallons. If it's still a baby, a 10 gallon will suffice. If you cannot provide this, please rehome your Goldie!
Goldfish have an extremely high bioload and do not ever fare well in bowls or under 5 gallons. It does sound like ammonia burns. You've never tested the water, EVER? This is very important. Pet stores will test your water, often for free. PetSmart, PetCo, etc.. bring in a plastic ziplock bag of the water in your Goldfish's tank just BEFORE you do a water change, and have it tested. It won't do you good to test right after a water change.
Also, constant 100% water changes are stressful on fish. You really need to upgrade!
I agree, your fish needs minimum 10 gallons with double filtration... I.e., 10 gallon with 20 gallon filter, 20 gallon with a 40 gallon filter, etc...
If you can post pictures and get that water tested to verify ammonia burns that'd be great. Because if it isn't, we need to still pursue this and figure out what is wrong.