Yes, gravel can 'go bad' if it's full of fish poop.
If your last fish died in the tank with that gravel, it should have been washed in boiling water and a touch of bleach (and then rinsed ten ways to sunday!) before another fish went near it.. but anyhow, giving the gravel a good rinse when you change the water is a great idea and will help keep ammonia down.
If your bamboo plant is under water, make sure none of the leaves are below the water line, they can rot can go smelly. If the stem is a funny colour, this can mean the stem itself is rotten, which might put off that smell.
A clean fish tank should not stink.. there's an 'aquarium' smell, but it shouldn't be unpleasant. Don't wait for it to 'look dirty' or smell - by that stage your fish is probably going to be feeling quite sick.
You haven't mentioned whether your fish's tail ever unclamps? That'd be very useful in figuring out what's going on for him.
Good on you for starting up on better water changes. I can tell you care about your fish. There's a lot to learn about keeping them, I know I felt very overwhelmed at first but it does smooth out after a while.