Welcome to the Forum!
From what did your Goldfish die? If you don't know you will need to either sterilize the tank or let it run at least six weeks. If it is cycled and you choose to let it run for six weeks and you lose the snail, you will need to add fish food on occasion to keep the cycle going. Even at that, I would do an initial 100% water change.
Just as a precaution, do you know ppm of Ammonia, Nitrites and Nitrates? And the pH? Pet stores will usually test for free.
I would add a lot more plants because Betta are not comfortable in such an open environment. Silk seems to be kinder to Betta fins than plastic. The little black rocks in the filter are most likely carbon that needs to be changed monthly. In a five gallon you will need to do 25%-50% weekly water changes to keep parameters safe.
FWIW, I'm of the school you can never have too much filtration.

So I would leave the filtration as is. There is no reason to remove the green/yellow rocks other than someone's sense of aesthetics.They are rocks, correct? Not those gel things? The big rocks I would remove as it is hard to remove all of the gunk that they trap and this can cause parameter problems.