Every two weeks is wayyy not enough for a 1 gallon.
Fish pee a lot. This produces ammonia that they have to breathe and live in, which eats away at their gills, skins and fins like acid. This in turn helps diseases like fin rot get a hold of them.
A 1 gallon tank should be getting water changes at least 3 x per week, with a good water conditioner like a drop or two of Seachem's Prime to help remove toxins.
The light might not be keeping his temp stable enough, either - have you got a thermometer, to test it for how cold it gets at night? Is the light running 24/7? If not, this could be contributing to his fin issue. If it is running 24/7 (to keep the heat more stable) that could be causing him stress.
I guess the simplest treatment I can think of is the one I already mentioned - but if you splash out on a 2.5 gallon or more, so you can fit a decent adjustable mini heater in it, and do more frequent water changes with a good water conditioner, this will probably help him heal really fast and avoid disease (I can't see fin rot on him.. )
Some soft silk plants might help him feel more secure, too.. burying themselves in rocks is sign they're too cold/too stressed.