Make sure the bag won't collapse, and wouldn't blow in it otherwise he's just breathing in carbon monoxide... it's wiser and safer to use the cup they came in with the lid or a tupperware container. (You can get a replacement cup at a Petsmart/Petco- you will need one for the water changes anyways).
one thing to be aware of is if your water treatment covers more than just Chlorine and Cloramine. They make Betta-specific treatments that also address hard water and other concerns.
Other than that, take a ziplock or similar sandwich bag, fill it with your current water and betta, seal it and let it float in the new tank for 15 min (i usually wait a 1/2 hour or more) for him to get Acclimated to the temp and new surroundings, then open the bag and let him swim out.
If you ever add another/ more fish, don't let the bag water enter the tank because it can contaminate your tank, but this is okay because he's the only fish and he's already been in that water
Normally just floating them in the cup they came in (or tupperware, whatever you have with a lid on it that can float in the tank) for 30 mins works. Every 5 to 10 mins remove a little bit of the water from the cup and add in tank water. You DON'T want to not acclimate him to the chemistry of the water- very high risk of shock and death.... ..
You don't need and really want to play with the pH, hardness of the water- what you want is just any water conditioner (Prime is the best brand) that removes chlorine, nitrate/ites and if you can find one that binds metals, even better.
hold off on the shrimp for a month or so otherwise there won't be enough for them to eat.
Spring water is pretty much the same stuff that comes out of your tap, they just add a couple chemicals like Chlorine to keep it clean and uncontaminated as it runs though the pipes to get to your tap, and Floride for our teeth. I've actually seen this in a plant where the water comes up from an artesian spring, and then went either left to be bottled, or right to become city water.
No on Spring water (sorry EvilVOG)... spring, bottled, filtered, distilled aren't healthy for them- they don't have the nutrients and minerals that is needed to keep them healthy. The filtration process removes them (even the "mineral water"). Tap water/well water is pretty much the only water one should be using with the bettas.
Wait until your tank is fully cycled before adding in any shrimp. (You should as it is for the betta- but they are pretty hardy and with the appropriate water changes and continuous testing the water it should be ok).
I'm surprised he lasted 4 years with no water conditioner- normally after only a short time they start to have ammonia/nitrate poisoning and burns, even with spring water- just goes to show you were doing the correct water changes

Wish you luck- get the heater, baffle the filter, 30-50% water changes a week with weekly to bi-weekly siphoning of the substrate, use water conditioner and he should be fine.