Yes, it will shorten their lifespan quite a bit to keep them a tiny, cold tank. Bettas are, in fact, tropical fish who need a STABLE temp of 78-82 degrees F. Cold and fluctuating temps cause lethargy, as your experiencing, and are very stressful....and when a fish is stressed, they are prone to disease. Its sounds like your guy already has finrot(search the forums and look on google for finrot, tons of info out there about it).
When properly cared for, bettas can live a good 5+ years.....when kept in a tiny, cold tank, you'd be lucky to have him around for more then a year. And I assume you don't do full water changes every day.....which is needed in such a small environment to keep the water free of ammonia from fish waste and uneaten food....which causes a great amount of stress and heath issues, and can even be deadly.
Like DQ said, we recommend at LEAST 2 gallons, which can be safely heated with a 25 watt, submersible, adjustable heater(don' forget the thermometer!)with full water changes every 2-3 days.
Hope this helps some :)