Hes probably a little stressed from the lack of acclimation, the trip, and likely the fluctuating temperature combined.
Whenever you do a 100% water change, and especially when you're going someplace else with a different water chemistry, you need to properly acclimate your fish to the new water by floating their cup/container for about 20-30 minutes, pouring out a little of the old water(in the sink or other empty container), then adding a little of the new, and repeating that process every 10-15 minutes or so for a few times before releasing. You may not have to be quite as diligent with regular weekly changes using the same water, but you still need to acclimate otherwise you run the risk of shocking your fish.
Algae isn't harmful, just a little unsightly. You're right, its not all that big of a deal...you can leave it in there or rinse it off, your choice.
A stable 76 is alright, but the very bare minimum. I wouldn't mess with him too much for the next couple of days. Maybe put a towel around his tank, let him settle in a bit, but I would suggest looking into a better heater, something adjustable so you can safely bump his temp up to a more ideal 78-81 degrees. I know it can be hard to find ones small enough to fit in a 1 gallon(I would also suggest an upgrade to something a little larger, that extra 1-1.5 gallons even really makes a difference and is easier to heat), but on Amazon.com there are some very nicely priced good heaters that will certainly fit. Like so:
The 50 watt Elite I've been told is fairly short, basically the same as their 25 watt(which I can't seem to find on Amazon anymore....), and it is indeed adjustable despite being advertised as present.
This is more of an 'in the near future' sort of thing rather then something you need to rush and get tomorrow to make him all better, but it will certainly improve his long-term quality of life and its something I would suggest picking up as soon as you are able.