Awww, what lovely little guys they are! :) Congrats on the new boys! And very cute little setups you have there. ;)
As far as heaters go....really its only when you get down to something under 2 gallons that things start to get a little more difficult to heat, and anything under a gallon is just about nearly impossible(though I've heard of it done). However, provided you have a good heater of the proper wattage you can certainly heat tanks under 5 gallons....and seeing as bettas are a tropical species, unless you live in an area with a fairly warm and mild climate and your home(or room your fish are in) stays in the low/mid 80s with very few small fluctuations day-night/seasonally.....its best to invest in a heater, if only to keep the temp stable as fluctuations more then a degree or two regularly can be very stressful.
Personally, I would recommend just ordering a pair of these heaters:
http://www.amazon.com/Hydor-AHY00322...rds=Hydor+Theo
They're very reliable with fantastic reviews, and you're going to have a hard time finding a 25 watt adjustable heater(which is what you want for that size of a tank....anything larger might run the risk of over-heating/would probably be too bulky to fit and you want to stay clear of the pad/present heaters as those are unreliable and often over/under heat your tank)in stores sadly.
Of course, as already mentioned, always get a thermometer to monitor temp. ;)
And about the food.....most all bettas prefer frozen foods like bloodworms over most other foods, lol, but sadly bloodworms do not meet the full nutritional needs of a betta and feeding only them as a staple diet, in the long run, will leave your bettas malnourished. They're a bit like treats or biscuits for dogs....wonderful on occasion, but you need a good high quality kibble(or in this case with bettas, pellets)in order to provide them with the nutrition they need to thrive and be healthy. ;)
I would very highly suggest picking up a good high quality pellet like New Life Spectrum(found at Petco) Betta/Small Fish formula to provide for their staple diet.
Once you have a good pellet, I'd suggest what I like to call the 'tough love' approach. This is where you offer them nothing but the pellets, removing any uneaten food after a few minutes, for as long as it takes for them to eat them. This can take anywhere from days, to weeks....but for it to work you must be persistent, patient, and strong...and remember its for their own good. Don't worry about your betta starving....that can take at least two weeks, probably more, and since you are offering food they will eat when they are hungry enough.
Hopefully they'll take to NLS quickly enough....it has garlic in it already, which helps promote appetite. ;)
Good luck!