Don't you worry! My Fabio had a picky phase when I first got him. He would refuse to eat pellets so I tried bloodworms without realizing that those are like candy and should not be fed to them all the time. He went on strike and refused to eat anything other than bloodworms but, after about a day and a half, he ate his food when he was hungry enough.
He should be fine for three days, so don't worry. He'll probably eat before you leave. Just do NOT try to "make up" for days that you missed feeding him. Just feed him as normal, no extra portions.