Its probably his personality or the fact that he doesnt know you well enough yet. How did he reacted when he first saw when you were looking at him before purchase? Some bettas have an affinity for certain people.
Those fish are clever. When he sees me and my father always 'dances' and comes on top despite the fact that only my father feeds him. Whereas he promptly ignores my mother altogether despite been around the tank most of the time.
What type of food do you give him? The little guys are selective eaters. Mine was given pellets and he promptly spit them out and turned to face us with a disgusted look. Then swam around the plants for half an hour or so not even touching the blood worms thrown at his tank, an act i would guess to be out of spite. He ate the blood worms after his ego was satisfied worrying about him.
It's so funny; it's like he knows when I've left for work or go to sleep.
Well the fact that he pays attention to your daily routine means that he pays attention to you. Thats a good sign. He either prefers seclusion while eating or the food is not the best for his tastes and he doesnt want to encourage you keeping throwing the same at him.