I've had to rotate and handfeed chinchillas for 8 weeks. I've also had Gerbils born..so I know a thing or two about handfeeding - lol.
It'd be good for you to get a glass dropper. You don't need to stick it in its mouth. Just drop a drop on it's bottom lip and it should suck it in. I used a mix of goats milk and water (baby rice cereal too.. But leave that out for a mouse) for my chin kits. You could use kitten milk too. You need to make it use the restroom. They can not do this on their own. Take a q tip and wet it, then rub going from top to anus (toward the tail) you should have something.
You can probably see a small white band on it's stomach if it's full. (I know this wasn't always possible on the gerbils)
Make sure you keep it warm also.
Can you please get a picture? It'd help to determine it's age.
Also see if you can find a wildlife group or vet willing to take it in if you can't. You'll need to feed every 2-3 hours through out the night too-gerbil babies wean at 4 weeks off milk then hard food the rest of the way.
With gerbils their eyes don't open until day 17 or so. Does It have ear flaps yet?
To be honest - it may not make it. They're very sensitive at this age. You have to be sure you don't get milk down the nose or it will get upper repriatory problems.
I'll look a little bit more on mice, but for the most part I hope this helps.