I love my inverts; especially the Dwarf Cajun Crayfish....which I would suggest over the shrimp.
Tony: If you upgrade to a 10, you can divide it and have two Bettas.
To have shrimp or any small fish in with your Betta, it should have plenty of plants or other hiding places. Bettas are going to occasionally chase tankmates and they need a place to slide into to dodge the Betta. When it's cycled and mature, you can put 10+ shrimp in a 2.5 with your boy. I have 15 Malawa shrimp in a 2.5 with Little Willie with no problems.