(wow, you're a little younger than my lil sis!) Happy-not-quite-birthday!
Well, walmart has the tank kits (5 gallon for about thirty dollars, and 10 for about 40 dollars), which have the tank, hood, filter, and samples, all you'd need is a heater (a good one for a ten gallon tank would be around 30-35 dollars), and the gravel and decorations.
I'm pretty sure Pet Supplies "Plus" has good quality fish stuff for a bit less than petsmart and petco offer.
34 dollars obviously wouldn't be enough, but then again you're saving up. with 20 dollars a week, if you save up, you should have more than enough to get a good tank soon. Also, if it helps, you could as for fish stuff for your birthday XD.