If you get the walmart kit you spend 30 on the 5 gallon tank and get a filter and a light. +12 for a heater and lets say 20 for gravel, plants, decor, your only like 10 above your budget.. and thats a conservative estimate.
Now I'm gonna tell you what I would do
-Walmart kit $30
-Live Plants $20-30 (enough to give good cover)
-Driftwood $5-10 depending on size (I'd go with one bigish centerpiece)
-One Male or Female ~$4
-Liquid test kit with Ammonia, Nitrite, and Nitrate tests at least $30
--- TOTAL: $130.. assuming you go max instead of min on my estimate.
Thats not covering food or aquarium salt.. but thats like another $5. The reason I'm telling you all this is that if you go all out in one foul swoop or just go little by little over a few months your probably going to end up spending about that much. The reason I say live plants versus silk is they cost less and if you do a little care they will grow like crazy they are really beneficial to your tank as well. You could nix the plants and driftwood and go with about $15 worth of silk plants and $5-15 of caves and other resin ornaments but you won't save that much.
I highly suggest you cycle. Fishless cycles are the easiest but you have to wait to get your fish (which is ok) you can also do a fish in cycle but it is VERY involved and you have to be willing to do testing and water changes daily. Please invest in a liquid test kit.. they are high priced but SO worth it. For example I have cycled 4 tanks (testing every day during each cycle), done weekly testing (2x a week) for the past 6 months and I STILL have tests left in that kit. It is definitely worth the money.