Before committing to partial water changes, you should really research the Nitrogen Cycle and get a water test kit so that you know what you're doing. Fish-in cycling exposes the fish to ammonia, and you don't want to do that blindly.
Until you have time to cycle the tank, you should do 100% water changes every 3-4 days. It might be prudent to do a 100% change now if it's been awhile--considering the poor guy isn't eating and probably sat in some really nasty water at the pet store.
As for a siphon, I highly recommend buying as many of your supplies online as possible. Pet stores mark up their prices ridiculously in order to cover the insane overhead of their big retail stores. I do all my shopping here: http://www.fosterandsmithaquatics.com/
Here's some items you might consider buying:
All these items are marked down around $10 or more--and shipping is a very reasonable 8.99 flat rate. So you can save a lot if you order a few things at once.