Welcome to the forums! I'm brand new here, too. I don't yet have my fish but can answer a couple of your questions.
Here is an excellent step-by-step article on taking care of your new friend:
http://www.bettafish.com/showthread.php?t=64078
Note that it highly recommends that you get a heater. That is one of the most important things you can do for your fishy :)
The frequency of water changes and so on can be read in the above thread. As to how to accomplish this, you can get something called a gravel vacuum. Any aquarium store should have one. If you have a Petsmart store near you, I can recommend their Top Fin "mini" vacuum. It's not the quickest, but it will enable you to take your time sucking up the yuckies.
I don't have much experience with cleaning - only that you'll only need to do it if you have algae growing on the sides of the tank. They sell specialty algae brushes for that sort of thing - just make sure you buy the appropriate tool for your type of tank. I.e., something made to clean glass will probably scratch an acrylic tank.
The substrate you'll take care of with the vacuuming. As for the decorations, I'd imagine you would only remove them if they were covered with algae. Clean with plain water and your algae brush.
I'm not sure about removing fishies during cleaning. Hopefully someone on here can respond to that.
As for setting up the aquarium, while you can do it at the same time, I would highly recommend setting it up a day or two before hand. This will let you know if your filter works, the tank doesn't leak, and so on. It will also allow the heater to bring the water up to temperature.
If you do all this before, you won't feel rushed when bringing your new fishy home :)