The gravel thing, I am assuming, is a gravel vacuum. It is just a plastic tube attached to some narrow, flexible airline tubing. Here is an example of one.
Once you get suction going on the gravel vacuum, it will siphon out the old water of your tank. It's definitely much easier than trying to pour out a 5 gallon (that's around 40 pounds of water!), or trying to scoop out half the water with a cup. The gravel vacuum also sucks up uneaten food, fish waste, and debris from the gravel.
Here is a thread where people give different methods and tips of getting the suction going:
Until you get a filter, I'd say do two water changes per week, one 50% and one 100%. Once you have a filter, as your fish produces ammonia, your tank will cycle, meaning beneficial bacteria will begin to breed and flourish in the gravel and filter media. While your tank is cycling (about a month) I would do two 50% water changes per week. Once it is cycled and that colony of nitrifying bacteria is established, you could do one 50% water change per week.
There are many different kinds of filters you could get. Many here recommend a sponge heater, which is gentle and produces very little current (bettas don't like a lot of water movement), plus the sponge is a great place for good bacteria to grow. Here is an example of a sponge filter:
You could do an external filter like this (sort of hangs on the back of the tank, with a tube sticking down into the water--that tube draws the water into the filter):
Or an internal filter like this (the entire filter is submerged in water):
If you get an internal or external filter, you might have to baffle the outflow so the current isn't too strong for a betta. This is very easy to do, just put aquarium sponge over the output and secure it with a rubber band, to reduce the strength of the water's flow. I have used aquarium sponge (and also sometimes pantyhose) to reduce the flow of all three of my filters. You may also want to use pantyhose (pantyhose comes in surprisingly handy for aquariums lol) to cover the part of the filter that draws in water, as betta fins can sometimes get sucked in!
Whatever filter you end up getting, make sure it is the right size for your tank; the packaging should say what tank size each filter is for.
Your setup looks fantastic, the different levels provide visual interest. Your betta's colors really stand out against that blue gravel!