The cave looks fine. I would add in your betta to your tank as soon as you can, your 5 gallon tank is going to maintain a temperature better than his cup or bag that he came in. Bettas are very hardy and can go without a heater but it is definitely needed for a happy betta. I would keep the tank insulated as best you can to prevent it from getting too warm or too cold.(wrap it in a towel, keep it in a heat controlled room with no windows) The temperature of the water is not going to kill him but the change in temperature will, so make sure the temp you achieve is steady.
As for letting the water sit, it isn't needed. Once you add in your conditioner it is ready to go, letting it sit will do nothing to benefit the water or tank.
As for letting the water sit, it isn't needed. Once you add in your conditioner it is ready to go, letting it sit will do nothing to benefit the water or tank.