Almost all commercially prepared wood has some chance of having seriously nasty chemicals in it, from arsenic on down. I would NOT use wood for your project.
As for waterproofing paper...The only thing I can think of is trying to laminate it...and even then I'm not sure it would work being submerged permanently. It would be cheap though, and worth an experiment. :)
I almost wonder if you could do something like shrinky dinks? Please tell me you know what these are so that I don't feel terribly old, lol! ;)
Another option is Styrofoam. It'll have to be well weighted, but it doesn't break down in the aquarium, and you can paint it with latex NON-TOXIC craft paints. After you paint them, you'll need to apply several layers of clear-coat Krylon Fusion spray paint, and then let the whole thing dry and cure really well. (All the smell should be gone), then soak a couple of days before adding to the aquarium.