I got good gravel at the LFS, I washed it very well until nothing was floating on the surface. have never had a problem.
You can also get small pebbles instead of the big rocks. He probably is looking for a place to hide, so getting him something to hide under, like a nice cave toy might curve that behavior, but I would not risk him getting stuck.
As to the screen, I bought some soft mesh fabric at a local fabric store and installed it on some tubing with aquarium silicone. For a round tank you can use a plastic rim from some kind of large plastic tub, like the kind from a cool whip or something. Or make a rim out of plastic air tubing like I did. My sister later made me a custom made acrylic frame for my mesh and I just LOVE it. My Betta Guppy gets plenty of fresh air while staying safe in his tank. and the mesh is soft so if he does jump, he will not get hurt when he hits the screen.
His sister Bubbles had jumped clear out of her tank right in front of me ones and hit the floor from a five foot drop... I almost died! - I picked her up and put her back in before the cat got interested, but she just "hovered" for two days after and one of her fins never worked again :(
It was the one and only time she has ever done that in her whole life, but it made me very paranoid about jumpers. So i recommend you get a cover ASAP.
You can also go to a craft store like Michaels and see if they have something you can use to cut to a nice fitting and attractive cover.