I bought something called Plant Sinkers recently, though I have been using them to sink other things, ha ha. I would link the website, but I googled it just now and there are dozens of places that sell them so just google it and take your pick.
The package says "Safe and Non-toxic" so I hope it's true. . then again it also says "Made in China" where they used to paint kid toys in lead paint, so who knows.
Also, maybe you would like tying some plants to driftwood like Mopani wood that sinks and is safe. I got some at a local fish store, and some in the reptile section of Petsmart. You can use fishing line or special twist-tie aquarium plant ties.