It's the nature of live plants. Hehe. The best you can do is pinch off any dying or yellowing leaves every day when you check the tank. They might be dying off cause they're new. It happens. Then they'll grow back. But live plants generally are like any living thing. They're messy. :)
Some plants I found that don't leave a big mess? Anubias (I have lancelota and nana), Java fern (as long as you pinch off dead leaves), Marimo balls, Wendtii (just got it but so far it's doing well), and green temple (this is SO pretty but also new so we'll see). All of these are low light and I have them in my 10 gallon. I love wisteria but it's SUCH a mess. I constantly have leaves dropping off and floating around. Although, Tango uses them to make bubble nests.
Shrimp will eat debris. But your guy might eat the shrimp. Corys won't do a thing for it. Mine cruise right over debris. I don't know about Otos. I've never had them but I think they're strictly algae guys. Maybe google them.
No, you don't have to move them or take them out. Just go around. I never move mine on purpose but sometimes they come out on their own from the siphon. Gosh I'm all over your questions today. You'll think I'm stalking you! :P