Thanks for the replies! Hmm, I'm not sure what type of wood it is. I have 3 different kinds that I found, outside, and boiled all of them.
One looks to be like a very hard branch.....sort of like manzanita. This one isn't rotting at all.
The next is a driftwood piece that looks like Malaysian. This also isn't rotting.
Finally, I have some assorted, smaller pieces of driftwood-looking stuff, as well as some cool vine branches I found. These pieces are the ones with the fuzz. :( And they looked so cool too. Do you think scrubbing will make it go away permanently, or will this be an ongoing problem? I just assume take them out, I suppose.
EDIT: Here's a pic of the wood:
The two small pieces, at the bottom left are rotting, and also the vines on the bottom/far right.