If you've ever been underwater, then you know that you can't hear what's above water very well
Though I don't know to what extend a fish hears like we do. Also, I think the glass would insulate sound.
However, if you drop something on the floor, the fish know....
I think if the sound makes the fish skittish, or puts them in a defensive position, then maybe it's too much. Again though, I would see if its just a momentary response to the new stimulus.
On a side note - something I was thinking about the other day. Dogs. So when we have company over and are chatting it up... How annoying that must be for the dogs
like being in a room full of people speaking another language, occasionally understanding a word here and there.