I can tell you a story: I used to have Tango in my 16 gallon and I thought he was the happiest fish in the world. I ignored the fact that he harassed my corys (he didn't hurt them, just chased them) and I thought that he could never be happy in another tank. Finally I couldn't ignore the chasing any more. The corys just didn't seem happy.
I moved Tango into a 5 gallon. I was sure he'd be miserable and mad at me. He's in heaven. He's so laid back now. He just looks MUCH more content. He hunts around in there and checks things out but he also is able to lounge a lot more because there's no giant filter pushing him around.
And Whiskey is a wreck. He looks like a flag that got caught in a high wind. None of his tears will heal no matter what I do. He's got a tumor and I can't do anything about it. I'd be mortified if you guys saw him. He's so bad looking.
If your boy is happy and healthy, that's all that matters. :D