You can bury him or wrap him in newspaper and put him in the trash. I bury all my dead fish under my weeping cherry tree. A female would be cute next to ol' Sparky.
The ammonia poisoning could cause him to float like that or, depending on when he died, rigor mortis could also be why he was floating like an arrow. The ammonia poisoning probably did cause the gills to be raised, though, and rigor mortis set them like that. I don't think he suffered though.
Keep up with the water changes and you should be fine.
Post pics if you get a new betta.