I have something similar ... when you are taking pictures of your betta, you just need to make sure it's in macro mode - the button looks like a little flower. On mine, it's the bottom setting on the circular button pad.

I have had similar results with and without flash. It's a pretty good camera, you'll have to figure out what works best for your individual situation as far as that goes.

For the best pictures of fish, if, say, you're selling, or you want to get a really good shot for evaluation purposes, you'll want to put the fish in a small, square glass tank. From personal experience - 1 gallon is TOO BIG. There's too much room for the fish to swim around and drive you crazy. If you can find yourself a little 1/4th gallon glass cube, that'll be good.

Put a piece of paper under the photo tank, and tape or stand one behind it. Use a desk light and illuminate the tank either from above or one side. That should give you lovely definition. Put another male in a cup next to the tank, or use a divided tank with two males in it, in order to get them to flare.

If you hold the button down halfway, it'll focus. So focus before you take your shot - try holding a pencil tip up at the distance you want it to focus - and then sit and wait until the fish decides to oblige by positioning himself where you need him.

...that part might take a while.

 

As a photography student, I think I can throw in my two cents.

First of all, I do believe that it's possible to get gorgeous pictures in the betta'a actual tank, providing you have a source of light above it. Even though my camera is awful, I still manage to get the occasional good quality photo of my fish. Obviously if the fish is for sale you would do what hrutan said, but if you're just taking pictures of your pet fish, keep it in it's tank: when photographing animals you want to make sure they're comfortable and relaxed.

If your fish is scared of the camera (my Lucifer flares at it but he doesn't hide, Medusa is curious of everything and she swims right up to it), there are ways to help it overcome that. Try placing the camera somewhere near the tank so that the fish can see it, and feed it some food - if there's a particular food that it really likes such as bloodworms, use that. Continue to do this until your fish associates the presence of the camera with feeding.

Now, for the settings.

Personally, I wouldn't use flash. I feel that flash alters the colours of the fish too much, sometimes beyond the point where it can be corrected. You will also most likely get a nasty specular reflection off the glass which will jut ruin the whole image.

If you're just getting used to the camera, keep your focus on automatic. This can be found of the side of your lens. Hold the shutter release button halfway down and wait for the camera to focus on the fish before pushing it all the way down. Once you feel more confident you can start using manual focus which gives you much better control of the camera.

Make sure that your camera is set for taking images in a RAW format. This will ensure that the images are of top quality. They will take up more memory and will be much bigger than let's say, JPG, but you can always compress them later. You can usually find this in your settings under a heading "image quality" or something similar.

Keep you depth of focus shallow if you're wanting good close-up shots. I feel that using the macro setting (little flower) is a little like cheating and doesn't always give the best results. Set your camera to manual (M) and adjust your aperture to about f2.4 or f5.6: I'm not sure how low your camera goes, but both of these settings will work just as fine.

Shutter speed should be relatively fast as fish move constantly and you want a sharp photo. 1/1000 of a second should work fine, but you have to see what your camera is capable of first.

Make sure that your images aren't over or under exposed. When looking through the viewfinder, you will see a row of vertical lines at the bottom, among other information. These lines will be either on the left or right side of a thicker line: your goal is to not have any lines on the side of this said line. You do this by adjusting the exposure with a ring (I can't remember the proper name of it) which should be located on the right hand side of the camera.

When you get your images, it's always worth to adjust them a little to make the best out of them. I'm not talking about heavy photoshopping, just simple things such as cropping, increasing the contrast and adjusting the colour temperature. If you have access to programmes that allow you to do this, great! If not, I'm sure you'll learn how to take great photos of your fish regardless of whether or not you can edit them

Edit: obviously I'm not familiar with your camera but these generally apply to all DSLRs. Also, please excuse me if some of this doesn't make sense, I'm half asleep haha.

 

Wow, thank you both so much for taking the time out to help! I will try taking everything you've written and apply it to future fishy photos. Hopefully I'll get some nice ones posted on my journal besides the ones just from my phone, lol. Again, thanks for the thoughtful help!

 

The thing about Flash, when you use it you have to figure out which White balance setting looks best. My DSLR has a "Flash" white balance option which gives me more realistic colors but on my Powershoot, I use either High Fluorescent light or Daylight to get better color. It took me a while to figure out where all these things were on my camera so it's good to look at the manual book if you have it to see where they are. On my Powershoot the options pop up when I press the "OK" button in the center, sometimes also it is "Set" and then I can scroll down and choose the things I want.

But with Flash also, NEVER shoot straight on, always always always always angle yourself, whether it is from above, below, left, or right. Also to achieve a nicer and more realistic color scheme you'd actually want to use a flash from above as well as from your camera. I use my strong LED lights on top of the tank and then my flash and that helps to illuminate the entire fish rather than just the front of the fish.

Unfortuantely with most point and shoots, you can't adjust your aperture setting (f-number, it's how open or closed the lens is, just like the pupil of your eye. It opens really wide for darkness and then closes very small for high lighting) and your shutter speed (how fast the camera "blinks"). When I used my point and shoot I set it to "Program" and use the macro setting always. I set my ISO generally low and then just "spray and pray" haha. Honestly, I'd take a good 200 pics at each sitting and just sort through to get maybe 10-15 good shots. It takes a while sometimes but try to be as patient as you can be!

Also there are way's to take photos of the fish in the tank but with keeping it at the front of the tank. We just recently had a speaker in my Fish Society on how to take good fish pictures. He used a 3 sided small glass "tank", it was about 1.5 inches wide and maybe 6 inches long, good for smaller fish but obviously you can make it bigger for bigger fish. But it had a bottom, two sides and a back because it's pressed against the glass of the tank so it doesn't need another piece of glass to shoot through

You could probably make one with acrylic too but you could go to a glass shop or Lowes/Home Depot (Some don't cut glass anymore) and ask for scraps that you could silicone together, just make sure they're of the same size is all (like the two ends would be 2 inches by say....7 inches or something tall). He would sit the "photo tank" on those clay pots and put some extra substrate up to cover the pots and viola, in tank photo tank! Granted, this doesn't always work but it's a nice option! But you can practice the same idea in a regular 1 gallon or 2.5 gallon glass tank with the smaller photo tank inside of it.

I plan to make my own photo tank soon, maybe I'll do a little tutorial on it!

And I'm not sure that the CoolPix shoots in RAW format, it's not overly necessary if you're just shooting for images here, you can use regular JPEG format which are the "smaller" formats. They aren't as sharp but they take up less space. I use JPEG for my fish pictures but generally opt for a larger size if I want to print them out nice nice and frame them for the photo contest at our Auction :-D

EDIT: WHOOPS, sorry, I didn't mean to write a novel haha!