http://dogaware.com/health/arthritis.html This may have some very helpful information.
Weight issues in dogs: Excersize may not be enough... not sure if you have your dog on a new diet for the arthritis like the Hills Prescription Diet, but you may want to consider going raw or home made - or half and half... Specifically, I would go to a grain free food if you're not comfortable with raw and portion control. This will cut out the mass amounts of carbs that are used as fillers in your average dog food and with the walks or swimming, you'll find his natural weight will come into view - he'll shed those pounds and less pressure on his joints.
Work with your vet on this though... be wary of weight loss dog foods, they have the potential to cause glycemic issues in dogs.
For the GLUCOSAMINE and FISH OIL -
http://www.zukes.com/woof/hip-action.html
Zukes products are 100% grain free. We feed the cat version of this to our cats for prevention purposes but you may find it a great delivery system for your pup.