Yes. You need to rinse it really well.
Put the sand in a bucket, shove your water hose into the sand, and turn it on. Swish it around a lot, and let the bucket overflow for a while. Once the water runs clear, even with a swish, it's rinsed well enough. What happens is the smaller particles will float in the running water, up out of the bucket, while the larger ones will not.
Pool filter sand is a better option, and sometimes even cheaper. Because it's filter sand, it barely needs any rinsing.