So is he going to the toilet in his crate? If so that's going to be extremely hard to stop. As soon as you notice him getting restless, dropping his nose to sniff around or look like he is thinking of going I would put him outside and leave him there until he has gone.
Then make the biggest fuss you can over him that he has gone in the right spot. Tons and tons of praise so that he will see that eliminating outside is the right thing to do.
We use 'go to toilet' with our dogs to see if they need to go outside (they will all run to the door and sit there if they do) and then tell them to 'do wee' once they are outside. It doesn't take them too long to figure out what you are asking if you get the timing right and give a reward always for good behaviour.
If he is eliminating in the house you need to watch him like a hawk. Any actions that even hint that he is thinking of using that area as the potty means it's time to go outside.
The dog has to learn it does not get to come inside until it has gone to the toilet. Even if this takes an hour, he has to stay out there until he is done.