Keep a belly band on at all times in the house to catch any urine.  Keep him on leash at all times when in the house so you can give the AH! AH! when the leg starts to lift.  You can take the belly band off when he's in the crate, but suggest using a VariKennel type so that urine stays in the crate if he also marks in the crate (wash the crate out every time he urinates in it and use Nature's Miracle).

After three days or so, he will have sniffed the entire house, so you can start letting him off leash, but still under observation and still in the belly band.  It might also help to spray all surfaces with Nature's Miracle or other similar product if you had a previous dog, just in case your previous dog left a few marks you don't know
about.  If you want to invest in a black light you can tell for sure.  If you can catch him every time for a week, you should be home free.

Just a note...if you have a dog door, you need to keep it closed if the dog is loose in the house with the belly band on.