I hate to say it but u are totally enabling his behavior. U let his GF move in? Are u insane?? I would have sent her to a shelter - no way would I let the OW move into my territory. Once u did that u lost all credibility in regards to setting ultimatums.
He is going to dance this dance as long as u let him. He moves in; he moves out; he moves in; he moves out. He KNOWS u will let him come back everytime he cheats, so why in the world should he stop? I understand u want to trust him but let's face it - u can't. After 10 years, u can't, period. U gotta think about the children involved. They see and understand a lot more than we think, and u are sending the message to your kids that this behavior is ok. I know u don't want that for them, right?
My heart aches for u because your post screams low self-esteem, and Dude is beating u down further and further. Please don't invest another 10 years into this man - he hasn't cared enough about u and your children in all this time I doubt he will start now. Anytime u need support, u know this board is here. God watch over u.