Move on and find someone who is actually committed to you. Like others have said, it he were truly committed to you in every sense of the word he would find a solid way to be with you 100%, but has that happened?? No, you are miserable. Is this really what you want?? I doubt it. The children in did not asked to be involved in all this. I do feel sorry for them, because ultimately they lose either way and it's not fair to them to have to go through something like this. This will affect them for the rest of their lives, directly or indirectly.
Look, I am not trying to make you feel bad about yourself, quite the opposite in fact, this man is really NOT worth your time. As long as you pursue him, you will ALWAYS have problems to deal with and the problems that come up in this particular situation will probably never be easily resolved. There will always be conflict because of the type of situation it is. So move on with a man who has no strings attached. Go find a man with whom you can hold your head up high with and live with happily for the rest of your life and STOP making youself so miserable. Let it go and move on.