[spoiler alert... you have been warned]
It's similar, at least in concept, to Shion.  Twenty faces wasn't ever looking for a lover, he was looking to groom a 'perfect' sucessor to carry on his legacy.  She wasn't perfect enough, she angers easily, has a short temper, and enjoys toying with or causing pain in others.  It's obvious why he left her, he is a careful man, and a perfectionist; he figured he would start all over with Chiko, who showed potential, and by getting her at a younger age, would have a better chance of being able to groom her into being a 'perfect' sucessor to carry on his legacy.
[end spoiler]