Responses

I can think of these as the closest to your descriptions: Imaginary Lovers (closest to your description), Motoyan-papa to Hitsuji sensei (they were not ex-lovers but they reunited and raised the kid together - MY FAVE), Seishun Catharsis (not his own kid but his sister's kid).
Please tell me if none of these is right, I'll try to find another.
I’m looking for a manga that was like this but the guy that breaks up with him to start a normal life have a kid. Then I think he breaks up with his wife or she passes away, and the lovers reunite and raise the kid....