Er......May I ask why? He had an extramarital affair the entire time in the 1st life; he didn't live badly, he was just average and incompetent.
And then in the 2nd life, he's finally happy with the woman he truly wanted to marry, and blessed with the unborn son he never had; while having good relations with the heroine.
I think he's plenty happy???
Feel bad for him? Why? If anything Aster's mother HELPED him get the best ending he could ever wish for on the 2nd timeline...
Think about it... On the 1st timeline he already had another woman in his heart but he was too weak hearted & couldn't protect her and their child (she was pregnant) from his mother... Which made them break up & have the girl abord the baby... After that he had a marriage of convinience with Aster's mother...He was someone with an inferiority complex about his height leading female lead to wear flats...His mother kept tormenting her and getting in their family matters but he never took sides and tried to "stay neutral" which I believe is enough reason to create issues... The fact that Aster their own child hated how her mother was treated while growing up, to the point of using magic to make her go back in time and give her a 2nd chance says a lot too...We never got too much details about their marriage life anyway...Sure compared to other isekai Manhwa he is far from a "villain" but these instances, though not painful enough they can end up killing a marriage slowly...

I feel bad for her first husband, he deserves better!!
( ̄∇ ̄")