I don't want him to live like he's got a clean slate just cuz he switched sides (sexually)

I still don't want him to be happy because of all the women he devastated, lives ruined, and probably driven to suicide after he bounced. But if they keep throwing trials and tribulations his way, I can begrudgingly keep trudging through the story.