Rather than being selfish, i think thats his way of protecting himself. I can only imagine the silent heartbreak he went through for all those years.
