
He feels guilty because he took advantage of one of his moments of weakness to satisfy his need for love. It was selfish behaviour or he thinks it is because they didn't declare their love in the first place and he took the selfish decision to keep and grow up the baby by himself. I also think that he would be accepted by telling the truth but I can also understand why his guilty conscience doesn't allow him to do so particularly after all that has already happened. He feels that by not telling him the truth is continuing using him but since he has true feelings he also fears the consequences of telling the truth.
I don't really get why he's not telling him the truth wouldn't Tae be happy rather than angry?