well, he does not have any human attachment other than the ML. so other people like concubine or children is not as important and are disposable at his mercy. You will see later how people are easily disposable when they touch his stuff (in the previous life)
Is he jealous of his own son? The crown prince IS his son right? Or is he his brother? I'm confused