Maybe... Maybe he just too strict on his son and that's all.
I got the impression that it's a combination of illness (remember, she had a hard time bearing kids) and potentially guilt. Guilt because she may be the spy sent from church to monitor her and keep Shuli a silent women when it comes to political affairs. The duchess, is afterall, a perfect example of down trodden quiet women, so I get the feeling she's under the church's thumb and that her husband is just fuckig cares too much about his 'duties' do he encourages this and slaps his son to keep him obedient. But I don't get the impression that he beats up his wife - there isn't any foreshadowing for that
Nora is probs in an abusive household, the way his mother reacts makes it clear the dad isn’t as nice as he originally seemed.