
They always talked about a wall he built around him and he doesn't let anyone else in
While the villainess is suitable for empress role he didn't choose her to love her
But for people with trust issues it's really hard to completely open up to someone and show them weakness, he isn't sure he loves her really , he knows she is all you mentioned but also, look he didn't do anything too special for a captive princess and his fiance already did all that

in chapter 41 doesn't he talk abt how he feels like she doesn't have any feelings for him? i dont blame him for putting up a wall when he doesn't think she has any feelings for him lol. in that chapter he also said that the responsibilities of being emperor are a lot, so he doesn't wanna get her involved for no reason (i think)

He want to restrain his feelings for her. He doesn't want to get much further beacuse if ever his feelings blooms into love he knew that he couldn't keep her by his side cuz our MC can't handle the intensity of magic or even a small release of mana if they ever live together our MC might die living in that country.
She’s beautiful, stunning, happy go lucky, straightforward with what she wants, WHAT ELSE IS HOLDING THIS GUY BACK? He’s worried for her safety so he’s not going to marry her (cause he has a lot of enemies)? Is that it? Usually its the female charac whos frustrating but this ‘father’ right here makes me feel like hitting my head to the wall lol