I think he grew in a place with maximum danger and assassination attempts. Maybe it was a reflex. Cuz every single interaction with the ml other than that was him being extremely respectful and affectionate to the kid
But still reflex or not it wasn't even his house. You don't point a sword at anyone especially a child just because you don't know who they are. It's even weirder that he didn't immediately put down his sword once he realized that it was a kid. It wasn't down until FL rushed over. You've got to admit that's pretty messed up even if it wasn't malicious.
Anyone who would point a sword at a child in a place with hundreds servants walking around everyday. Then have the audacity to blame their mother for not watching them afterward has something weird going on in their head.