
I think it's a cultural thing, rather the culture of the writer. A lot of manga list folks in their 30 to 40s as old. It is weird. Like I'll read stories with characters in their late 20's, so a few years younger than me at 33 and they will be called old...Like if that's old what are manga characters that are in their 70 and up? walking piles of dust??
Though in comparison to his companion that is still in their teens he is old physically. But only old as a love interest. Like if a 16 year old girl was dating a man in his late 30s most would think that was odd.

***Usually in manga around 40-50 is "old man" while 60 is "gramps" .. and not to forget that most of the people calling him old man are young men or teens sounding either over familiar or rude. And not to forget that most adventures are young, adventurers around his age either retired, dead or have families ( ̄∇ ̄") so he's typically an old man to them

Considering that he is 39 in an age where the oldest person is probably 60, maybe at most 70 if they are a noble with a cushy life, pushing 40 is considered to be pretty old. Old enough to start settling down and getting ready for retirement in 10 years. A dangerous job like that would have retirement around 40 at the latest unless he got a desk job instead of a field one if he wanted to stay in the adventurer's guild.
I mean, the MC is 25. How is that old? He's barely a fully grown, independent adult by today's standards. Barely out of college. Sure, maybe in that world you become an adult at 15~16 or something, but the average lifespan (of people making it past a few years old) was still 70 years or so. So how the hell is 25 old?