
I feel like the reason Jaeha agreed to let him go there is because Jaeha saw that Yujun dislikes his possessiveness (the "you know how much I hate people touching what's mine"). It was right after that line and the cut to Yujun's unhappy face that Jaeha gave in. I think the possessiveness reminds Yujun of that blonde hyung and he doesn't want a repeat of being fully caged and unable to escape.
Though they're both possessive of Yujun, once again, Jaeha at least tries to treat him like a human. Otherwise, Jaeha wouldn't have initially given him a normal job, kept him out the politics, and looked so worried at the end. Blonde hyung literally uses Yujun to do dirty work and other shit.