He’s not worried about him leaving. He’s worried about him remembering. The kid was a mess and tormented by memories of what had happened. So the ex yukaza promises himself that if he remembers he’ll release him. But this doesn’t mean letting him go. If you look at the shadow under the kid during this promise, it looks like blood. He is promising to release him from the torment and this life. So essentially he’ll kill him, since the kid will probably hate him anyways
http://www.mangago.zone/read-manga/like_as_the_time_will_come_when_it_will_rain/tr/56966/page/51/
(⊙…⊙ ) Whoa... Um, I totally missed that but I totally get what you mean now, too...
Oh WOW I'm speechless(⊙…⊙ )
That Ex Yakuza is worried about the kid leaving!? .... if he gets his memory back You'll be the one to leave... THIS WORLD! He'll Kill You! Lol