
ahh i'm so sorry i accidentally disliked your comment ╥﹏╥ but i'm pretty sure he's saying "i won't annoy you" or something like that? cause he said that there's only one person who doesn't annoy him and then says that he'll match his pace aka he'll do the same as the mc aka not annoy him?... i may be wrong but that's how i interpreted it ( ̄∇ ̄") there's probably a deeper meaning to the 'annoy me' part as well...

It kinda means I'll match your pace and follow along and slowly get closer to him as tae min was kinda doing ie getting tutored offering help with his hand so hes saying he won't push or force tae min to much yet he will wait and back off when he needs to as he did in chapter 14 at the door when tae min felt the urge to run away
http://www.mangago.zone/read-manga/geu_kkeute_issneun_geos/uu/geu_kkeute_issneun_geos-chapter-13/43/
anyone can explain what his mean? his word seem not related with the previous page