
I think it can seem like Seo-In is just jerking Yeongwoon around(and he is, a little), but truly he's just lashing out bc to him, Yeongwoon doesn't reciprocate his obsession. Seo-In's trauma is that his relationships with others(his mother, Yeongwoon in the past) all broke, either bc they were conditional or they left. He essentially wants an enduring relationship with Yeongwoon where the latter reciprocates his obsession.
Side note, the funny part about the others gossiping is that they aint exactly wrong. Seo-in can be kinda an asshole.