Knowing seojin's personality, he either truly doesnt remember the kiss due to the curse or he is saying that he doesnt remember just to avoid his feelings and what would come after it.
I'm thinking this, too. If it were the latter, it would make sense; he has a habit of running away and not taking charge. It's like he's afraid of fully accepting what he feels for yoon-hee.
Knowing seojin's personality, he either truly doesnt remember the kiss due to the curse or he is saying that he doesnt remember just to avoid his feelings and what would come after it.