Maybe because he had to leave just after that, so he said sorry for leaving without any explanation? ヾ(❀╹◡╹)ﾉ~
I think the reason for the sorries was different for each of them but also misinterpreted by each of them.
Tojou meant the kiss to be a confession, though Saki, densely, didn't realize it.
"Sorry" is a common way to reject a confession. So when Saki said, 'sorry', I believe Tojou believed that Saki was rejecting his romantic interest.
Tojou's 'sorry' was therefore an apology for kissing Saki.
However, Saki wasn't rejecting Tojou, he was just embarrassed and wasn't sure what to say.
Yeah, I thought the same too. It's like he's saying sorry because he didn't mean to do that for now and he's gonna save it for later
Hmmm why did he say sorry?