Wait...so is Grave saying that Killian may also have memories of their past lives? (๑•...

I think he did! The thought bubbles around Killian seem to be his since he denounced seeing Aster as a rival. Seems like a few people somehow also retained memory

Potential theory i just thought is that even though they weren't close, being married he was probably still plenty exposed to her energy and time magic