thought we knew that every reincarnation was him…..?

No whenever they tried to tell him he’d get horrible pain or it’s like it was never spoken

We the audience knew, but the main character didn't because anytime someone tried to tell him he would get massive pain in his head and his memory would reset to before the person said it.
So he never knew that he was the previous 'outsiders'.
Now that he's made a contract though, his connection to the goddess was severed and he is no longer bound to forgetting his past life or be forced to forget any information that he had a past life in this world