It says in the alternative manga title that he was kicked out
The title did say that but from what I read in chapter 1, there was no mention of him being kick out. The chapter implied that he retire voluntarily.
He assumed they didn’t want him based on one guy. He’s an idiot. If it’s not done in front of the team it doesn’t count.
The manga kind of simplified the whole situation and made it seem like Ares was the single reason why he left. In the LN, it's implied and mentioned that the other team members also didn't seem to appreciate him much and looked down on him a little. Though they didn't want him gone, unlike Ares. But yeah, the MC kind of lacks self-esteem, hence why Ares's comment about him being unneeded was the final straw for him.
In the novel, he didn't leave the hero party because he want to live a carefree but because he was kick out. He still want to fight the demon just not at the front line. However, its been quite a while since last read the novel. I might be wrong.