I Divorced My Husband and Married A Novelist 3 Days Later. AITA?

tldr at the bottom

Hey everyone, I (23M) recently divorced my ex-husband (28M) and married the novelist (30M) who lives at the beach 3 days later.

I moved to this cute town in a valley two years ago to escape from the hustle and bustle of city life. Immediately upon arriving at this quaint town, I set off to greet all the villagers. That's when I met my ex-husband.
He was quite rude to me at first, I didn't care much for him either. But after accidentally learning about his struggle with depression and alcoholism, I started to feel sympathetic towards him.

I'd talk to him everyday and give him at least 2 gifts a week. Slowly but surely, he started to recover, even started going to therapy and cut back on the alcohol! However, soon our friendship blossomed into something more...

In the end, we decided to get married on the 7th day of Fall. We adopted a beautiful baby boy next month. Life was good. But I couldn't help but notice my ex-husband was getting duller by the day.

He quit his job at the local supermarket so that he could help out with the farm and our baby. He only went out on Mondays and Fridays. He'd always talk a about his glory days where he played rugby for his highschool team, always pondering over the 'what ifs'. I'd caught him sneaking in a few beers late at night too.

I was devastated, our marriage had changed this man for the worse.
At this point, I started spending more time away from home, guilt eating me up from the inside, completing chores for others to occupy my mind.

One fateful day, while I was fishing at the beach, I met the novelist. Standing at the pier, gazing at the horizon with his long, chestnut hair dancing in the gentle sea breeze. I was entranced for a moment. I had never paid attention to this man, his schedule rarely aligned with mine and thus I had barely any chances to ever bump into him.
I'd heard that he moved to the Valley a year before I did, we were both newcomers.

As destiny would have it, we both hit it off fairly easily. He liked lobsters a lot and I catch a few for him once a week thanks to my fishing prowess.

The best word I could use to describe this man is "romantic" . For that is what he was in every fibre of his being. Hell, he even dedicated a novel to me and read it out at the local library!
As I started spending more and more time with him, the more I noticed how dull my husband's life is. He had no ambition, no dreams, nothing, it felt like he just lived *for me*.
This novelist, however, he was different. A man who kvew his ideals and stood by them, iron willed.
One evening in spring, he invited me to a boat ride. He confessed to me, we kissed. I just felt so guilty yet so *so* conflicted.

On the 13th day of Spring, I filed for divorce. I was scared, sad and mostly bittersweet. My ex-husband was a wonderful man, willing to change and become a better version of himself, for himself and those around him. But I wasn't the right man for him, for I just stunted his emotional growth. This had to be done.

That night, I visited the local witch's house and wiped the town and my ex-husband's memories of our relationship.

The next day, I proposed to the novelist, he said yes. 3 days later, we got married. The whole town was there, so was my ex-husband. They cheered for us with joy, blissfully unaware of all that had happened. I was happy and, in a way, relieved.
But I couldn't shake that feeling of guilt, if only I could wipe my own memory too. But I think did what was best for my ex and I.

AITA?

tldr: my ex-husband looked dull and lifeless after our marriage, I fell in love with someone else, divorced him and wiped the whole town's memory of our relationship. AITA?