“Friends” that mention your ex when you are being hosted by the current wife and girlfriend should walk out the door. That is rude and they are just creating drama even as a joke.
Lyle now is clearly bothered, it is good Ash warned his friend but maybe he should have warn them before the visit. And of course Ash should have been honest with past relationships to Lyle so he didn’t have to feel this insecure even by accident.