Like you, the MC, her brother, and Ares's group don't know. From the brother and Ares's perspective, they were just practicing swordsmanship before all these people started coming out to their area while from the MC's perspective, she was looking all over for Elise before deciding to ask her brother if she'd seen him, being confused and surprised by the sheer amount of people.
The 3 of them know something is wrong, especially by all the nobles claiming this to be a coincidence and that Edith was just out for a night stroll. As the MC said, it was a trap. Notice how Edith didn't show up and bring poor Elise out until after the MC showed up from her frantic search to ask her brother if he'd seen her maid.
Ohhh tenks man, I think what confused me so much was that it was suddenly night time (so I guess they did sword training for hours)
The artist was trying to represent the passage of time with the MC searching the entire palace for Elise after her outing with Lancelot before you see the nightfall.
I thoguht I accidentally skipped a chap, liek why was everybody suddenly outside?? What lead to that?