
In the novel things are mentioned in more detail, but apparently her real-life brothers are the ones who started bullying her so much that other kids thought of her as a punishing bag. When her brother beat them up - similar to when the pink-haired brother dragged that maid - their thought process was more like "how dare these nobodies look down on a member of the X family" it was more of anger due to a superiority complexion rather than because they cared about her.
Though I think the people in the game are programmed differently than her real-life brothers. The characters in the game have a different reason to treat her this was that is explained later on in the novel.
both past life and present.
It's not her fault not understanding her brother's love. It's their fault for not conveying it correctly. Even when they show concern, she still hated them because they weren't someone she could trust. That's why I'm not really sympathetic towards the brothers because I know she suffered more.