He only has nights and his assistant/butler So she doesn’t come in contact with any other servants because they all run away because they’re scared of him and only the Butler would really know because we don’t see her interact with anybody else besides the Duke and the Butler
So none of his subordinates wanna look for her knowing full well what she looks like? Or even tell him her description