Responses

Both can be correct, but calling someone your "brother in law's brother" is a more complicated version of saying that you're not very close. It's still his brother in law since he's the brother of the sisters husband :) (I think, I haven't actually read this and just looked at a few pages to see what the family dynamic was haha)
Wait should not it be his brother in law's brother, or (English isn't my first Language) you can call your brother in laws brother , brother in law too ?