I'm not from japan but I do live in asian esp south east asian. I cannot speak for the whole culture or community but I do grow up around ppl (and culture) that do this. There are school that has specific label, and even my parents prohibited me to get close with the students from some school that has bad past history, and the students pretty much act the same from outside view (speak bad words, no manner, not taking school seriously etc). So yeah it did happened in some communities in homogeneous society I think. That's why it's pretty understandable for me personally about the whole current plot of this story.

In order to understand the whole situation and especially what's Rintaro is saying about his appearance, we need to understand that Japan is a homogeneous society. It's kinda hard to wrap my head around the whole conflict between the two schools, cause for sure there are bad apples in every school but you don't demonize the rest of the students because of them. Maybe it's not so far fetched considering the Japanese societal view of being associated with something bad = you're also bad. Idk, maybe it's just the manga....