@mangeurdenuage @kaia I don't think looks has much to do with it.
It's all about culture and one of the largest contributors to culture is how our brain works. If a japanese sees an african-japanese (there are some) at first they'll feel shocked, but after talking to that person they'll realize that it's just a japanese person. Both instantly become "color-blind". Why? Because african-japanese grow up thinking they are the same as the rest, they don't realize they are different. The local culture prevails.
Now compare that to America. Hellhole. Why? Because slave culture was never erased there. African-americans are indeed different inside, culturally.
Now, culture comes from your parents and relatives. This is when biology steps in and says "I am here". Blood is thicker than water. And looks only reinforce the resulting culture, but doesn't create it.
To me, however, the most decisive factor is behavioral patterns determined by brain features specific to race/lineage/family. An african brain does not work in the same way as an asian brain. Very different configuration. This, by logic, cannot be less of a culture reinforcing factor than just looks. It determines how people react to situations, worldviews, affinities, etc.
Race/lineage/family genetic heritage -> thinking patterns determined by brain -> experiences -> cultural heritage.
Diversity in cultural heritage is what determines social distrust. Because people begin understanding the same ideas differently. Babel tower.
Appearance becomes assimilated after few generations. See Japan for instance. They used to have entire towns populated by racially black people, hundreds of years ago. Where are they now? "It's a tan".
Culture > different looking race / time.
You can live with people that speak and think like you, no matter how they look. But you can't live with people that oppose your worldview, even when they look exactly like you.