As a Jew, who has lived in parts of the country where there aren't many Jews, I am somewhat familiar with being perceived as an outsider. But I am not religious and am very assimilated, so many don't know whether I am Jewish or not.
I think the sense of "otherness" really stems from a lack of familiarity with customs - - and it just is kinda easy to understand why the customs of others seem so darned strange to us (the insiders).
Yet, when you are surrounded by the unusual custom, it seems normal.
For example, no one really questions the fact that a medieval torture device is the symbol of Christianity worldwide - - to the extent that people wear it around their neck and build huge ones on the sides of buildings. Crosses are so so prevalent that we are comfortable with them.
But if a religious Jew wears a head covering, it really marks them as different and seems very strange to those of us who do not wear yarmulkes.
I just think that lack of familiarity with other cultures and what the meanings behind their rituals and symbols really are that just feels very uncomfortable. We don't want to offend others - - by staring, by asking questions about their symbols or rituals, so instead we take the easier route of avoiding. Which can be perceived as ostracizing.
So, "other" is just a descriptor for people not taking part in the majority culture and sticking to their own.