The Mishna (87a) says that according to R' Yehuda, we do not apply the damage of embarrassment to a slave. The gemara explains that this is because the Torah uses the word "אחיו"- his brother, in reference to the one getting embarrassed.
Rashi explains that since a non-Jewish slave can not marry a Jew, he is not called your brother.
This would seem to indicate that one can only be called your brother if he can marry your sister and become your brother-in-law.
The mesoras hashas has a second explanation from Rashi, and that is that the non-Jewish slave doesn't have brotherhood
at all, even with his own kin, as we find he would be able to marry his own sister or his brother's ex-wife.
Interestingly, the difference between the first and second understanding of Rashi is that in the first explanation, he doesn't have brotherhood with Jews. In the second explanation, he doesn't have brotherhood with anyone.
Something that needs explanation is that according to the second understanding of Rashi, that he doesn't have brotherhood at all, where is there an indication from the passuk that this is what brotherhood means? If anything, the verse says that the damage was caused in an altercation between "איש ואחיו" - a man and his brother. This would seem to imply that there is a brotherhood that is necessary (and missing by a slave) between the Jew who attacked and his 'brother' that was embarrassed. Where is the implication that the person attacked must intrinsically have 'brotherhood,' thus implying an exclusion of the slave, who has no brotherhood at all, even with his own kin?
It seems that according to the second understanding, we are not learning out from the relationship implied by "איש ואחיו" - a man and his brother, but rather, simply from the fact that the person who was attacked was referred to as a 'brother,' as opposed to, say, a 'fellow' (עמיתו) or another similar term.
It is also interesting that according to the mesoras hashas, this second understanding was the one Rashi preferred. Perhaps he preferred it because the Gemara, when stating the drasha, does not say, "to the exclusion of one who has no brotherhood
with him (ie the one who attacked him)," but rather the Gemara says, "to the exclusion of one who has no brotherhood," without mentioning the one who attacked. This seems to be implying that he
intrinsically has no brotherhood, even with his own sister and brother.
It is notable that Tosfos says, in explanation of Rashi's second understanding, that we are speaking about the slave's children, that they do not have brotherhood with their siblings. This is actually indicated from Rashi himself, when he says that the source for this idea that a slave would be permitted to marry his sister is from the verse that compares a slave to a donkey. This comparison teaches us that a slave's child is like a donkey, whose offspring may be mated with one another.
This is an important point, because this verse specifically applies to a slave, and not to a regular non-Jew, who, it would seem, would not be permitted to marry his sister. The slave himself would also be like a regular non-Jew, forbidden to marry his sister, as he was the product of a relationship between non-slaves. Only a child born of a slave would be considered like the child of a donkey and not related to his siblings.