There is a danger. The select few of “us” who are “more equal” will become “them”.
How do you think slavery sustains itself? A few slaves get to become slave-masters when the old masters die.
I guess it boils down to the age old questions of what is the value of a human life, and who gets to decide what laws we base upon the answer to the first question.