Let me know when any of the big social media platforms start banning specific minorities and you'll have a point.

Racists are not a protected class

What happens if people disagree on what's racist? Who decides?

The law ?

I was referring to not protected classes of people.

For example, I have the right to refuse to serve someone who has written bad checks at my establishment, for example.

Or I have the right to refuse service to someone who has caused harm to my clients.

I get it, but your wording in the OP missed that nuance. I was merely pointing out that as a society we have decided some reasons to refuse service are unacceptable. It is therefore a lot less inconceivable that other reasons might be considered unacceptable.