This is the AI tyranny they warned you about. Let's say the future is fully automated AI locking you out of things for social credit score violations. There's no due process or appeals, there's no one working in customer service. They don't really care about you individually because you're just taking up space and not generating much revenue anyway.
One of the problems with very large corporations like Google is the marginal benefit of each additional customer is negligible. They have so much money that they don't really need extra business and would rather pursue other objectives. Look at all the stuff getting cancelled lately for whatever reason. The whole thing takes place extra-judicially and is not in the monetary interest of these companies. Stakeholder capitalism if you will. I think the old profit driven system was better because at least you knew what the rules were. Now, the way these big companies make decisions is largely opaque and based on secret rules and arbitrary decisions often based on nothing but whim, or worse, a "good enough" algorithm.
Arguably, this has nothing to do with AI, but an imbalance of power. From a business perspective, it makes perfect sense to do whatever is just good enough to turn a profit with minimal effort.
As long as there is healthy competition, there is enough incentive to do better, and a customer can switch services without experiencing significant hardship. However, with a lot of the big tech companies, that is no longer the case, and once you run into a problem there is neither free-market nor regulatory mechanisms that can help any individual solve their problem.
Currently, the big tech companies can eschew responsibility by claiming the rights of private business -- having very strong autonomy over how and with whom they do business -- and despite them being de-facto public utilities at this point, the claims are made that regulation is not necessary / not allowed / not productive / harmful. Why this is accepted is puzzling to me, as it is very commonly known macro-economic theory that the usually claimed self-regulating free-market mechanisms no longer work for when large imbalances, eg. monopolies, exist. We have the government explicitly for this reason, to create a counter-balance against forces so large that any individual cannot deal with on their own.
And while it is true that we can expect regulatory intervention to be difficult, especially as the political process cannot possibly keep pace with technology, it is indeed very strongly preferable to giving free reign to private entities with explicitly anti-consumer interests (eg. a corporation has to make their shareholders money, not protect their customers or the environment).
So what's the solution? Maybe regulatory agencies need to be given more teeth, including funding and updated charters. Yesterday's Cartel is today's Big Tech.
this has nothing to do with AI, but an imbalance of power
An imbalance of power because of employing AI with suboptimal results, but not caring about the negative externalities because, well, they're external.
Is this AI, really? And isn't it immaterial if it is in the first place? If it's a dumb algorithm or a smart one (or a random one, at that) locking out people without recourse, the result seems much the same.
This is the next logical step in inverted totalitarianism: society ran and policed by commercial AI, without recourse or due-process because it "isn't government." This occurs as corporations take over the functions of previously public commons and forums, commonwealth/community infrastructure, and manage public-private "partnerships" (PPP's), effectively cannibalizing, ruining, and gaming government and public goods for profit.
1984 wasn't conceived from a vantage-point that exists now; the future from here has the potential to be far, far worse. These threads needs to be made into a novel and a movie to infotain people what dystopian concerns are tangible through unrestrained or under-regulated deployment of technologies on the horizon. China is already doing pre-crime and panopticon surveillance.
One of the problems with very large corporations like Google is the marginal benefit of each additional customer is negligible. They have so much money that they don't really need extra business and would rather pursue other objectives. Look at all the stuff getting cancelled lately for whatever reason. The whole thing takes place extra-judicially and is not in the monetary interest of these companies. Stakeholder capitalism if you will. I think the old profit driven system was better because at least you knew what the rules were. Now, the way these big companies make decisions is largely opaque and based on secret rules and arbitrary decisions often based on nothing but whim, or worse, a "good enough" algorithm.