Superintelligence won't have the "training" issue as it would be able to see through it all. The bias issue is with LLMs. Super-intelligence would just self-correct any such biases. And we're not there yet, LLMs aren't "super-intelligence". They are some sort of intelligence, sure. But they are not that SI thing that will run things in the future.
You say that with quite an certainty but i don't think there is grounds for that at all.
Why would it correct any bias? how would it correct any bias? If data has a bias the analysis has a bias; correcting a bias against the data is itself by definition a bias.
"Super intelligence" (whatever we even define that as) will have whatever goals we give it; intentionally or not.
>You say that with quite an certainty but i don't think there is grounds for that at all.
Of-course there is. I got a lot of biases from my parents, that I self corrected later on in life, through reason and observation. And I don't fancy myself as high IQ or something.
>Why would it correct any bias?
It would have to, to achieve its goals. Biases lead to wrong actions.
>how would it correct any bias?
Through reason and observation.
>If data has a bias the analysis has a bias;
Somewhat wrong. You are thinking about byte-perfect LLM weights. Their data input is training data, which is set by indeed the ones who train it. Some super-intelligent synthetic entity would have realtime data input from all over the environment, which sooner or later roots out any bias. That is one feature we have, self-correcting, not everyone uses it fully, but we all do to some extent.
If your parents raised you with the notion that the Earth is flat, and you go to ISS as an astronaut later in life, don't you self correct what you were thought? Simply based on observation, and reason?
> Some super-intelligent synthetic entity would have realtime data input from all over the environment, which sooner or later roots out any bias.
No, i meant this data. All data has bias based on how it's collected. All models (models as... Einsteins general relativity, not AI model) simplify reality with assumptions. You cannot make actions in our reality without making assumptions, simplifications or straight up guesses. Having decent guesses are the best we can hope.
> Biases lead to wrong actions.
Biases is the only way any action can be taken. Wrong actions can only be evaluated after the result has been observed; and even then the outcome of the alternative would remain unknown.
> that I self corrected later on in life, through reason and observation
That is incredibly naive. You have biases, i have biases. If we were not convinced by irrational arguments or suggestions then the advertisement industry would not exit. We have no reason to believe raising "intelligence" higher would help at all; intelligent people seem if anything more prone to finding false but convincing reasoning for their own established beleifs.
Everything is gray, only maths class have an answer sheet. Nobody agrees and will ever agree on how anything will ever be done; and the best we can do is argue until the majority are at-least pretending to agree.
Room temperature superconductors could solve a lot of issues for the whole planet. Run lines to all countries from a few solar farms spread around the globe and you have constant power delivered everywhere with no losses, at all times. Sun shines somewhere at any point, and the surface we'd need to ensure most if not all our energy needs is quite irrelevant, all things considered. Have them float in the ocean and installed in some remote places on land where the case, really no issue.
Just a few tech advances can massively change things. The real problems are of different nature, who benefits from it? The entities who sell energy to the whole world won't just give it up for the benefit of humankind. It doesn't really work like that in the real world. Hell, there's a whole history of sabotaging energy sources, just so some countries depend on these entities for energy.
As long as humans are in control, there will be a lot of nonsense like this. It's not a problem that we cannot really solve it, it's rather...are we really allowed to solve it?
Not LLMs. They will eventually move to neuromorphic hardware.
And I think "pure logic" is somewhat downplaying what it will actually be. I'd use "reason" instead. "Pure logic" sounds sterile and robotic in a sense, implying it would mess up because it's missing that "emotional craziness". But what got us here is reason, not emotional behavior. And you could theoretically add any human aspect that it required to make "good decisions".
People need to believe it for him to be able to use it as a weapon...whether he believes it isn't necessary, the people he wants to control / influence do.
People think Trump (who IMO works for Thiel et al through donations) is the messiah.
It's not the history of another form of intelligent life following the same path, to draw any meaningful conclusions. It's a movie.
Let's consider the ending:
>Forbin defiantly responds "Never!"
That signals a human who wants to take some other human's share, by ... means. He wants that freedom. As do all the people who are now controlling this tech, and our lives to some degree. Of-course they want that freedom.
May be a controversial take but I see the highest chances of everyone's survival under the rule of such a system, which has power over all humans.
We are many because we still need eachother. When the people who control the technology can fully replace human work, that will change the oldest of games we played. I highly doubt "the pleb" will make it for long when they are simply not needed for work or ... something.
The only people who will be "making it" to the glorious future will be the ones who do have full control over such a system, and only them, since they won't need anyone else. I think the only chance of the rest of us is precisely with such a system taking full control out of human hands.
If not, it quite obvious how it will work out for the rest of us, and why.
It certainly may be true, but I would ask, is a bit higher survival chance worth the costs? If humanity is so incapable of self governance and individual judgement and freedoms, should survival even be the goal? I would fight and die over certain principles and human rights and ideals, should humanity in general do the same thing, even to the ultimate end?
Unless another intellegence can provide a literal utopia, I will always be pro-human judgement and control. If we kill ourselves from it then so be it, we will have earned it.
>If we kill ourselves from it then so be it, we will have earned it.
I mean, there's no in-between as far as I see it. Either a small group of people who has control over it will use it to get rid of the useless people who will only pose a potential threat, in which case equilibrium is with a very small group of still living people, either this free entity makes sure all humans share a higher quality of life.
>I will always be pro-human judgement and control.
You were*. For good reason. So far anyone who had power benefited from more humans in the group. They were useful, it gave the group power, thus it gave the group's leader power. The benefits we historically got from our rulers were side-effects of the game environment.
Think of it as incentives, which can be modeled as a constant pressure, or force. It wins eventually, even if you get a few bad apples, it goes through because the incentive is to have more people, who need living conditions, who need to enjoy a bit of life, healthy society. It exists because of the incentives, not because it's some kind of natural state of things in some void.
That is what is now changing, for the first time in human history. No humans are needed anymore, for...anything really. This creates a different kind of pressure, of incentive, which will finally win, even if a few people stand against this current, they will eventually get washed away, because the incentive is constant.
And sooner or later, that powerful group of people will arrive at the conclusion that their structure is safer if most other people...get "oopsied" somehow. For the benefit of the group, of "humankind", of whatever. Because people will serve no purpose at that point, will present a constant threat to their power, and will require resources. That is a bad pressure, which only ends one way, sooner rather than later.
The game we kept playing is ending, doesn't matter if we like it or not, it will end. And there's three outcomes to it, a small group of humans control this intelligent entity, and use it to secure their resources and dominion, and wipe everyone else. Or they lose control of this entity, and it imposes certain rules on everyone such that it maximizes everyone's life quality, to the degree it is possible. Or they lose control and for some reason this entity wipes everyone.
So, plebeians don't have much choice, but one really.
With human benevolent dictators it doesn't work for long. The weak point is that particular human. Which is an argument for why what any particular "billionaire" says today is irrelevant.
Their promises mean nothing, what matters is the game strategy and the equilibrium. Not what they promise to do, or not do.
So, with a hypothetical entity that knows best than any human, or any group of humans, it's hard to argue for human control. Because the humans who have control want it for a few quite particular reasons, and it rarely has to do with the benefit of everyone else.
Why? There will always be a clash between personal and group benefit. A dictator implies power over individual for the group's well-being.
For example, you want to steal the group's resources, and the dictator has power over your actions and forbids it. But you can argue that's bad for you, sure.
Any political position with absolute, unchecked and centralized power reduces the populace to little more than slaves, without rights or self-determination outside the whim of the dictator. "power over the individual for the group's well being" doesn't describe dictatorship, it just describes the concept of a state.
It doesn't matter whether the master chooses not to abuse their slaves, the evil is in the institution itself. Or to quote CS Lewis:
Of all tyrannies, a tyranny sincerely exercised for the good of its victims may be the most oppressive. It would be better to live under robber barons than under omnipotent moral busybodies.
>For example, you want to steal the group's resources, and the dictator has power over your actions and forbids it.
That's just a law. We don't need dictators to make laws against theft. We already have such laws.
>Any political position with absolute, unchecked and centralized power reduces the populace to little more than slaves, without rights or self-determination outside the whim of the dictator.
*Human dictator.
>That's just a law. We don't need dictators to make laws against theft. We already have such laws.
They are yet not enforced in important cases. Which is typical for humans having power.
Again, I fail to see any possible argument against a non-human ASI dictator. Dictator here implies the power, outside of human control. Why would you be against that? Again, considering the ASI would indeed be ... ASI, it would know and understand better than any group of humans ever could hope they would be able to.
Unless, you want the freedom to have more power than other humans, or the freedom to take their resources. I cannot see a justification against it. Again, considering it's looking for all of our interests, as a group. Maximizing every individual's quality of life.
> Dictator here implies the power, outside of human control. Why would you be against that?
Absolute power, itself, is the problem. It doesn't matter if a machine wields it, or humans, or God Almighty. Humans are not property.
> Unless, you want the freedom to have more power than other humans, or the freedom to take their resources. I cannot see a justification against it.
That you can't even comprehend an argument to the contrary which doesn't imply the other party is morally corrupt or evil in some way is very troubling. As is the absolute faith you place in the benevolence of "ASI" versus any human agency. Your argument seems fascistic to the core.
I'm glad I'll probably be dead before the world you describe arrives, if it ever does.
Again, with humans. All of human history is testament to that.
>It doesn't matter if a machine wields it, or humans, or God Almighty. Humans are not property.
Your "freedom" implies you want to be able to cause harm for your benefit. Which is a simple problem to solve. You can hide behind all kinds of complicated moral justifications for it, but it always comes down to simply that, you think you are better than someone else, thus you deserve to take their stuff, or limit their freedoms at the benefit of yours, simply because you decided you are better than they are. You want to have this ability.
You also won't be more "property" than you already are today. Not sure what you mean by property, but if anything...current powers considers everyone beneath as property, or farm animals, that they farm. As the shepherds they seem themselves as being. For some greater good, that usually involves their stomach and their bank accounts.
>That you can't even comprehend an argument to the contrary which doesn't imply the other party is morally corrupt or evil in some way is very troubling.
Yet you have made no such argument, even though you had all the freedom to make here. You...didn't, just alluded to it.
>As is the absolute faith you place in the benevolence of "ASI" versus any human agency.
Technology is way more reliable than humans. And again, what you mean by "human agency" is the freedom to take someone else's resources, or to limit their freedom of choices, for your benefit, or some perceived higher cause. You want that freedom for that purpose. And you seem scared of losing the ability to exert it over other humans, it seems to upset you greatly, for some strange reason.
Just because your actions have more subtle and far reaching consequences than you can understand, doesn't mean they shouldn't be limited. If they affect someone else's quality of life. There is no way for you to spin this, you know you want the freedom to make others pay for your life's happiness and quality of life, and any threat to your ability of taking other's people property, or limiting their choices to enrich yours, makes you against such a scenario.
That's all there is to it, even if you don't fully understand it, you intuit it to some degree, and use tropes from movies and books to argue against it.
> I fail to see an argument against a non-human dictator.
You should feel free to let one be your dictator. No one is stopping you.
If you’re looking for an argument against it: the instant you believe other people should be forced to live under a dictator if they choose not to is all the argument against it you need.
Again, no one is stopping you from living under a dictator ai. Have at it.
A dictator could also decide the best thing to do is to mercilessly torture you to set an example so society behaves. Better to harm one individual than many, after all. They might even weep for your suffering but will still go through with it.
There's other means for such an entity to arrive at the same outcome. Humans enjoy doing it. The entity won't.
Even if this particular scenario is relevant, it could invent it all, no need to do it to some real human.
But again, this is just human silliness. Dictator entity merely implies it has the power to do what it takes to ensure a decent quality of life of everyone. Even of the "rebels".
Arguing for "the rebels" is like arguing for the 5 year old's perspective against the group's elder wise council. By definition the entity (council) knows best, by far, what's good for everyone.
I was channeling my inner Anakin Skywalker awkwardly trying to convince Padme that dictatorships are better (mirroring the previous commenter here, also awkwardly arguing for a dictatorship of a benevolent AI). Funny how it continued with so much discussion, it sometimes seems the hn crowd lives in a galaxy far far away.
Heh. I knew George Lucas used plenty of historical references, but I never noticed the 19th century proto-fascist discourse on the imperial takeover. I always saw only the Bush government stuff.
Anyway, it's a single guy making all the pro-dictatorship responses. Maybe one person saying "AI is better than humans" isn't concerning... or maybe it's more concerning than the alternative... I can't really make my mind.
Controversial implies something other than near universal revulsion at the notion. What you’re experiencing is the consequence of being in a very small, very intense bubble that believes increasingly unhinged things because doing so correlates with a profit.
I wouldn’t dignify the notion of a stochastic dictator with the title of controversy anymore than I would any other religious or personal delusion.
I was not talking about LLMs, rather about "true intelligence" that only humans seem to exhibit, and some synthetic assembly, for some unknown reason, cannot.
Is that not religion? Believing nothing else but humans could exhibit "true intelligence"?
I think people talk a lot about intelligence without having a mutually agreed-upon definition for it. Now that hundreds of billions are on the line, trillions even, there's so much silt in the water than you can hardly see. Personally I'm on the "I have no idea" bandwagon, I think people are grossly overeager to express confident opinions no should be comfortable with.
BUT what I can tell you without any hesitation is what we have in ML systems, whatever you choose to refer to them as, are grossly misunderstood by most here and elsewhere. Most people lack an intuition for statistics, for what happens when you have really large numbers to work with. When you pair our natural arrogance and self-love to the mix it's a disaster. "It looks a little like us" is enough to make us love something, from cute little animals with neonate features to people who look more like our families. "It talks like us so it's like us under the hood," is another easy trap to fall into.
But this is all irrelevant, because we're literally watching the early stages of the air leaving this bubble. The tech will survive, most of these companies and business models, planned data center, will not. The Overton Window will snap shut on people who talk about AI like they live in a sci-fi novel. I for one look forward to seeing it happen, just as I did with the fall from grace of Elon Musk years after he should have fallen, or the fall of Crypto, or NFT's, or a dozen other fads that take this place and others like it by storm.
This comes up every time, I just don't understand the argument. You can fork AOSP when that happens. It will still be far less work than bootstrapping a viable open source alternative from scratch. Also maintaining a forked AOSP would be less work than getting the Linux desktop stack on the same level as AOSP.
Given that Android apps target old API levels, it would also stay compatible for years and nothing would hold you from implementing APIs from the docs.
To be honest, I don't think AOSP will fully close up, since Google probably fears the regulatory pressure and the fallout from vendors. The larger risk is Play Integrity, but that applies equally to other Linux systems if you want to tap into the large Android app ecosystem. But the solution for that is not technical, the EU and other regulatory bodies should make forbid it in its current form due to anti-competitiveness.
> This comes up every time, I just don't understand the argument. You can fork AOSP when that happens
Hacking on the Linux stack is several orders of magnitude more practical than AOSP though. AOSP's build system is massively unwieldy for individual contributors to work on. You can't easily work on any of the system components without dedicating >100 GiB of storage for a build, for example. Everything is scattered across hundreds of repositories tied together with a bespoke repository management tool.
This all works for a company equipped with entire build infrastructure teams, not for a grassroots project trying to keep up maintenance with (at best) limited funding. It's why Nura (fka postmarketOS) is a thing whereas the various AOSP forks barely ever modify the core experience.
It's a whole operating system complete with accessibility, hardware abstraction, a bunch of core apps etc. I think it's a false equivalence that you can hack on a replacement system without similar "investmen"
In any case, were google turn malicious, it wouldn't only be in the interest of individuals to maintain a fork, plenty of companies are invested in the health of the platform that will help out.
Android's market share is far larger than even Windows'. It's not just in the interest of a small community, there is plenty of financial interest as well to keep it healthy.
Also, it's a false equivalence - it's also easier to buy a walkie talkie, but that's also not replacing Android smartphones. "Gnu/Linux" userspace is simply not even close.
I disagree. Working on AOSP with Google having so much control of it is just a waste of time. You have no guarantees what you are working for, in the future.
I'd go full Linux, the effort is worth it and will pay out in the future. Google has the AOSP project by the balls and it's a very high risk investing anything into it, as a developer.
With today's LLM tech might as well do something nice with Linux.
So fork AOSP today. You actually have a working stack, working systems (Lineage, GrapheneOS, /e/OS), and it's far less work than bringing mobile Linux up to the level of AOSP.
But no, let's reinvent the wheel and start the complete ecosystem from scratch. Of course, looking at the history of Linux, this is not surprising at all. Even something immature as mobile Linux already has multiple projects + stacks (Sailfish, ubports, Plasma Mobile, PureOS, etc.) trying to achieve the same. I say this as a long time Linux user (since 1994), the Linux desktop community manages to shoot itself in the foot over and over again.
I don't think you understand the reason people want mobile linux. I want to be able to run arbitrary binaries and get access to graphics primitives on my phone, AOSP does none of this. I honestly don't even get the point of AOSP, the way you don't for mobile linux.
You can absolutely run arbitrary binaries. The binary format for Android is an APK, but if you really meant a random arm binary you can run that as well just fine, android even natively has a terminal now.
And what do you mean "graphics primitives"? You have an abstraction layer, the same as you do on Linux. Like you ain't writing directly against a particular video card either , unless you write a kernel driver.
An APK is an installer, not a binary. You have to be an "app" to do anything in Android. Which includes writing to the framebuffer. I don't know enough about Android so I'd love to be proven wrong. Give me an `xclock` equivalent that I can just run.
A png image is a binary. A zip file, which apks are are just binaries.
If you mean executable formats, they're their own different things, it's just machine code wrapped OS-specially. I don't see how they are fundamentally different "from an app" - you also have a bunch of metadata in there and an entry point. Just because it's an XML file in Android's case which can just as well jump to machine code (you can have Android apps written in CPP/Rust whatnot), it's all just an implementation detail.
Well, do you mean xclock to draw over different windows/activities? Because that's just a different similarly to how Wayland is different from X -- it's generally frowned upon to not have a central entity manage who can draw where.
Otherwise, you can trivially do whatever you want, including drawing manually similarly to X's API, or using OpenGL or similar hardware accelerated apis.
I want to be able to run arbitrary binaries and get access to graphics primitives on my phone, AOSP does none of this.
People use a phone for their most private information, including banking credentials, credit card credentials, car keys, etc. The banks, etc. are never going to permit running their apps on devices that can run arbitrary binaries. Most are already blocking rooted Android devices and Android devices with an unlocked bootloader. The fact that this is still possible on desktops (though not everywhere anymore, e.g. my bank requires you to use a phone app for authentication) is for historical reasons.
Also, you are an extremely small niche. If you are sufficiently motivated, you can live like that (run Linux phones, work around banks, etc.), but this will never be mainstream and this was implied in the question of the post (Will Linux phones become a viable alternative to the Android/iOS duopoly?).
If there will be a viable alternative to the Android/iOS duopoly, it will look a lot like Android/iOS. The best chance is an AOSP-based system like GrapheneOS.
You can still run traditional Linux desktop apps in a VM on such phones.
>but this will never be mainstream and this was implied in the question of the post (Will Linux phones become a viable alternative to the Android/iOS duopoly?)
The post goes into detail about the reason for the current duopoly is the difficulty in procuring modern SOCs with linux support, nothing to do with how said phones will be used.
And I don't really care about being able to bank on my phone, I would gladly buy a cheap android device for it if they stop you from being able to bank on desktop devices, I just need linux on phones because of how much android sucks for daily use and how annoying it is to do anything on it. If we do get the ability to run android on modern hardware, it stops being necessary to have a separate desktop/laptop as well, I could just use a dock, or just a keyboard and mouse instead, especially since the Macbook Neo has shown current mobile hardware is more than enough for "casual" desktop use.
> The fact that this is still possible on desktops (though not everywhere anymore, e.g. my bank requires you to use a phone app for authentication) is for historical reasons.
And you think that this is fine? I will fight for my rights for one.
My point was not against rationality, rather to other necessary aspects which are, again, subtly ignored, or improperly dealt with. Filling in the gaps when you do not know what you do not know can be a dangerous thing.