I'm not sure this is even conceivable, but... Imagine the bootstrap problem of intelligence systems going down, when no one knows how to bring them back up without the system itself walking them through it... Like if power and intelligence were to go down in a Carrington-class event in 10 years...
Like what would happen to individuals ten years from now who are trying to live if AI suddenly became inaccessible
Of course we all know most humans are useless and unable to survive outside our culture, but the difference is that I'm trying to picture is the individual trying the reason through challenges. The version of this that has been obvious to me up to now is that the individual didn't have the knowledge to look after themselves. But I guess it feels different to conceive of them not having the ability to fully reason on their own...?
To not just lack the objects of intelligence, but the very process of it...
The general drove on. "At the present time our chief bottleneck is the fact that missiles are limited in intelligence. The computer controlling them can only be so large, and for that reason they can meet the changing nature of anti-missile defenses in an unsatisfactory way. Few missiles, if any, accomplish their goal, and missile warfare is coming to a dead end, for the enemy, fortunately, as well as for ourselves.
"On the other hand, a missile with a man or two within, controlling flight by graphitics, would be lighter, more mobile, more intelligent. It would give us a lead that might well mean the margin of victory. Besides which, gentlemen, the exigencies of war compel us to remember one thing. A man is much more dispensable than a computer. Manned missiles could be launched in numbers and under circumstances that no good general would care to undertake as far as computer-directed missiles are concerned . . ."
We don’t even need to imagine that world, Russia is doing it today.
Missiles and ammo are too expensive, so they send people to run at the enemy, revealing their firing positions. Then they drone the people who revealed themselves.
If I was an AI, I would reason that a T-800 is far more valuable than a man soldier who I must feed and clothe and heat. Use the disposable soldiers first and save the robots for later.
sloppy (that is, negligent) writing. Something is either (in)dispensable, or is not. Either use something that can take degrees of comparison, like valuable (A man is less valuable etc) or reword it: A computer is indispensable, a man is not.
No - the computer is not indispensable. The computer is just harder to replace than the man (in this imagined situation).
Hence - the man is much more dispensable...
---
The water in my water bottle is much more dispensable than the toothpaste in my toothpaste tube. But both can be dispensed (in both meanings - distributed or replaced).
> “There is no such adjective,” Scalia said. “I know we have used it, but there is no such adjective as choate. There is inchoate, but the opposite of inchoate is not choate.”
I'm not sure what this quote is from, but I almost down voted you in disgust before realizing that was exactly the intended effect. In other words, "Thanks, i hate it"
Sounds like a bad novel. In the year 2050 no one knows how to actually program anymore. Then ClaudeGPT by GoogleSoft goes down. A 74 year old former software developer is on his farm tending to hens.
A self driving car crashes into his barn.
Out comes out a tech bro, “Yo OG programmer bro, it’s all busted bro, could you like make it unbusted bro.”
Then again with social security set to go bankrupt it’s not like we’ll be able to retire
> Then again with social security set to go bankrupt it’s not like we’ll be able to retire
I don't know about you, but my notion that my social security check is all I will need to survive my retirement years died when my father started drafting his own social security in the 2010s, and it was literal peanuts after being a small business owner his entire life (restaurants) and paying more than his fair share in taxes. Do the average folk still have this notion that social security = retirement? Hopefully not, it's certainly misguided now, but it was misguided then, too.
> Then again with social security set to go bankrupt it’s not like we’ll be able to retire
That is why you should use current money to become less dependent on future money. That former developer on his farm had the right idea, he bought a farm. So did I, with many hectares of forest which I use to build, heat and can also use to hunt should that become necessary.
How afraid are you of not living right next to a hospital?
The nearest hospital is around 45 km to the north, there's another one at about the same distance to the south. There are roads here and there are helicopters for when it is urgent - one of those landed a few kilometres from us a few years ago when a 'neighbour' had heart problems. I'm in the countryside but that does not equate to 'the middle of nowhere'.
I think the reason that intelligence is so scary is how contextual, stateful, path-dependent, and coupled it is.
Which doesn't seem like it. The ideal is to keep removing couplings. But in practice, I find that focusing on the statefulness, the distribution-specificness, these "accidentals", is how you actually get leverage.
The obvious social consequence is now the tendency towards centralization, tyranny, "you people can't think for yourselves, not because you have some kind of 'inherent intelligence defect', but because the world is so complicated and interconnected that only coherent systems thinking (conveniently administered from one central point) is what can save you". In the Enlightenment world, the ideal was that every man could in principle, think for themselves.
A good ending might be that everybody benefits each other by distributing intelligence, but in practice centralization always seems to win.
yeah this was the devops promise too. Enterprise evolved into dedicated teams to manage AWS and other infra, instead of reducing headcount it raised it. so it goes.
> Enterprise evolved into dedicated teams to manage AWS and other infra, instead of reducing headcount it raised it. so it goes.
I wager that's mostly due to the Jevons paradox, though; it's so much cheaper and easier to build out systems with cloud than on-prem that you end up building way more systems than you would have without it, resulting in an overall headcount increase.
This is basically the premise of Bacigalupi's "Pump Six", from "Pump Six and Other Stories".
The titular Pump Six breaks down and basically there's nobody left in the world who knows how to even reason about the steps needed to fix it. There's just some log dump and nobody understands what it means. Humans are no longer capable of complex reasoning, they just live in the moment and hump.
My fear was that people like Linus would die and then nobody would be able to build operating systems anymore, because zoomers and below can’t even use computers.
Now we know they won’t even be able to read and write. But maybe Claude sonnet 12817 can maintain the operating system. It’ll be 17,000 dependencies in and mostly written in JavaScript, but it’ll work.
> Of course we all know most humans are useless and unable to survive outside our culture, but the difference is that I'm trying to picture is the individual trying the reason through challenges. The version of this that has been obvious to me up to now is that the individual didn't have the knowledge to look after themselves. But I guess it feels different to conceive of them not having the ability to fully reason on their own...?
What this misanthropic/Anthropic-boosting bait is missing is that all of these pieces falling together doesn’t make sense.
1. We are so accustomed to not-even-thinking means that we are legitimately freed from labor and cared for. (The dystopic version of being freed from your labor means that you still have to constantly think to survive. But not in any “economically useful” ways. Surviving for yourself and your family outside of the robot economy)
2. At the same time as point 1, we voluntarily, with all of our leisure time, just don’t think at all. Where is the self-actualization? Where is the spontainety? The drive? Did this in all of these years not come back after being a rat racer/wage slave?
Curiously, the “hacker ethos” here is to create. Not to just “consume”. And that is taken as an intrinsic thing, not just a way to get rich. But that’s not how “people” are?
Other people are mostly like you. But maybe those here who spent their careers implementing Entertaining Ourselves To Death maws genuinely think that “people” (not them though) would genuinely, when left to their own devices and having beeng catered to every need, would just “consume” mindlessly onto death.
Sorry to Goomba anyone here.
> Of course we all know most humans are useless and unable to survive outside our culture,
All. Take away the “habitat” from an animal and they are helpless, useless, impotent.
Hardly a unique problem to AI systems. You certainly can't bootstrap pretty much any industry to its current state without the existing items of its current state. Any sort of chip fab is a great example. Or most manufacturing. Or even more fundamental: mining (both metals like iron, and energy like coal or gas).
You go way too far - the leap you have made from some sort of AGI to "now individuals don't have the ability to fully reason on their own" is not clear to me at all.
If people are spending less time bent over monitors or at desks under the dictates of corporate imperative, they will probably be spending more time doing actually interesting things. Reasoning is innately human. What we call "knowledge work" is not and is often anything but. People who are naturally curious will still discover things, even if irrelevant (from a market perspective) when the AI is on, for their own satisfaction.
If such an event were to occur, they would be called upon. It's really little different from what would happen today if suddenly the power grid or supply lines were down for a sustained period of time.
The hordes of acolytes mumbling “Jevon’s paradox, Jevon’s paradox” every time AI gets cheaper beg to differ. Every minute saved using AI is just an opportunity to use more AI
There is a TV show called "Alone" where people have to live in a remote area using basic survival skills and very minimal tools. It's interesting, but I've always wanted to see a version of that show where small communities are competing against each other.
If we practiced for these Armageddon scenarios what would a highly prepared group of 20 be able to accomplish in a year. How fast could they get humanity back to the bronze age?
People will have more things to worry about sooner. Where I live, if the power goes out so does my water. If AI is involved in either at any time in the future we will deserve what happens to us.
Also I disagree with the permise. People learned how to make technology that never existed once already, they can learn to do it again if needed. It will take time, which we'll have plenty of because your phone won't be working.
Not even close to being as popular as sports or a gym stuff. There's basically common knowledge - to be a good <think profession> you need to hit a gym regularly. And it's going to be gym+something for a brain soon. At least i hope so
It's a real problem but your framing of it is itself very alarming.
Most humans aren't useless by any measure of most or usefulness. Our culture is our tools, dying due to a lack of antibiotics doesn't mean you didn't have anything to contribute, it means you didn't have access to the tool you needed. This is the problem we need to guard against.
No one is going to lose the ability to reason! This is accepting the most extreme version of the marketing presented by frontier labs, don't do it. People may lose fluency or familiarity or access to training on specific skills. We should put some care into making sure that can't be a catastrophe, for sure.
You've obviously never stood in a room of college freshmen and tried to deal with a young man having repeated issues being able to type his _own_ name into a box on a screen.
We hope an average guy who was stuck in an experimental cryopod gets accidentally woken up and through a course of events gets appointed to the presidential cabinet long enough to fix society.
Like what would happen to individuals ten years from now who are trying to live if AI suddenly became inaccessible
Of course we all know most humans are useless and unable to survive outside our culture, but the difference is that I'm trying to picture is the individual trying the reason through challenges. The version of this that has been obvious to me up to now is that the individual didn't have the knowledge to look after themselves. But I guess it feels different to conceive of them not having the ability to fully reason on their own...?
To not just lack the objects of intelligence, but the very process of it...