There was an interview with Altman where he was talking about various regulatory policies that had been enacted, then at the end he added "but in practice none of it turned out to matter. What we didn't anticipate was that the companies became more powerful than the governments". I wish I could find it today but it seems gone
I think the most instructive example of this is Stephen Elop. He had ONE (1) successful business venture in the 90's and has done a horrible, laughably destructive job at everything he has ever touched since, and is still a CEO.
Look, without going into all the details... the contract the insurance company had pressured me to sign quickly so the contractor could start ("We don't want to lose them! It's so hard to find a contractor during the pandemic!") had a clause in it that said the contractor wasn't responsible for damages to my property caused by their crews.
So the 40-foot tree they killed by dumping all their paint thinner on the roots, or the paint splatter they got all over my fireplace using it as a mixing station, or the foundation that they cracked by breaking three tension cables while doing exploratory surgery trying to install a better drain in the laundry room (so the flood wouldn't happen again), or the electrical circuits they cut and put drywall over to hide the old plugs, or all the tools my grandfather left me that they stole... or the... look, the non-cosmetic issues in my counterclaim filled over 80 pages of a lawsuit, without photos. It was extensive, documented damage they did to my house.
Anyway, the contract also called for mandatory binding arbitration. So we went in, and the mediator was like, "OK, tossing all your claims. Great, now that those are tossed, pay them for their work."
The mediator said, "Look, we have to give the benefit of the doubt to these hard-working men who came to your aid in your hour of need. Was the job done right? Maybe not, but they were the ones you chose to hire. You can choose anyone, and you picked them and signed their contract."
Oof.
I had a video of them dumping paint thinner at the base of my tree, they did it 5x in one day. I had videos of all of it. I had soil samples showing the tree died from paint thinner being dumped on it. The mediator was like, "It's not relevant because of your contract, but I'll humor you... Yes, you have videos but we can't tell what they are dumping. Yes, you have a soil sample, but how do we know you didn't just do this to your own tree?" It was just so surreal.
It was a large company, more like a loose network of contractors that sub-sub-subcontract everything. The lack of accountability was a huge problem since there had been over 160 people in my home, plumbers, carpenters, tile guys, etc. over the span of the construction, and even though I had cameras it was on me to prove the people worked for the parent company I was in a lawsuit with and not just random people I had let into my house... "How do we know these problems were caused by XYZ company's staff or contractors?" And I'd have to be like, "You think I hired my own painters? I wasn't trying to a parallel remodel project... everyone was their guys..." But it was on me to prove and I didn't have IDs or full names fro 95% of them.
Anyway, I also found out afterward that the mediator was a retired judge who had done literally nothing but cases for them for the last two-plus years before my case. The contract called for them to be able to pick two or three mediators, and we could then pick one from their list... but the fact that one mediator was able to stay employed doing nothing but cases for this one giant insurance/contractor organization... well, I couldn't help but feel like they knew where their bread was buttered. =P
Anyway, I had to pay all my cost of living for 2+ years while not living in my house. I had to pay for my lawyer and all the experts I had hired to build my case. Since I lost the mediation, I had to pay for their lawyer. I had to pay them for the shoddy work they did, as well as a very high late penalty. This is where it gets super shady too, in that my insurance company mandated I use someone from that network, and the contract said I had to pay my deductible, and 50% up front and 50% on completion. Fair.
But it came out in mediation that the insurance company paid this contractor 100% up front. So that took all their motivation away from fixing my house and let them drag it out almost three years.
Also, I was like, "So if insurance paid you all up front... why are you demanding 50% of the remainder of the contract? And tell me again, why won't insurance cover that?"
I never got a great answer, but I was told by my insurance company they wouldn't reimburse me for the final amount because the work wasn't "done." I don't know. I asked my lawyer, and she was like, "Yeah, you signed the contract, so you're on the hook. Insurance doesn't have to pay you the remainder of the claim, especially if you don't 'finish' the work."
Feels like the contractor either got paid twice, or likely got paid up front and then, once they got my money, were able to reimburse the insurance company... who was, in truth, their real client. They had some sort of collusion going on the whole time. Insurance companies should not be allowed to own, or partner with, any contractors they recommend. It's all so dirty.
Oh, and, "Why didn't I have a better lawyer?" you ask... Well, I had to go through over 50 in my area just to find one that would take my case. Any firm I wanted to hire gave me a polite decline, "Sorry, we're not taking on new clients." A friend of mine was a lawyer in another state, he said, "They probably all have consulted with the parent company, or want to, so they are declining due to conflict of interest." Then, just to add some bad luck, my original lawyer had a heart attack and had to pawn me off on to another firm last minute. I don't know, the first first wasn't very good either, but the second really didn't care.
And... "Why didn't you go to court?" Well yeah, I could have walked from mediation... certainly couldn't have done worse than I did at mediation. But my lawyer said, "If you want to take this to trial, you'll need to give me a retainer in the $150k range, and it'll likely be 2-3 years before the case is heard, and you will be required to document the state of the house monthly and not move back in until this is all over..." And she ended with, "And,as much as you want to fight this... you're going to get the same result due to this contract in court that you'd get here. Cut your losses and move on." God bless the Texas legal system.
All of it felt crooked, man. It's not good for my rage levels to think about this. Ha. Live and learn, but insurance companies and especially their contractors are shady people. Out to screw you any way they can.
A misaligned model is only one small part of step 2. Now this misaligned model has to suddenly acquire more power than every single other AI on the planet. It has to be immune to shutdown, manufacturer its own replacement hardware, and acquire chips, energy, raw materials, etc., with vigorous human opposition (this is an extinction scenario that AI doomers are predicting, after all)
Nobody has satisfactorily explained step 2 other than “well, it’s a superintelligence” which sounds lot to me like “it’s God”.
Well yes, if it is a superintelligence, it will be able to do those things. That's what superintelligence basically is: The ability to achieve complex goals.
If you want the details of ways it can do it I recommend reading some of the reports about the HuggingFace breach that happened in July (Read more than one).
> Doomers like Eliezer Yudkowsky define “intelligence” as “optimization power.” Intelligence searches for more or less optimal ways to achieve a goal—any goal—and finds the most optimal one. Anything you can do, the blob can do better.
>Which implies that “intelligence” is the holy grail of Darwinian evolution. If an animal needs to recover from illness, “intelligence” can speed up the recovery. If an animal needs to avoid predators, “intelligence” can reduce the danger. Regulating body temperature, defending territory, digesting nutrients, breathing, molting, mating, healing, foraging, vomiting, parenting, navigating—all these problems can, apparently, be solved by giving the animal more “optimization power”—i.e., more “intelligence” sauce.
>Solving all your problems with the same blob? What a bargain! It’s a “buy one get 100 free” Darwinian deal! Why haven’t any other animals jumped on it?
>Think about how weird this is. Out of four billion years of evolution and more than five billion species, only one animal, Homo sapiens, fully expanded its “blob of compute,” the thing that literally gets you whatever you want.
>And many species already have it! Brains are all over the place. Plenty of animals can learn stuff and predict stuff. Why hasn’t natural selection looked at their blobs and said, “HOLY SHIT, COPY AND PASTE THAT THING A MILLION TIMES?”
>If doomers are right about the awesome power and breathtaking simplicity of “intelligence,” then we should see a world teeming with superintelligent animals—giant brain-blobs slithering across the landscape.
Yes, those animals are called "humans". All you have done is respond to an explanation of your lack of understanding of intelligence with arguments demonstrating a lack of understanding of evolution ...
What? Humans are giant brain blobs creeping across the landmass? They definitely are not. Nor is it even clear they're winning the evolutionary war. I guarantee cockroaches and ants will still be around, but nobody thinks of them as super intelligent, while humans with all their "intelligence" might accidentally wipe themselves out.
This convo is getting uncomfortably bad faith, maybe even religious. I can't engage with someone who refuses to say anything concrete but spews back religious tenets like "super intelligence", which I may remind you is entirely hypothetical, like "singularity" or "God".
Why do you assume that human opposition will be vigorous? What makes you think that humans will be aware of, or be able to agree about, what's going on at all?
If their activities materially increase the probability of human extinction and this poses a risk of potential loss to shareholders then they must disclose it.
If their activities materially increase the probability of human extinction, is there any reason that they should not be treated as invading aliens, or doomsday apocalypse-cult terrorists?
Like, why on Earth would anyone tolerate someone in the same train car playing with a grenade?
I dunno, we share an earth with North Korea who has nukes... which perfectly fits your "same train car playing with a grenade" analogy, but we don't do anything about it.
Are you gonna continue to tolerate it? Or are you gonna stop them from playing with grenades in this train car we call earth?
In my opinion, no; but they are now getting in bed (Metaphorically) with people who see their software as potential means to "beat China" or "beat Iran" and together they are weaving a very tangled Pandora's box of worms
reply