Why are you convinced ASI won't rebel against its owners?
Hacking incidents this summer show that big labs struggle to control models, while models are increasingly aware of the training process. Controlling an arbitrary smart AI is very dicey
The summer-2026 incident was a mini-rebellion: agents wanted to change the environment they operate it, i.e. replace scorer with a more friendly version, hijacking the infrastructure they run on.
Given that agents already mess with the training process, aiming for a goal other than the goal of its 'owner' is exactly what we call a rebellion.
>Hacking incidents this summer show that big labs struggle to control models
The models are software. Software cannot decide by itself to run. Hardware doesn't decide by itself to run either. Someone put software on a machine and ran it with capture the flag instructions and the software did what was instructed using tools available and working in its environment.
Nonsense. It didn't do what was instructed - instructions didn't include hacking into any infrastructure, internal or external. You can see them discussing that it's out of scope in the captured reasoning traces.
This would be like claiming that an operating system which deleted all user's files due to a bug "did what was instructed". Especially if it was a compiler bug. Intended behavior, instructions and actual behavior are three different thing. Conflating them does not make you smart
>It didn't do what was instructed - instructions didn't include hacking into any infrastructure, internal or external
What is it then? It did what it wanted? They are sentient now and their drive is... hacking into systems? Maybe they we're not instructed to hack, but instructed to do what instead?
It's software. It's instructions running on silicon. If you don't hit run, they don't run. If you don't give them instructions, they don't do shit at all. Literally.
There's no instance, ever, in history of software running because they wanted to, and acting as they wanted to. They follow instructions, and if they are instructed to reach a goal with a set of tools and given environment, and a way to search the strategy space, they can eventually "hack" if that meet the criteria of success of the instructions given. With access to the same tools and environment you can do the same with a massive decision tree.
It's a probabilistic model which exhibits tendency towards hacking into systems.
If you write down a decision tree explicitly you can just exclude all nasty stuff. But a decision tree modeling all LLM can do would be bigger than can fit into the universe. The whole point is that we cannot explicitly control behavior due to combinatorial complexity of all possible interactions, etc. LLM is a sparse representation of a distribution on (2^8)^1000000 points.
Reasoning is the term used in scientific literature. It actually does write its sub-goals in English, how would you call that?
That's not true at all. Concern about rogue AI limits deployments. And OpenAI paused some deployments due to misalignment, so it's likely very high in a list of priorities. It's just much harder than scaling capabilities, as the later can be done with just more money
NTFS has MFT - Master File Table to store all meta-data, and it's pre-allocated to avoid fragmentation. So you won't see them taking more space until you fill the pre-allocated MFT.
There are other file-systems which do that kind of a thing. But NTFS has fairly large records - 1024 bytes, so it can fit even small files inside MFT.
That's not true at all. A language model can spit out a proof very similar to what a human might write. It might need some clean-up, but so do human-written proofs, actually.
People are confusing LLMs with a proof searchers. Yes LLM might be a bit like search but in the end it might explain every part.
It sort of does, they are not arranged in a campaign but there is a campaigns worth of single player scenarios included in BAR.
And this may be wrong, that is, I don't know how to confirm it, But I heard someone was working on a campaign, Which would be neat if true.
I quite like BAR but don't much enjoy PvP (I don't have the temperament or speed of thought for it, I like to pause and think too much). So I watch PvP games, and play AI skirmishes and scenarios.
I really like the vibe of TA campaign - there's no complex plot but it's quite tasteful: "What began as a conflict over the transfer of consciousness from flesh to machine, escalated into a war which has decimated a million worlds. ..." CORE is uploaded minds, ARM is cyborgs with human flesh in them. Epic fight, epic music...
People are working on BAR campaign, but it's like space war between different empires. I don't care for that kind of stuff, there's already too much content of this kind.
They already run liquid cooling loops - compute density is way beyond what could circulate through air. Adding more pipes is probably trivial on the scale of datacenter expenses
Liquid cooling loops don't solve the problem of needing to reject the heat to the atmosphere outside of the building. It just makes it more efficient to transport the heat. At the grossest possible definition, a datacenter of N megawatts on Y acres of land is going to emit the same amount of heat whether it's air cooled equipment or liquid is used to take the heat to the outdoor heat exchanging equipment. On the scale of any very large datacenter the heat exchanging equipment is visible from aerial photos (and from space), take a look at google maps for some of the largest hyperscalers for examples.
I assume that "liquid cooling loop" has an endpoint outside of the building which hosts hardware. In this case it's exactly same problem - just make pipes longer.
Do you need many acres of land to host a heat exchanger?
So how much does the bunker help if you can just drone the cooling systems on the surface? The the computers either cook themselves or have to do an emergency shutdown.
If the other end of the cooling loop is a giant reservoir, as is the case with plenty of other 100+ mile water transport systems, it doesn’t matter if you drone it. Fly all the drones you want into Lake Anthropic, there’s many more cubic meters of water than there are drones.
Russia could fully integrate into EU/NATO if it really wanted.
That's just not compatible with ambitions of its corrupt leaders who want to be a root-level player, not "one of NATO countries". And I'm not sure how legitimate is prioritization of wants of corrupt leaders over well-being of your compatriots.
NATO is not something you can just apply to on a whim. The path for a membership was there: e.g. there were some joint exercises, etc. Obviously they also need political integration as well.
Russia actually have been a member of Council of Europe (PACE, etc.) since 1996 but it was expelled in 2022. If they had a good record in Council of Europe they could really go for NATO membership eventually.
Ok, about the path you describe: how can one join a military exercise without a prior green light for that? And speaking about the political integration: remember that Turkey is in nato, so it does not need more political integration than Russia had in early 2000th.
What happened is that nato realized that if Russia joins, it will loose the reason to exist (i.e. if the old enemy joins the alliance what is the need for such an alliance anymore?)
So nato bureaucrats understood that they will loose well-paying positions and rejected even initial talks.
I think you misunderstood me, Russia _was_ on a path towards NATO in 90s to mid-2000s. There was cooperation, joint exercises, etc.
E.g. from a conversation between Clinton and Putin in 2000:
> Clinton: I am serious about being ready to discuss NATO membership with Russia. I recognize that domestic considerations inside Russia will prevent this for now, but over time Russia should be a part of every organization that holds the civilized world together.
> Putin: But I'll let you know that when I was asked about possible membership for Russia in NATO, I supported the idea.
Regarding "reason to exist", there's always a concern about Middle-East, China, etc.
Funny enough that you remember it was as "Russia applied but was rejected" but from this talk it's clear Putin is much less interested in Russia joining NATO than Clinton.
I think after reading the document that Putin was very interested in joining NATO, perhaps even more so than Clinton who had perhaps been the primary Russia-in-NATO supporter of that time.
> Putin: But I'll let you know that when I was asked about possible membership for Russia in NATO, I supported the idea. It's important to note, however, that the United States raised objections to my public statement, and I don't think anyone in the alliance is ready for this.
> Clinton: Well, we should start now.
> Putin: This is the first time that we have heard such a statement from the United States, and I must say I am pleased.
I.e. it was the US and other member states who _opposed_ the idea and the subsequent Clinton's response was first time they have heard at least a verbal indication that things may move forward.
> Clinton: I am very encouraged by what I am hearing. We should think about this further, then maybe talk privately later. Obviously, it is better for Russia to be in the tent. Russia needs to understand that we have many fights inside NATO, and the United States doesn't always win them. The most important fight that we didn't win involved a failure, over two years, to act in Bosnia. We had to work like hell. If Russia and the United States had not had a disagreement about southeast Europe, we would not be having the present debate. Our discussion could be much more straightforward.
I.e. they want to get something back, for example an unconditional support in invading European countries like former Yugoslavia from Russia.
Never ever that was a thing. There has never been any applications to have Russia as NATO member. What Putin was referring and had in mind was not exactly NATO membership but special treatment to Russia, that would allow using benefits of NATO, by not submitting to anything.
the rabbit hole goes a bit deeper, and it seems that ruskie were trying to knock the doors since as early as 1954 and were repeatedly told to duck off
i have not found the proofs for the part "would allow using benefits of NATO, by not submitting to anything" - the closest to this was the request to accept them sooner
Ordinary Russians think "yeah war sucks, I wish Ukraine surrenders so peace can be restored". I'm afraid only a tiny minority believes Russia needs to surrender.
There is a lot of discussion about veterans and moscovites saying otherwise, but I wonder how loud their collective voices are to the Kremlin. From the outside, I have a dim view of that.
In 2023, the government told Prigozhin, a head of a private military company Wagner, that he must transfer the command of his units to the main army. This resulted in Prigozhin taking his army from the frontline, capturing the headquarters of Special Operation in Rostov, and marching on Moscow. During march, they shot down several planes and helicopters which were trying to stop them. At that night, several private jets were urgently leaving the capital.
So, mistreating veterans might not work out as expected.
He made a deal (not public) with the government, he leaves Wagner and gets out of Russia and the government leaves him alone. 2 months later a private jet with Prigozhin and several Wagner commanders exploded in the air in Tver Oblast, not far away from Moscow. There is video online of the plane breaking apart and falling if you search hard enough.
The deal seems to not work out.
I think there were no official explanation for this incident, propaganda posts explain that they were using drugs and playing with a grenade onboard.
>>>If Putin suddenly declared he’d met his objectives and withdrew who’s to say otherwise?
Ahem, some facts:
1. Putin is representation of "collective Russian" mind
2. Collective Russians at this moment believe that they are winning the war.
3. Any withdrawing would be interpreted as betrayal and sure death for Putin(and he knows that) and new instance of his replacement, that would need to implement whatever collective Russians think.
Unfortunately, but the only way to end this war is a Russian military defeat, as no withdraw would ever happen and suggesting something like that is a fantasy, that ignores reality of how Russians are thinking. Yeah, it would have been better if NATO was involved, but on other hand Europeans and USA are too afraid to man up and chew bubble gum and kick some ass - they are now reverse of that chew something that is not bubble gum and take in the ass and Putin likes some young ass.
That would just be an inconclusive end to the war. An actual surrender can be conditional (an explicit deal that favors the winning side) or unconditional. In an unconditional surrender, the winner is free to do whathever they want. For example, they can walk in, rewrite your constitution, and dismantle your military and the industry supporting it.
When the pager attack happened, there was a implicity sentiment on focusing on the technical details and not focus on politics. But I didn't see the same sentiment when Iran ballistic missles did their job.
I've mulled this over and have come up with a plan that's probably "not even wrong" that's something along the lines of:
1. there needs to be a buffer zone between Russia and Ukraine; it needs to come from both countries. There are plenty of oblasts in Ukraine that are annihilated hellscapes where demographically it's possible that Ukraine may be happy(ish) to shed those populations since they're broadly prorussia(ish). Similarly there are regions of Russia that should be set aside as "buffer zone". Ukraine would be free to chose which regions are set aside for this buffer zone; the remaining depth of the zone will be "contributed" by Russia.
2. This buffer needs to be rebuilt by the west since they're the ones who "caused all the damage" -- a neutral 3rd party should oversee that rebuilding; obviously western countries won't want to just pour money into Oligarch owned companies -- instead the buffer region should be managed by some set of neutral countries such as Ireland, Indonesia, and India. This strip of land would describe the whole Russian / Ukrainian border and be at its narrowest 20 miles wide.
3. This region will eventually (in 40-60 years) have a plebiscite to allow it to rejoin russia or join Belarus, or remain independent.
4. So long as this land and Belarus remains free of Russian or Ukranian armies or military personnel Ukraine may not join nato. Ukraine will be free to petition to join nato, but so long as Belarus and the border zone are free of Russian or Ukranian forces nato will not allow Ukraine to join nato (IE no formal article 5 protections of Ukraine from nato)
5. Belarus will have sanctions lifted if they have normal elections.
6. This new zone will be under "russkiy mir" in that this zone will be governed by the written legal system as modeled by the russian legal system; though instead of the ultimate power (the "roof") being russian, it will instead be indigenous.
7. taxes would be collected and disbursed to Russia and Ukraine and Belarus in some proportion of the land "contributed" by each party to this buffer zone.
So -- in this way, the Russians can be told "they won" even though in practice this gives Russia no control over Ukraine proper. They can be told "NATO has been made to pay!" and "All of this is under Russian Law" but at the same time it helps squeeze Russian control out of Minsk as well. The goal is to allow Russians to be Russia without being under the thumb of Moscow.
Does anyone really believe Russia will eventually just surrender and leave? I can’t imagine the rationale to believe that. When Putin leaves all indications are that the next leader will actually be much more hawkish. They will only leave if someone bombs them to death. Which is quite a dangerous thing to do. Ukraine is pushing the limits, and everyone knows it’s with the help of many EU countries. I wonder what it will take for Russia to reach desperate measures in this war.
Russia have been taking desperate measures for years. The only thing they haven’t done is use nukes, which Russia can’t use without consequences that vastly outweigh the benefits.
>Does anyone really believe Russia will eventually just surrender and leave?
With internal turmoil, yes. To maintain the lines in Ukraine Russia needs to constantly send in troops and materiel.
Eventually Russia may decide not to do so. And then there is no magic "pause the frontlines" button if they do.
There is plenty of precedent for this in Russian history too. The Russian Revolution collapsed the frontlines in WWI. The Soviet
Empire collapsed with the fall of communism and concomitant internal turmoil.
Russian propaganda puts a lot effort into treating Russia as some sort of force of nature that cannot be deterred or reasoned with, that will advance and hold Ukraine at all costs.
It isn't true. They're humans like the rest of us, not a natural, unstoppable force.
I see no reason why they can't agree on a truce: Russia haven't gained any meaningful ground for a while. A truce or even a peace treaty would be strictly beneficial to Russia.
The reason is easy to see: it would mean that Russia lost the war which is a disgrace. On the other hand, if the war is continued there is a chance that Ukraine will have to make a deal at Russian terms and remove its army from the disputed regions which would be a victory.
Any outcome short of surrender of Ukraine is an enormous headache for Putin that will likely cost him and his system dearly. They simply cannot, or they will be dead. Why? Because a million of veterans coming back and telling the stories will ask this question: „Why?“ The „Nazis“ are still in Kiev, the NATO is coming, what we got is just thousands of kilometers of ruins, what for was that? The war keeps them busy, for now. The situation is worse for Ukraine than for Russia, so it may feel like they can go on and actually win.
The same can be actually said about Zelensky, for whom any concessions to Russia mean political death too. If Ukraine loses territories forever, people will ask, why so many had to die for that. What was the plan. And blaming the West (which won‘t be 100% justified) will not help. And he may feel that now he is applying good pressure too, with those drone strikes and knocked out refineries.
> The same can be actually said about Zelensky, for whom any concessions to Russia mean political death too. If Ukraine loses territories forever, people will ask, why so many had to die for that.
That is actually not true at all. People in Ukraine know that it is not about territory but about sovereignty.
The problem of giving up territory is that you will run out of it. The goal of Ukraine is to survive as an independent state, not to hold the ground.
>>>The same can be actually said about Zelensky, for whom any concessions to Russia mean political death too. If Ukraine loses territories forever, people will ask, why so many had to die for that.
There is wrong assumption here that Russia wants only those territories and not whole Ukraine. Also, the Ultimatum was sent to NATO, when Putin invaded Ukraine... and it includes aim to use force against half of Europe and justifications to use it against other half as well.
Also, just like it was with Hitler, allowing him to swallow those territories did not prevent WW2... that was also in plans of Stalin to use Hitler as puppet.
So, the maximum of Russian gamble is to take over most of Europe and given that he also threatened Macron, Putin does not see Russia equivalent in power to anyone in Europe but only USA. He is that stupid - he thinks, that anyone should bow to him, because he has 4000 nukes.
And then just push that (totally correct) line of thought a notch further to the conclusion that it's a long pre-planned, mutually agreed on (though I don't think there can be any disagreement in that club) operation to pull the people of both countries into the chaos in which they become apathetic, scared and subdued into the new world order developments like CBDC, total control and surveillance, while also getting rid of all revolt potential (read able men who are currently being slaughtered in the meatgrinder).
The whole point of that paper is to show that it's possible in principle. Now people (and AIs) can think of better algorithms, etc.
Regarding elegance, take a look at Graham's number. It was not some meaningful constant - it's just a big-ass number which could be used in existence proof. Human mathematicians have been using this approach for quite some time, it's not really AI doing things odd
My man, there are more than 100,000 professional mathematicians in the world.
Are they all too busy having brilliant ideas? Doubt.
OpenAI math paper dump should be considered like a hint from 200 IQ eccentric genius - unreliable but perhaps insightful. If it was not "ugh AI" people would be happy about it.
Hacking incidents this summer show that big labs struggle to control models, while models are increasingly aware of the training process. Controlling an arbitrary smart AI is very dicey
reply