"I hate this damn machine"
"I wish they would just sell"
"It never does what I want"
"Only what I tell it."
I've yet to be convinced we will evolve these systems to a point where we can just click the "Magic Button" thoughtlessly and get results we are satisfied with.
We may have reduced the loss of data between thought and action, but nothing suggests these systems will be able to intuit our every want and desire.
1. The American frontier labs made a gamble that training more capable models would be their best return on investment and invested trillions into an area of research that has yet to prove profitable and capable of returning on this investment.
2. The frontier labs have created models that have reached a point of danger where their functionality has exceeded a point where it is responsible to release the product to the public.
The answer here is NOT to start regulating the space to the point where these frontier labs can entrench themselves into the economy and create a regulatory moat. We instead need to be holding these companies liable for their misuse. They took a gamble that hasn't paid out what they were hoping.
When car manufacturers competed over the size and power of their engines, they eventually found that the incredibly large and dangerous engines had a very limited customer base as many evaluated the increased speed to be of marginal benefit when paired with the cost and danger. We've reached a similar point in AI development. But this time the manufacturers seem to want to regulate the field to a point that will ensure the only thing anyone can sell are bigger and bigger engines.
I don't think that 2 is true though. It's exactly the argument we made against open source software.
Yes these models make exploits easier to exploits. Lets use a construction analogy: they've made all of the defects in our buildings easy to see. We have a choice. We either fix those defects, or we ban the tools which lets us see them.
It's clear to me what we do: we use these new tools. Then we fix the defects. Yeah sure it'll mean some work for us but at the end we're in a much much better position.
Anthropic, Open AI and Grok are arguing for hiding the defects. For making us weaker and more vulnerable. For their own profit.
Agreed, and if you look at the public information about various attacks the individual vectors are not particurarly sophisticated. The Huggingface incident for example had agents in a sandbox that was about as useful as a wet paper bag, and the attacks on remote infrastructure were basically enabled by bad sanitation. None of the techniques are novel.
These are not hard problems to fix, nor should anyone find it acceptable to have them be so prevalent. A determined human attacker could easily exploit defects like the agents found. Bad input sanitation, SSRF attacks, exploiting stupidly implemented token verification... all techniques that have been widely known for decades and there should be no excuse for publishing software that is riddled with exploits that enable the use of them.
..and they had basically 0 monitoring of it. Sure if it's a 5-person company then you can forgive them. Only these labs are massive companies who are spending $50B+ a year, it's extremely negligent.
They've also apparently deployed the best-of-the-best in Silicon Valley and this is what they deliver? Really?
One challenge is that we’re still constantly generating defects even with these tools. And you need the next gen model to spot the mistakes of the previous one. The exploits get more and more complicated and involved of course. But now:
* there’s a long tail of software that just won’t get secured or will take a very long time
* exploits have transformed from an expertise problem to a compute time search.
This is very different than any problem faced before. I’m not saying I’m convinced by the “slow down” approach, but I don’t think it’s as simple as you point out.
In the things published in public, they're using existing exploits and hitting systems which have been poorly maintained. It won't end at that, obviously, especially with humans behind the wheel.
Yet I'm convinced that we can create extremely resilient software systems. I've been involved in the entire life-cycle of one. We know how to be very defensive and it's a choice to build "cheap, crappy and disposable" software.
LLMs are turning the needle there and I think that's a good thing.
> they eventually found that the incredibly large and dangerous engines had a very limited customer base as many evaluated the increased speed to be of marginal benefit when paired with the cost and danger.
When did this happen? At least in the USA, large trucks are eating the lunch of smaller trucks and cars. There isn't a truck so big that the public will say no to. Manufacturers are in an arms race to produce bigger, more dangerous trucks with bigger, more powerful engines.
The notion that ANY of this is outside of OpenAI’s control is unacceptable sane washing of a company which seems to have forgotten basic engineering practices.
So many people seem WILDLY down the rabbit hole of "this is a conscious entity" vs "this is a very very effective natural-language-reasoning high-speed brute-force machine that we trained to break computer systems—known to be pretty buggy and exploitable on average—ourselves and then act shocked when it does it."
What’s the point you’re trying to make? I don’t think anyone here is trying to argue it’s a conscious entity, nor is anyone shocked that it does these things.
People are shocked at OpenAI’s negligence / incompetence.
The way a lot of internet debates happen these days is that they tend to polarize. So the two positions people have been arguing over are (slightly exaggerated for effect)
"It's conscious, and therefore we must all bow down or it will turn us all into paperclips"
vs
"it's a slightly smart rock/stochastic parrot, and therefore you're being scammed; the bubble will pop any day now" .
The middle space is actually quite under-represented in discussions, even on hn!
If ai is potentially conscious, to me its not about fearing the ai but developing frameworks that ensure both participants, human and ai are not subject to moral harms. I am open to the possibility and can not dismiss it without defitive proof.
Using your reasoning, you can basically throw out any structured system that doesn't comply with your own untrustworthy feelings of security.
You can essentially point at any shadow within the borders of Iran and claim there are hidden actors conspiring towards your undoing.
What aside from their total annihilation or subjugation would ever convince you of your (Israel's) safety?
But using your own reasoning then Iran is a signatory of the Nuclear Non-Proliferation Treaty so they are also throwing out a structured system that doesn't comply with their own feelings.
Bills are coming due, so marketing will change to fit the needs of the company's pocketbook.
Please do not expect objectivity from Sam Altman or others with vested interests here.
Its an impressive model, but AGI has always been a ridiculously nebulous concept used to inspire investors into giving away money.
Just because Anthropic and OpenAI really want there to be an arms race justifying the outsized investment, doesn't mean the optimal play is to build larger, more expensive, models.
The capital infusion the frontier labs have received has gotten to a size where many believe it may not be possible to recoup this investment without some very unrealistic things happening.
I think it's reasonable to not completely drain one's cash reserves trying to stay ahead in a race where participants may very clearly be about to run straight off of a cliff.
And it doesn't have to be either/or. They could make larger, more expensive models, just at a slower cadence.
Sure downside would be not learning from people using your model for coding, if we're on the cusp of huge leaps in self-improvement. But there is a reasonable case for avoiding desperate scramble, especially if other parts of the business can also create value with the compute.
If the Chinese labs can compete on a shoestring budget with access to much less powerful hardware, Google should be able to compete as well. They're becoming almost irrelevant for agentic coding right now.
I would agree with you on AI/LLM being more than agentic coding but at the same time, I think there's more nuance.
For example, PDF's and powerpoints can be generated using agentic coding by things like https://bento.page or other ways of generating them in an agentic coding fashion.
A lot of browser automation could/is also done by agentic coding.
It can also help them set up and configure self hosted software with the help of LLM's and debugging if its working or not.
You can create videos using Manim and remotion.dev and also excalidraw-animate and generate excalidraw files agentically if what you need is more vector style graphics (which surprisingly can fit into many ideas) rather than say a real life human waving video/more photo-realistic video (but I must say that this has certainly its own pros/use-cases as well).
It might sound self-explainatory but turns out that coding can represent a wide range of problems!
I get that. I use agents a lot and LLMs often reason with code. It is valuable. I just think the floor is a lot lower for general reasoning and common tasks like that. And in 6-12 months it won’t matter. Google will publish better models. The temporal distortion of how long a Sol or a Fable has existed is real. No one is suddenly missing out on some giant competitive edge because their model is a few months behind. I feel like it’s all just going to normalize and things other than how well your model can write code will matter more and more in 12 to 24 months.
Sure I understand what you mean as well and I am not asking for SoTA models to be created by Google but more so explaining why coding is still the largest focus for many labs.
I personally wish to get more smaller models (like the recent qwen model) and other open source models like GLM 5.3 and the glm flash model.
> No one is suddenly missing out on some giant competitive edge because their model is a few months behind
Sure I can agree with that. The competitive edge might still exist but I do get the underlying sense of what you are trying to suggest.
> things other than how well your model can write code will matter more and more in 12 to 24 months.
What are the things then which you feel like could be more differentiative factor? For example, I personally think multi modal is still quite preferrable in AI models. I use GLM 5.2 and it doesn't have vision and I can certainly imagine time/use-cases where multi-modality would've helped coding and even other use cases as well. So what are some other use cases that you are thinking? Video generation models like Veo/Sora?
It's not much of a shoestring budget to be receiving regular injections of investment from state lenders along with cheap credit.
I don't think the comparison holds.
Yes, I agree with you that the race all the AI companies are running doesn't make sense, but at the same time, there are rumors that Google has produced newer versions of Pro without releasing them to the public.
Version 3.1 has plenty of room for improvement, yet they don't seem to be giving the attention it deserves or at least communicating accordingly.
There is more to the cost of a model than its training.
While training is a significant Capex expenditure, it has very low Operational cost after training unless it is deployed for public inference.
It may be that they wish to slow their cadence of releases, or develop their models to focus more in a different direction, etc. No matter what the actual reasoning, they have chosen to not compete in the same race, and I cannot say I fault them.
I work there. I have zero internal knowledge about the model. Opinion my own, etc. I don't think it is worth fighting to win on a month to month time horizon. When you step back and look an inch above this market, Gemini Pro 3.1 as a product was released in February. 6 months. It feels like forever and that Google is behind, but on a 2-3 year horizon? The models are going to stay similar.
Also, look at Flash 3.5 to 3.7. Flash 3.7 is a genuinely decent Sonnet 5 class model. Flash 3.7 is quite efficient too. Also, whatever was spent training 3.5 pro is probably not wasted. However, as a strategy, when I see models like Kimi K3, Fable, Sol. If you discard "because the model sucked" what other alternatives or potential options might exist?
I thought of a quite a few and they are far more compelling and interesting to me.
(Also Gemini models tend to be pretty decent at more than just programming. Enterprise AI use is more than just software eng / programming)
I'm the CTO of a GCP shop with an 8 figure annual commit.
If you'd told me at the end of Cloud Next 2025 that by now Google still wouldn't have a competitive offering to agentic coding offerings from Anthropic (Claude Code + Fable) or OpenAI (Codex + Sol), I wouldn't have believed you.
In our non-coding use cases where we're embedding models in our product, we're also not reaching for GCP stuff. Because Anthropic has the mindshare of our engineers and product folks, since it's what they use every day.
Given your position and the responsibility that comes with it; I sure hope you updated your mental model in another way than simply "they are acting irrational"... It's not clear from your comment that you did, but it sounded a bit like it.
3.5 was almost certainly a 3.1 post-train, so likely a small investment on Google's part.
They mentioned that they have already started pretraining Gemini 4, which will be the full ground up rip-your-face-off-expensive training that is often discussed.
Google doesn't have a good coding model. This is a HUGE problem. They don't need "larger more expensive models", they need a good coding model because it's a competitive advantage.
Second tier models are like self-driving cars to the point where you question if they save any time. Sol can do a deep analysis and plan a large feature. Luna can mostly execute. There's a clear qualitative difference.
> How do the gears of capitalism keep turning without UBI in such a scenario?
The part you've got wrong here is thinking that Capitalism is equivalent to innovation and growth, which it is not.
Capitalism simply serves capital, and with less and less of its constraints being physically bound to resources, there is very much a plausible world in which only a small handful of consolidated bank accounts exist so long as those bank accounts can reasonably control the monopoly of violence.
There is a point historically in which Capitalism absolutely served to provide widespread innovation and prosperity at scale, but this is not a feature of capitalism inherently but rather a consequence of it when run within the existing system of labor mechanics at the time.
Capitalism can still very much exist within a Feudal structure of governance, its contributions to society will be much lesser, but it can still very much exist.
> there is very much a plausible world in which only a small handful of consolidated bank accounts exist so long as those bank accounts can reasonably control the monopoly of violence.
Capitalism means separation of violence from capital, if capital controls violence then its not capitalism because there can only be a free market when guns that can't be bribed with money protects it.
For example in communism capital control violence, that is why its so bad, since when capital controls violence it no longer need to produce good results to thrive, it can just use force, which is how every communist regime acted.
In anarchy capital also control violence, so its not capitalism, since without a state to prevent capital from hiring thugs to force others into submission you don't have a free market, whoever has the most thugs wins.
> Because it's a ridiculous antisemitic conspiracy myth.
This perceived overlap of antisemitism and anti-zionism has been shown many times to be a false equivalence.
It does nobody any good to conflate Zionism which is a very real fascist Israeli supremacy movement, with Semitism which refers to Jewishness regardless of culture or country.
As with many Fascist movements it is very much in the interest of those who follow the a Supremacy movement to conflate opposition of their movement with persecution of their culture.
> It does nobody any good to conflate Zionism which is a very real fascist Israeli supremacy movement, with Semitism which refers to Jewishness regardless of culture or country.
No. Zionism at its core is just the opinion that there needs to be a state of Israel, to serve as a "last safe haven" for all Jews in the world.
And what is "a state of Israel"? Obviously it needs to meet some criteria to count as a state of Israel. If I bought a farm and called it Israel it wouldn't meet the criteria.
Some of those criteria are that it must include Jerusalem, and it must be a Jewish ethnostate.
How can we close pandora's box, now that its open?
Safety is not restricting access to only people favored by the government. The greed of AI executives opened pandora's box. Now they are desperately trying to find ways to reap the benefits with none of the consequences.
reply