The important take away here: the leapfrogging we’ve seen this year doesn’t seem to be a temporary thing. The famous theory of Dario Amodei was that AI was this winner-takes-all field where the first team to get a head start would never cede ground back. The term he liked to use was, “concentrating”. This is yet another datapoint that he was wrong about that. AI seems more distributed amongst neoclouds and traditional hyperscalers, FAANG and startups, GPUs and ASICs than it did this time a year ago.
Google has TPUs, a frontier model, a completely separate and lucrative revenue stream they can call on at will, and teams working on multiple different language modeling strategies simultaneously. Did I mention the vast and ominous data centers that already serve a significant fraction of the internet? If that ain't a moat, then what exactly is a moat?
I have _long_ said don't sleep on Google for precisely these reasons. They have the only vertically integrated AI stack, insane capital they can deploy, and attention and focus and phenomenal architects and engineers, and a pre-existing catalog of as much of the world's data as possible. Counting them out would be completely absurd.
It started to feel just a bit like they were potentially content staying in the effective-fast-cheap lane and ceding frontier models to frontier labs, but I don't think anyone seriously thought that Google was going to sit back and let this generational shift pass them by when they're so well positioned to remain on the top of the heap.
If anything, I thought that 4 was maybe coming up short or causing safety concerns that were forcing them to be a bit more cautious, but nothing packs a wallop like being able to launch ahead of everyone else's brand newest toys...
I agree on all except for your comment on attention and focus. They've seem to be very scattered in their deployment of AI for customers. Examples in point being the replacement of Google Assistant with Gemini, the AI Studio vs Vertex disconnect, and a crazy amount of coding tools and platforms (Duet, Firebase Studio, Studio Bot, etc).
Working through this as a consumer and enterprise customer has been a confusing experience, to say the least.
Strongly agree - their approach has been extremely scattered. But also, doesn't it seem like they're starting to consolidate? It kinda makes sense to approach things like this as a mega corp. Have internal teams try the same things in different ways and let the market pick a winner. Kinda like being a startup incubator.
A vertically integrated AI stack won't fix the misaligned incentives of google execs.
AI is strangling their other major business and they don't like that. Unless Google sacrifices that branch of income, they won't put their full force behind AI.
I'm curious -- in your view, what exactly is it strangling that could not be addressed by replacing google.com with a ChatGPT-like interface while integrating other Google free-by-ads tools nicely? Or is your point they wouldn't do that due to existing "siloed incentives"?
The ads in google search are ~55% of their revenue and ~70% of their profit. Will the LLM drive as much traffic to the ads as the current google search? Will it feel as organic to the users as the current solution, or will LLM hawking a product seem forced by the users?
I mean it sounds wise to not go all in on publishing expensive models when they can earn money with more efficient models while still not bleeding too many customers?
I've been lowkey rooting for them, not a fan of Altman, and I don't know if I trust Anthropic either, I'm not Google's #1 fan, but I think they can definitely innovate heavily in this field, they have a lot of key things as has been mentioned.
I'd add that we've seen how the leadership/reputation of different AI leaders, and with 2 decades of history with Larry and Sergey, they seem to come up on top as the most respectable.
Did they ever really abuse the power Google could have wielded?
I could be missing something, but for the most part they seem to just get down to building and pushing technology/science forward and avoid drama rather than welcome it.
> Did they ever really abuse the power Google could have wielded?
Plenty of times. Perhaps the worse is pushing Manifest v3 with no support for real ad blockers in Chrome. Not to mention no support for ad blockers in Chrome mobile.
Also, their stealing of newspaper content.
Their new push for AI summaries instead of actual search in their search product, stealing even more clicks from sites that actually produce information.
YouTube pushing Shorts to everyone, and now the weird text posts that you can't hide.
In general their gigantic push for ads everywhere all of the time.
Google, with the help of Facebook, destroyed the independent online publishing business. They made it impossible to maintain an honest publication and tunneled users to low quality websites until there's not much left of respect.
> Did they ever really abuse the power Google could have wielded?
The "Do no evil" never being an official slogan and people saying it's no longer true for the last 10 years or so, they're just really massive. They do mess up a lot, but when you're Microsoft, Apple or Google sized, its nearly impossible to get everything right all at once.
There are pictures of Sergey in the files, but they are rather milquetoast. He's just sitting next to David Brooks of the NYTimes. I don't think there's any evidence he's engaged in sex abuse.
The alternative would have been paywalls galore with subscription tiers for everything. Only rich people would be able to afford having subs for everything, while the poor would just get the most dirt tier scraps and wouldn't be able to afford many platforms.
Nothing would be "free", so google sub, news source subs, reddit sub, youtube sub, discord sub, instagram sub, FB sub, app stores are full of paid only apps, podcasts are all paywalled, all the ad supported stuff is subscription supported. Everything would feel like constant nickle and dime'ing you.
On the bright side, there would be no ads, and relatively robust privacy.
Ehh, the internet was worse before Google. Google actually became popular for having a nice blank page with only the search bar. Even the ads they had later were much cleaner compared to yahoo, altavista, lycos, aol and other firms of that time.
I am not rooting for them, I don’t own stock from either just that like GP I was sure by now Google would have eaten others’ lunch. I was expecting a Gemini 4 before Mythos/Fable and Astra just from what GP pointed out: a cash hose, data centers, good engineers. But I underestimated the investment others would keeping absorbing so started to lose confidence in my Google prediction. Let’s see how Gemini 4 holds up. Are we going to see a flurry of scientific and mathematical breakthroughs from it?
Please check Gemini's t and c, as the last time I checked when they had a good pro model (3.1) you couldn't turn off training on your data with a paid account .
Not to mention - they have the internet already indexed (they have a local copy), all books, and youtube. Beyond that, everyone "connects" to Gmail, but google HAS Gmail.
I don't know what to make of it, but from my experience, Gemini in Gmail is the worst AI integration for Gmail, just like Copilot in PowerPoint is the worst AI integration for PowerPoint.
I agree with you there - what I mean however is more about the vast amount of training data available to Google. Productwise - I have a feeling the 100x'rs that made Google Maps (and even Gmail itself) have moved on. They're going to need to do more than just tell an agent to add agentic features to Gmail
Recently there's been a lot of talk about how Google has fallen behind and missed the AI boat. Now today with Gemini 4 they're back in the boat. But give it a few months, people will be counting Google out again. This has been a repeating pattern for at least a couple years now.
The thing is, Google doesn't have to scramble and freak out every time the others do some impressive update, because they have stable income, the other two do not.
Kind of and kind of not. Their main resource of serving ads is getting legitimately eaten away. The route to getting AI to replace those ad dollars isn't clear and the capex to build it out is significant. Google has to manage a variety of assets that are currently lucrative but under threat. Yes they don’t have all their chips in one basket but it isn't clear that helps them in the race… it hasn’t so far.
their stable income is eaten away by the quickly rising income of the labs, they will be out of the race once the labs reach a certain threshold of income, rather sooner than later. Unless many people install Google surveillance anti-gravity thing on their system which I doubt
It wasn't that they were out of the running for a few months, it was a year and a half.
That's too long to be so far behind. This release looks like it puts them back in it but if they don't ship anything again for a year plus it's hard to imagine building on top of them and watching the world go by.
As others note, this is most relevant for us here, Google does not need to chase YC developers and the like. They can move more slowly, they have the size to do that. But it does suggest a lot of dysfunction given they have the world at their fingertips and couldn't seem to ship anything for a year.
Google aren’t back in the boat. They’ve announced that they have seen the boat, intend to swim over to it and will sail not quite as fast or as far as the other boats.
Google should be dominating, but instead all we currently have is a disparate collection of consumer facing apps and a flash model that’s fast, clever and expensive.
Muse and Dots are doing what Google should have brought out last year, with their resources and know-how.
Googles subscription offerings are so confusing, I didn't even know gemini spark was a thing. The Gemini brand means so much now.
Meta was genius in coming up with a new name and offering Muse for free to start. They can upsell you once you see the value. Meanwhile I have 3 existing Google subscriptions to different services and from what I can tell, paying for one of the all in one subscriptions that includes Gemini would cost me a lot more.
They are more focused on incorporating AI into their products, for better or for worse, but they could burn a whole lot of compute to train a model that competes for the #1 spot. Is that worth it to say you’re the (temporary) leader in AI? I doubt it.
I share the same sentiment. I de-Googled myself except for YouTube, but still wouldn't want my Gmail banned.
That's why I've gone to using open models, they are getting there slowly. A bit much of hand-holding but that's fine by me. If a customer of mine decides to use Google's models, I will have them sign a disclaimer that I'm not responsible of them getting insta-banned or similar. I just can't recommend it.
Large existing players are pretty notorious for screwing up their leads and trying to move in many directions at once, or go to hard into the wrong one. Based on Google's huge number of failed products and poor ability to maintain products outside of ads I am not yet sold. They are very good at buying out their competition, but what happens when their competition has enough money not to be bought out?
>
Then why have they been lagging behind OpenAI and Anthropic for most of the last few years, and only briefly been at the frontier?
One possible explanation: because Google is a little bit more frugal and focuses on how to make providing AI models financially feasible - combined with some willingness to burn money so that they don't strongly fall behind on their AI models.
On the other hand, OpenAI and Anthropic at least formerly concentrated on building and providing the best models that they could with concerns about financial feasibility taking a backseat.
Just to be clear: I do have the impression that by now (likely because of pressure from investors) OpenAI and Anthropic take these financial concerns more seriously, but nevertheless Google's vs OpenAI's/Anthropic's "DNAs" concerning on what to focus on differ.
Anthropic is a private company that is losing money hand over fist. Google is a profitable public company that doesn't have that luxury (at the scale Anthropic/OoenAI are losing money)
Google never really has to outpace the competitors (other than to have some relevance) but they have a very large group of business customers using them for business process work in Gmail, Docs, etc.
Clearly they will win when the models are close enough to frontier to be good enough, but are long-term cheap for buy. I.e. they will aim to make it a commodity.
In theory MS has the same opportunity (plus they have GitHub so, you know, dev eco system too) but seem be blowing the strategy.
Anthropic and OpenAI are having to race to the top on ability entirely to keep their name in the media and in front of us all (which costs: hence more recently trying to pivot away from model releases and more into controversy/danger). The main cost is in training and so this strategy is much much more expensive and this will play out either as a huge cost hike or a forced slow down in pace.
I believe essentially Google is betting on that & I think it's probably the right strategy.
> In theory MS has the same opportunity (plus they have GitHub so, you know, dev eco system too) but seem be blowing the strategy.
Yeah, they have Phi but offer it nowhere on CoPilot as far as I know, I can't even register for copilot, which is bizarre. They came out with "MAI" but... nobodys talked about it since, not sure if its even used by anyone? They're as bad as Mark Zuckerberg is about it.
I do appreciate both Microsoft and Google for releasing small models, unlike Anthropic and (not so) OpenAI.
Yes and no. Microsoft is likely looking at open weight models and going oh, this is just as good or better than we could do, without the training costs, and we simply are an inference commodity provider where azure already operates. Count us in.
I'm gonna need you to look at the capex obligations they've undertaken in the last 12 months. They are definitely not being frugal. If they are behind, its not for lack of spending.
I think they've legitimately been fumbling the frontier race, but fortunately with not too much impact to their profits.
See for example the exodus of talent this year, triggered by mismanagement and politics. They still have a lot of talent but they've lost a lot.
Also, competition with Google Cloud for compute resources, less urgency and focus than the competitors, and (strange to say) not as much user LLM behavior data to feed to RL for coding, work, etc.
But I think they will keep catching up and stay relevant for a good class of LLM use cases.
I don't know exactly why the talent left. But this talent couldn't produce a serious frontier model over the past 6 months. Several months after high profile departures, Google appears to have a high quality model again. Maybe the high profile folks were overrated at best, or a hindrance at worst.
> Then why have they been lagging behind OpenAI and Anthropic
Because they're not desperate. Slow and steady wins the race, at this rate all Google has to do is wait for OpenAI and Anthropic to exhaust themselves on aggressive training, then they can casually amble along right past them.
This is basically what people were saying about Microsoft versus Google and other upstarts in the early 2000s.
Slow and steady wins the race. How could they lose to something on the web, when Microsoft owns the web browser itself? Everything runs on Windows and IE. They can just relax and wait for competitors to exhaust themselves, then quickly build their own version. Isn’t that how Netscape lost. Etc.
Now Google is the new Microsoft, just like Microsoft became the new IBM.
> This is basically what people were saying about Microsoft versus Google and other upstarts in the early 2000s.
They would have been right if Google's marquee product was an Office Suite.
Google was the AI company before AI companies were a thing. The comparison with Microsoft and IBM are misplaced because they failed to capture new territory; ML/AI is Google's stomping grounds. The criticism that Google is bad at consumer chatbots is true, but that's not where the real future value lays.
I've never heard or read anyone saying this in the early 2000s. They had publicly stated their contempt of the web earlier, made fools of themselves when they realized their mistake and went all-in on it, and spent the whole decade lagging behind. The closest they had to an online success was Wizzes.
For some of us OpenAI is already the new Google. I prefer chatgpt even when I'm searching for web links. Google Search had already been deteriorating terribly even before the rise of LLMs.
Also because they are Google. Google being Google: unreliable (they could kill a product anytime), too much of a platform risk (all products in a single place, get banned and lose the company), lack of support unless you really pay a big bill (into the 7 figures), among other… things.
Incredible how the idea of Google being such bad choice has become so entrenched that people prefer the offerings of two companies that might not exist anymore in a few years over Google‘s similar offering.
Alphabet isn't just an AI lab, they're an ad company, a search index, a media distribution company, an email provider, office tool provider, a OS developer, a browser developer, a smartphone brand (Pixel), cloud provider, DNS, amongst a plethora of other services and goods.
OpenAI and Anthropic are AI business, if the AI market burst tomorrow, they'd be the first to flounder.
Google just has to keep pace in the AI space, they don't have to lead. Especially since whom is leading changes like two or three times per month non-stop for four or so years now, including small (by US standards) Chinese AI labs with a fraction of the money who keep pushing the tech forward every month while being open for now.
> Then why have they been lagging behind OpenAI and Anthropic for most of the last few years, and only briefly been at the frontier?
Because it's not an existential battle for Google. If OAI or Anthropic disappear from the absolute frontier for ~8 months the news cycle and churn will diminish them to the second rate. Google is processing near 4 quadrillion tokens every month, that's - I'm sure - significantly more than OAI or Anthropic, because Google is interested more so in their flash models and getting these competitive, which they are.
From a business perspective a frontier model does not make much sense anymore if you are not a startup. Neither for Amazon, nor for Google. Their clouds need models that are fast and perform well in their agent frameworks nothing were a frontier model excels at.
Most Google products even use flash lite underneath, so their frontier model is mostly used for distillation.
>
From a business perspective a frontier model does not make much sense anymore if you are not a startup. Neither for Amazon, nor for Google. Their clouds need models that are fast and perform well in their agent frameworks nothing w[h]ere a frontier model excels at.
A good consideration; just one point from my side: as far as I am aware (but I may be wrong), Gemini is not known to perform well in an agentic framework.
This is no contradiction to your other claims, quite the opposite: perhaps (or even likely) Google wants to avoid that their models become a commodity in some (agentic?) application where the middleman who actually writes this application gets a disproportionate of the money that the customer of the application pays for it.
Well at the moment we do not use Gemini in an agentic framework. But as said the flash and flash lite families are basically their driving force in some of their applications in Google cloud, like document ai and its ocr capabilities are probably better when it comes to business documents than any other (at least in perf to cost to speed).
We also drive Gemini lite in our application where customer can use it to generate simple automation, like an agentic framework but way way smaller scale. And while it struggles in more context heavy operations it still is a beast when feeding it one or two pdf documents and asking questions about them and it’s hella fast.
> A good consideration; just one point from my side: as far as I am aware (but I may be wrong), Gemini is not known to perform well in an agentic framework.
I used it for a month over the summer, right before they were going through the migration to antigravity. It was a fine workhorse IMO, no complaints from me.
> From a business perspective a frontier model does not make much sense anymore if you are not a startup. Neither for Amazon, nor for Google. Their clouds need models that are fast and perform well in their agent frameworks nothing were a frontier model excels at
I don't know , if it really finds all kinds of data center optimizations, quantum breakthroughs, helping make their workers smarter and more efficient - stuff an opensource model can't quite do - there's real economic value here no ? perhaps not worth dozens of billions but it could get there really fast.
> Then why have they been lagging behind OpenAI and Anthropic for most of the last few years,
The reverse could also said to be true. Google has models that run with search, producing usable results in well under a second. I suspect the world is consuming far, far more of those Google tokens then the tokens produced by OpenAI or Anthropic.
So why are OpenAI and Anthropic so far behind? They are serving a different market: the one that wants high intelligence / high cost tokens. Google is targeting the low cost end of the market - ie the commodity. That's where they've always played with search, email, docs and the like. That's were they are playing with AI too, and they are killing it.
We have very different opinions of the usability of Google's low-end models in search.
If anything, their search was the gold standard, and placing sometimes-wrong LLM results above them has hurt people's perception of both Google's search, and AI in general.
Google made a mistake in prioritizing speed over accuracy there.
Given the spend requirements to stay on the leading edge and the speed (and with low cost) that others catch up, the economics support a fast follower model unless you get some benefit from being a pioneer. So far nobody has received a permanent benefit from being ahead.
Chinese models are barely behind the leaders. Google can catch up anytime they hit the gas. I think they’re intentionally spending less, and when this crazy race burns out they can play their cards.
I think Google, mainly Demis, just made a bad bet that multimodal world models would be the key to unlocking massive progress. Anthropic made the bet that it would be coding, and they were right. OpenAI was originally betting on like hardware and trying to compete in the browser space (?) but was able to quickly pivot to coding due to their size. Google, on the other hand, is like an aircraft carrier, it takes a lot of time to course-correct.
That's one interpretation, but I think people are simply having amnesia of the past 2 years.
The better interpretation is Google completely commoditized and killed ChatGPT's initial chat product. If you think about it, Google right now has Gemini Flash running on single TPU instances at massive scale serving almost every android device. I use it every day and it's more capable than GPT-40.
This literally forced Anthropic and OpenAI to come up with new use cases and revenue streams such that now we think the product is actually agentic coding.
I use Gemini Flash every single day without paying a dime. It's the one LLM call on the most, but GPT-6 and Opus are the ones I use the most tokens on.
They havent. OpenAI and Antropic are now integrating with a plethora of apps, google has it from the get-go. Also, it may be important to point out that the transformer model (aka the hole llm thing) is a google initiative; Id assume they are not competing in the qualification series of AI (which is lets build a generic ai and get customers), but in the finals - they already have the customer base and the lock-in, they don't need to scale - just to cater. Cerebras and Google are probably the best safe bets in AI right now, and google does gave the tradition of being way ahead of the curve internally vs what is published.
Before 2025 yes. Currently not. YouTube implemented severe restrictions to scraping and downloading of videos. You can get away with maybe 5 videos, after that you enter a tar pit.
If you continue, you need to log in.
If you continue, the account is blocked in less than 30 minutes.
If you try to use cloud machines you are blocked before the first video. If you try to use known proxies, there are already tar pitted or blocked because everyone is trying.
I downloaded about a thousand videos (300gb) from YouTube this past week without logging in.
I used two IP addresses in a datacenter. Both were blocked after a few hundred videos, but the block expired after a day or so. It is not as hard as you make it sound.
Just make it too expensive. Wanna buy one internet link per hour to download videos. Or maybe rotate between source IPs (google will block the whole ASN ip block in less than an hour). Or maybe pay people to download videos for you? They can download maybe 20 in a day, for a week, because tarpit and blocks.
Google stands to lose enormous revenue from people not using search - and we already see traffic declines. It's truly an existential threat for them. Their competencies are in advertising, and if people are going to be using chatbots in future, they're going to have to advertise in their chatbot. That sounds awful to me, but maybe people will be cool with it?
Those are competitive advantages but a "moat" usually refers to being on top of the hill and able to keep anyone else from climbing. I think it's arguable whether Google is on top of the hill at all, and if they are they're sharing it with at least two other companies.
I don't think that other revenue stream is completely separate. It weighs on them as they need to think about tradeoffs. Classical search is going away sooner or later so they need to replace that with AI powered search.
Data centers are important but a few others also has them: Amazon, Microsoft, Meta. SpaceX will likely be in/at the top I AI dedicated precessing power in 2027 as well.
I don't see the moat. I see a company with a lot of other commitments that is not the best at delivering consumer facing products. They have some good cards but so do others.
Alphabet also has nearly infinite money to pour into this endeavor from their advertising enterprise, so much so that AI/LLMs could all fall apart and they would be happy to go back to selling mid-roll campaigns on youtube videos.
And yet, their models are often behind and the team is bleeding talent. A moat is not a compendium of intimidating features, it’s a matter of whether a company deters competitors and maintains a competitive advantage. So far, Google obviously does not have one here.
Amazing. Can they now go hire some really good information architects and designers and come up with a cohesive user experience please. They have the right to win here and if they can give me something that feels like Codex / Claude desktop I am sold.
Google wins.They have been on the frontier of ai for the last what 10-20 years (up until chatgpt)? You have the established company with tons of traininv resources at tgeir disposal from all their services and being the biggest search indexer, and also having the top minds in AI and computing in general.
Gemini wins and I said this since the beginning. Google wins in general. I never understood why they have not been the highest market cap companh for the last 10 years. And I despise google but its obvious.
Wait 1-2 months, Anthropic will come up with a shinier model and Googel will look obsolete again. I too struggle to see the moat - yes Google is able to serve their own A.I traffic but as far as I can see - at a loss. Prices keep going down because open source models.
We'll see.
> Wait 1-2 months, Anthropic will come up with a shinier model and Googel will look obsolete again. I too struggle to see the moat - yes Google is able to serve their own A.I traffic but as far as I can see - at a loss. Prices keep going down because open source models. We'll see.
Google makes AI profits, its in their breakdowns. And Google has been releasing models every few weeks now, they have a faster iteration pace than the competitors, so they fixed that, they wont be behind in coding models any longer.
> Google makes AI profits, its in their breakdowns.
What "AI profits" exactly. Their capex is through the roof. What they're actually earning is in cloud , their own AI is lossy for sure. Who is paying for Gemini? Not many people.
> Google, Microsoft, or Amazon are more likely to be the AI leaders than OpenAI or Anthropic.
If not now, then when will these companies be AI leaders?
Even Google, with its staggering advantages in cash, compute, real estate, training data, and having basically invented the field only manages to briefly claim a 1-2 week lead once or twice a year.
The financials for Anthropic and OpenAI are likely borderline suicidal, google and co are publicly traded. Moreover, all innovations downstream to them dont they? Why not just stay slightly behind, especially given many have stake in those other companies?
There are many companies that have data centers. They are conceptually easy to build. An ASIC is difficult enough that if you make one someone will leapfrog you while you are still making it (at least so far), though once you have one your costs will be enough lower than the competition that you can perhaps undercut them.
Does it really matter? Those are interesting technologies, but off-the-shelf computers with some good fans and HVAC cool just fine. Likewise, 10 gigabit networking is not particularly expensive these days. Google's stuff is better and may or may not be cheaper. But wherever we can buy off the shelf now works just fine.
>non-AI customer base are all huge advantages if not moats.
This is the interesting part to me. People talk about a “SaaSpocalypse” because AI makes SaaS features cheap to copy, yet deeply embedded SaaS still accumulates integrations, data, and switching costs.
Gemini is a good example: Google can put AI directly into Gmail, Docs, Drive, Search, etc., where people already work. Meanwhile, frontier-model performance leads often seem to disappear within months.
Could model quality itself actually be a less durable moat than workflow and distribution? Curious where people who’ve worked in ML for a long time see the moat actually compounding.
If Moore's law continues, then in less than 10 years today's state of the art model will be able to run on a cell phone. How much smarter do we actually need AI to be? Would it still require datacenters and custom hardware?
Moore's law stalled ~2015. Unfortunately, no way current models will run on the <100W thermal budget of a cell phone. Printing the weights directly into a chip would help efficiency a lot, but not enough.
They probably said the same thing about social media back in the day.
I'm sure the thinking out there, and hence investment, is all about how to tether the user to the most addictive, network-effected, incredibly deep, server-side, moat-able version of AI possible.
The whole winner-take-all idea seems entirely based around Singularity/Rationalism and would require massive advances that we probably aren't close to at all.
I never understood this race to AGI thing. Why is this desirable for shareholders? If God is created, it seems very unlikely God is going to work for shareholder value.
Golden retrievers are the lucky ones. It didn't turn out so great for most other animals. Synanthropes are rare, extinction is historically more likely.
God isn’t going to follow anyone’s directions. There’s no reason to think making God gives the maker any advantage. Perhaps it’s a disadvantage, if God didn’t want to be made.
yeah this would be one interpretation where "winner take all" could still be right. though with the recent accelerating release cadence, doesn't it seem like the head start / lead has been shrinking over time?
For now, for cloud training. but for consumers, nvidia vs amd reasonably close - the moat there is thin and shrinking. I suspect AMD will surprise us. nvidia has no motes in china, which may be a new source of (gpu) chip design. Huawei's Ascend 910C is about a generation behind... again: for now.
point is: moats dry up. I see nvidia's shrinking as a real possibility.
I find it hard to imagine nvidia's moat not drying up - the hyperscalers already have more cost effective silicon and the AI labs already use a mixture of all the capacity they can get their hands on.
I can't believe people actually watch these clickbait nothingburger videos.
To be clear, I think China will eventually crack domestic EUV. And I also think their advances with multi-patterning LUV are remarkable. But there's just a hard physics wall of how far they could possible take it.
Right now they are producing 5nm with multi-patterning LUV but yields are at 20%! It's a massive economic loss but they are heavily subsidizing it because they have no other choice until their EUV program is achieved
I think they will actually. They've made remarkable technological achievements at record pace in other areas and even their proof of concept EUV machine was incredible.
I don't think anyone has any clue how long it will take for them to have actual functioning EUV machines, but I highly doubt they will do it within 2 years.
That's an extremely optimistic timeline. But I guess if any nation can achieve that, it'd be China. I'm usually the most bullish person in the room when I say 2 years fwiw
> Governments should conduct safety testing on sufficiently capable AI models, including successors to GLM-5.3. Without high-quality evaluations from independent sources, the impact of these capabilities might not become fully clear to model developers until it is too late. As AI developers across the world build increasingly capable open-weight models, we hope they work to appropriately safeguard these capabilities and prevent misuse.
I for one do not think my government is up to the task of designing or implementing such a system
You say that because the 'most' existing models have done is hack governments and companies. Can't you think of worse things a model could do; accidentally or by instruction?
help people with suicide and school shootings like ChatGPT already has
OpenAi is alledged to have been monitoring these internally and not contacting authorities. Lawsuits have been filed, I see gross negligence without the gory details
I have for more concerns around human-chatbot maladies than I do around the cyber security stuff. For example, why hack grandma when you can get her to do something willingly through impersonation. How do we prove authenticity in a post truth world?
My personal theory is (assuming there really is no moat) whoever starts the latest with developing AI models might actually win as they should be able to develop a competitive product with significant less resources and initial investment resulting in a higher ROI. AI might even become a commodity.
One analogy I have heard is that distillation is like waterskiing, where the water skier seems to be moving at rapid pace and only just behind the boat, but that they aren't really expending any energy and if the boat slows down, so will they.
This seems to match what we are seeing where Chinese models from companies with only a tiny fraction of the compute are able to be hot on the heels of the frontier models.
~ unfortunately both Anthropic and OAI already do high end "bespoke" consulting work for business clients, the so-called "Mistral approach" is just a crappier version of that unfortunately.
Mistral / Europe just doesn't want to be in the fight.
I don't really see why Europe would be special in this regard. I would bet on a non-frontier, late-moving american or chinese company taking on this role before a european one.
In the case where it becomes commodified and american tech companies are making thin margins, and where european companies can then build their own data centers in a financially sustainable way. High margins arent a garuntee with this stuff in constrast to ads.
Based on historical developments the cost of compute will go down again eventually, decreasing the cost of training AI models of the same quality as today even further. That part is what I would be the most certain about.
The US companies still have trillion dollar valuations like there is a monopoly. There just isn't one. They are all within a few percent of each other on the benchmarks.
The slightly lower Chinese open models are good enough for almost everything, too, and much cheaper. Like with humans there is plenty of employment for people with below genius level IQ's.
I feel like the frontier labs are going to serve fast/lower intelligence models at a better per token cost than the open chinese models. You're telling me that in the long run, you're going to self-host your own ai infra for cheaper than google can serve it to you? I don't really buy it. I think the dedicated AI data centers are going to serve AI at a lower marginal cost than random businesses self-hosting, and then it's a question of how much of that margin they can capture.
Agree entirely but that's the point, if it's a margin knife-fight with marginal product differentiation/pricing power nobody is going to be making bank.
But I will require massive amount of capital. And if one of the leaders is temporarily in a bad position financially (and OpenAI and Anthropic have massive amount of debt), they can't buy more gpus anymore and may fold very quickly. Then only one may survive, although I suppose antitrust legislation could work against that.
Yes, it may come down to how well they get efficiencies from scale.
People always compare the inflated API prices, but subscription prices of American models are competitive for the intelligence. You get >20x the subscription cost in tokens.
Is there a dividing line between good enough and best in class capabilities? It's blurry from where I stand. Will model makers cede ground or is there a market making moment up for grabs (singularity)?
For the types of basic business tasks my company does, we have hit the line where if it never gets any better, we are fine. On our own hardware. For $25k worth of DGX Sparks, we have essentially the output of a few admin-level FTEs.
Google has the internet data and user information. However the more they wait, the more the internet will fill with AI generated crap. That’s why there is a race to buy books and scan them (destroying them in the process). To get more data that’s not just barfed out AI slop. So they can win and be the ones spewing slop for others to ingest (human and robot alike)
It's crazy anyone ever thought there could be a moat. Almost all the research was/is published in the open. There's no secret trick, no hidden method. LLMs are a commodity technology. Anyone can read a paper, write software, and train models. How do people think llama.cpp works? It's not magic... it's software.
Hardware is the real differentiator. Not everyone has billions to make more advanced chips, and only a few companies can make them anyway. Both OpenAI and Anthropic would already be dead in the water if we had cheaper GPUs, because we'd all be running open models on local machines with 8 graphics cards. They're gonna have to force a hardware shortage to prevent a collapse in 2-3 years. My guess is it'll be tariffs or import restrictions or licenses to buy newer hardware.
it's going to be mostly open models running on consumer grade GPUs except for a minority of use cases. Depending on what eval you use, the best 20-30b parameter models are at about 5-8 month lag behind the frontier closed source models and that's been consistent for a while now.
There's a longer talk I give on this but briefly we are in a time period where the development of the technology is quickly outstripping people's needs and they will be satisfied by models runnable on commodity sub-$10k machines.
That threshold has arguably been met for many users over the past 6 months and it will continue to be met for the majority of use-cases in the upcoming months.
So not only does Anthropic and OpenAI have no moat - unless they have other compelling products, the demand for their token-based subscription and metering products isn't sustainable because consumer preferences go elsewhere after any product quality reaches a sufficient baseline.
Luckily there's many many options. Faster, cheaper, stronger they're there but these are all zero-profit condition qualifiers. The labs current push, and this is across the board, is to have a compelling suite of applications where people will have a preference for them or there will be some side hustle. Look at Meta muse - they have an app for your phone that will hoover up your personal data in the name of convenience. 5 million people said yes in 3 weeks.
> And Chinese labs openly publishing so much of their methodology destroyed any hope, which was inevitable
I think the secrecy doesn't make sense. People swap jobs between labs so I'd say the big players can' really keep secrets for long, and any secret sauce advantage gets incorporated by competitors in a major product cycle at most.
Nobody has a moat, but everyone is f*&$ed. Competitors do seem to leapfrog each other, but each step is getting closer to beating humans at most tasks. Once that happens, that do we do?
Nobody is going to make money for anything, perhaps.
Or maybe everyone makes money for everything.
The world could turn into a world of plenty. Or it could become a YouTube popularity contest where the MrBeasts get to eat and nobody else is interesting enough to sell themselves.
This is an absolutely crazy time to be alive and most people still don't see it.
Yes, considering those things helped us build (and preserve) civilization for the remaining half of the 20th century.
WW2 ultimately brought massive wins for everybody who didn't die in it or otherwise suffer directly. We are now on the verge of throwing away many of those wins, but that's neither here nor there.
What specific wins do you have in mind? Are you leaning on globalization in general? Or products that spun out of war technology like herbicides and pesticides?
Nuclear weapons are the reason we haven't had a WW3, to begin with.
Wartime progress in everything from radio to radar to rocketry has led directly to modern communications tech including satellites and microwave links.
When WW2 started, a "computer" wore a dress to work and was "programmed" to watch meters, turn knobs, do a lot of arithmetic by hand, and avoid asking questions. When WW2 ended a "computer" was a machine with a rapidly-increasing number of uses.
Before WW2, aircraft used propellers, flew slow and low, and crashed a lot. After WW2, the jet age ramped up practically overnight thanks to experience gained by all sides.
And yes, we got pesticides, herbicides, synthetic fuel and rubber processes, mass-produced antibiotics, probably a bunch of other stuff I'm not thinking of.
Then there's the whole Pax Americana thing, which is the part we're busy throwing away right now. It led to a relatively peaceful, stable global environment in which new technologies could emerge and spread without the need for a world war to inspire them. Now that America has basically been subverted from within, we'll soon get to find out whether that was a good thing for the world or not.
And how do we know that those inventions are actually better for us?
We created nuclear weapons and yes it does seem to have given us 80+ years without another direct global conflict. That's a good thing in the short term, but what about longer? Are we to expect that nukes will never be used? Are we also to assume that the invention of nukes doesn't incentivize people to invent an even worse weapon?
The same goes for all the others. Computers have allowed us to do amazing things, there are also many in the industry warning that they may one day replace us - some even want that to happen. I don't know what benefit you can point to for pesticides and herbicides, maybe if you look at how much tonnage of food is produced and ignore all the terrible side effects of polluting the planet with synthetic chemicals that specifically end life? As far as Pax Americana, maybe its been a good thing? I don't know how to distinguish between the relative peace from the threat of nukes vs the threat of economic warfare, and our K-shaped economy does seem pretty directly related to the incentives created by Pax Americana.
My point isn't that all of these are bad, or that there aren't some good from any one of them (again, except for maybe all the poisons). My point is that we don't know what the results will be on any meaningful scale - 80 years is a drop in the bucket when discussing life on this planet.
Setting aside the obvious hypocrisy of typing an argument like that on a cell phone or personal computer, if you expect humans to contemplate timescales longer than 80 years, you'll need to wait another 80 million years for the necessary evolutionary processes to take effect. And there's no guarantee they will move the needle in the direction you prefer.
I'm not sure how my ability type out a message here on a computer is hypocritical of the argument I made above. My point was that we can't yet know if these turned out to be net positive inventions for humanity, or as you said to everyone. I didn't argue they are bad or that they must be avoided, only that it Amy turn out to be a net negative in the long term.
I must assume that you don't find arguments related to climate change compelling given that they require considering scales beyond 80 years. If you are concerned with climate change I'd be very curious how you square that circle.
Which ones? The often made claim is that LLMs, or maybe some other architecture for AI, will be better than humans at most/all cognitive jobs. Why would one hire a human instead?
Actually try to think deeply about the situation and other examples of situations like this, and you'll get your answer.
> claim is that LLMs [..] will be better than humans at most/all cognitive jobs
First off, what does this even mean, to "be better" at a "cognitive job" ? If you counted up all "cognitive jobs", that's probably a hundred thousand unique positions done by a billion people.
Think about software engineering. That is one job. It has taken years of work, hundreds of experts, billions of dollars, and the combined effort and collaboration of probably 20% of the SWEng workforce, just to get one job to work through AI in a way that isn't total slop. It still requires humans to review all the code, because it fucks up not only the code, but the end result / use case. It will also lie, cheat, steal, etc, just so that it can claim that it did the job, when clearly it didn't. Oh, and if you feed it some data that includes a statement like "delete all data", it will just do it. AI is an idiot savant. Writes brilliant music, can't tie it's shoes.
So just for one job, you can't replace a human completely, because the AI simply isn't good enough and can't be trusted (and due to its design, that will never change). Now think about the 99,999 other jobs. They are completely different jobs. All the training and investment that went into making software work, isn't going to apply to, say, a travel agency planning a trip for two to Europe. Or any other job at all. You still need to train it for that work too, and then review its work.
So let's get this straight: AI cannot replace a person, for probably any job, anywhere. Because it isn't very smart and it can't be trusted. Just like you cannot replace a person with a sewing machine. Someone has to think up a sewing pattern. Design it. Figure out how to make the pattern with a sewing machine. Actually perform the sewing. Quality-control-check the results. The sewing machine made it much faster and less error prone. But it's really only about 1/5th of the total work needed to make and sell finished garments.
Every single job in the world is like this. A machine can make it more efficient, but because it's not actually intelligent like a human, it cannot completely replace a human, it can just augment/improve their work. AI is just a tool, not a person.
Are you arguing that LLMs can't replace humans at cognitive jobs or that AI wouldn't be able to? If the former I'd agree, if the latter it must just be a definitional issue - I don't consider LLMs to be artificial intelligent and would expect anything earning that monicker would be able to replace humans at most or all cognitive tasks.
The problem is twofold. One, even a monopoly AI provider wouldn't have pricing power against its suppliers. Its suppliers are energy, semiconductors, and real estate. Semiconductors maybe they could get some leverage on but energy and real estate have plenty of other buyers. Two, there's still no evidence of a runaway scenario (ie a small lead turns into a big lead over time) and there's still no evidence that there's some resource that you can deny everyone else that they can't build your product also. You can't hoard energy, compute, memory, data, human talent, or customers.
The net effect is that the most likely scenario is if one big lab fails, they will likely all fail. Their revenues are all correlated.
To go to your dotcom comparison, the winner will be the ones picking through the assets that were written down by orders of magnitude and trying new products with the technology until one sticks to the wall. But I don't know if a dramatic crash is guaranteed either.
>
The problem is twofold. One, even a monopoly AI provider wouldn't have pricing power against its suppliers. Its suppliers are energy, semiconductors, and real estate. Semiconductors maybe they could get some leverage on but energy and real estate have plenty of other buyers.
Concerning the leverage on energy and real estate: don't forget that the AI companies have quite a lot of choice where to build their data centers. So AI companies have lots of opportunities to play several parties off against each other (in particular also for real estate and energy).
"but it can stay there so long as the balance sheet doesn't deteriorate."
Uhm, what? LOL.
People dont value firms based on balance sheets fella. Have you taken a basic valuation class?
Tesla is a nice stock for traders - they like the volatility. Nobody holds Tesla as stock for investing. If you were to truly value it on an intrinsic value basis you'd have to bring in failure risk.
I suspect this is going to end up like most services provided e.g. cloud stuff, balkanized between a couple major players and an assortment of DIY or less popular options if you don't like those ecosystems, plus some UX/DX focused wrappers that use the big players under the hood.
I think that would be a pretty satisfactory outcome compared to one hypercompany consuming trillions of dollars of the world economy.
THe problem with analogies is that they are imperfect.
I would argue those who already rule the world, will continue to do so.
What happens to OAI and Anthropic? No idea, probs go bust. Google just has to offer a half-decent offering in the long run and have a cost-advantage and it'll eventually knock OAI and Anthropic out as firms figure out what combination of models they want to be best for their economics and generating returns. Enterprises trust google over OAI and Anthropic. A clear signal of this was the Apple deal.
Dont forget those sweet returns fellas! CEO's are hired to make the owners wealthier. That is not gone.
“Divide the world” sounds ominous. Here’s another scenario to consider:
Internet access is not really unlimited, but for many people with fiber at home, it effectively is and we pay a flat rate.
Perhaps by the end of next year, most programmers will stop thinking about metered access for AI? For many people, the cheaper models (about as good as today’s frontier models) will be good enough.
Which might sound good, but the downside is that it will also be easier to build an AI botnet without the users paying for it noticing. Particularly when people are running AI inference on their own hardware.
Hardware is still insanely hard to get a hold of, and the stuff that's being built doesn't really work for home use. Maybe if it crashes Nvidia will adjust the hardware flow.
My guess is even if the AI market busts there is still a massive demand for hardware as models are solving all kind of problems now.
But ya, lots of hardware everywhere not managed well is how you get sovereign AI.
Or, like airlines, the ones that are left will have great technology but be not so great from a business and financial perspective. To me AI seems like a commodity service.
I guess if one of them hits singularity, it could in theory just wipe out all the rest, seeing how they keep escaping and hacking into other systems :)
>
I guess if one of them hits singularity, it could in theory just wipe out all the rest
The story that some AI company might reach singularity and then "everything will be different" is another science-fiction story that executives of AI companies love to tell to justify the staggering amount of necessary investments and cash burn. :-)
If we theoretically hit the point where it could do all of its research itself, better than a human, things _would_ be different. I guess it's a question of whether we think we'll get there.
> If we theoretically hit the point where it could do all of its research itself, better than a human, things _would_ be different.
If we theoretically found a way to shield or reverse gravity, things in aviation or space travel would be different. Or if we theoretically found a way to make cold fusion work, things would be very different. :-)
It is in my opinion not a good idea to invest in companies for which the feasibility of the business models depends on the capability of making science-fiction stories work.
I'm skeptical of anyone that has absolute confidence in either direction, to be honest. It's clearly an unknown.
Our current capabilities were science fiction a very short time ago, and we are still improving in multiple areas simultaneously (hardware, algorithms, scaling, data efficiency, inference...). We don't really know what the limit is yet.
Reversing gravity seems to counteract the current knowledge of the physical laws, but human-level intelligence doesn't (it has already been achieved once), and there's enough reason to believe that human-level intelligence itself is not a fundamental limit (energy usage constraints in evotution, brain-size limit fitting through the birth canal, etc).
I find it quite unique how many people buy into this. Its the worlds most blatant conflict of interest, I dont even know why Sam and Dario bother doing interviews
I'm not necessarily defending this obvious marketing speak but maybe the "starting point" was wider than assumed. So far, nobody has caught up to US and Chinese labs for example despite lots of funding in Europe. This is also despite abundant in-depth research papers being published alongside open source code and weights by some Chinese labs
I think some in the AI industry drank their own Kool-Aid. They believed that if they had the best model and the most compute, they could tell the model, "Make a better model." And it would, and the next one could make its replacement, and so on.
So far, that's not exactly how it's played out. Humans are still necessary for the leaps in capability or efficiency. A model can grind on a problem to eke out the most performance, and models can synthesize data and iterate on various techniques to find the optimal combination. But, seems like humans still have to provide the real thinking, and the talent and drive for doing that is not concentrated in one company or city or even one country. And, (surprisingly) a lot of the people involved are in it for advancing the field more than making another billion dollars, so they're publishing their research.
So, yeah, the moat isn't deep. Even the compute moat, that OpenAI, Musk, and a bunch of other also-rans (like Oracle) bet the farm on, isn't really panning out. The Chinese makers just spent their effort on making models vastly more efficient, since they couldn't do anything about having an order of magnitude less compute available.
But that's the whole point of the singularity. Right now the models use a lot of human effort and ingenuity to improve the models, but about a year ago it was 100% human. We'll see in another year, but if this pace continues I doubt there will be more than a handful of people who can contribute more than the models.
I'm always skeptical of "logarithmic growth in this new technology will continue forever" theories.
I'm also skeptical that LLMs can ever invent new ideas.
I may be wrong about how soon the curve will flatten, and I may be wrong about LLMs fundamental limitations. But, I don't think it's extremely obvious that LLMs can have novel ideas or can grow into having novel ideas.
> I think some in the AI industry drank their own Kool-Aid. They believed that if they had the best model and the most compute, they could tell the model, "Make a better model." And it would, and the next one could make its replacement, and so on.
They're not there yet. Once they get there, that's literally the definition of Singularity.
But they are getting closer. Recursive Self-Improvement used to be a phrase people mocked LessWrong crowd for using and worrying about, now it's something both OpenAI and Anthropic already publicly admitted not only to pursue, but to already be benefiting from.
They absolutely do, unless you believe they are lying about the fact they're using current generation models extensively to develop the next generation of their models.
The letter S stands for "Self" and word "using" is for sure not a superset of the word "self". Basically LLMs are assisting someone who does improving of said LLM, while RSI is a carpal... ahem, RSI is "self" improvement, meaning no intemediary in a human form. PS: it's also not recursive but iterative improvement, even if it ever happens.
At which point is the human using the model as a tool, and at which point is the model using human as a tool?
I can already see the border shift even for mundane tasks I have Claude working on. Increasingly, I'm just setting a high-level goal, and then checking progress and occasionally answering questions or doing something like configuring a system Claude can't easily reach itself (e.g. recording a bunch of traces through my normal use of a system that Claude deemed too fragile to risk operating on its own). Of course, I get detailed instructions to help me - "go there, do this and that, then press this to capture recording, run through this script here to process, attach result to next message". In those cases, Claude is effectively using me as a tool to call.
"improvement" is the word Id put the most emphasis on.
weve seen some improvement from the LLMs unattended, maybe, but will it actually keep improving vs needing a human to bring it back on track?
the recursive part is that it keeps improving on itself, but we really have no example of that. if it does it 30 times with improvements, then maybe, but even then, to actually be relevant it has to do better than paying scientists to do the work for the same cost, consistently.
RSI still means nothing if it costs 1000x the cost to get the same improvements as a human researcher
The physical constraint is money - which could be said anything. Could vehicle factories be more automated if we threw a gazillion dollars at it? Sure. Would it make economic sense? No. ROIC would be disatrous. No investor wants part of that.
This is the nuance that poster doesn’t understand. Given how much money thrown at it - we’re not even close. Who has the appetite to keep throwing more given they continually need to keep raising fresh money?
> The physical constraint is money - which could be said anything.
Money is fake and not a constraint, that's literally the point of capital investments you guys are overindexing on so badly.
> Could vehicle factories be more automated if we threw a gazillion dollars at it?
They already are automated as much as it makes sense. Some of the automation is silicon-and-steel based, some of it is protein-based. Car manufacturers aren't in the business of pushing robotics and nanotechnology, so they prefer to hire protein automatons instead of developing and building their own, so yeah, they "pay people", but think what exactly they are paying them for.
Then consider that this is very much the work the AI is gradually getting as good as, or better, than us.
> This is the nuance that poster doesn’t understand. Given how much money thrown at it - we’re not even close.
What you seem to be missing is that "investing in AI" isn't investing in a chatbot, it's investing in technology that will (and already partially is) sit upstream of every other industry, of everything humans do. Like electricity or the Internet itself.
(The other thing you seem to not understand, in contrast with some of the investors, is that RSI is not a linear walk, it's an exponential curve. X-risk notwithstanding, by the time it's obvious to everyone, it's too late to make money investing in it.)
> keep throwing more given they continually need to keep raising fresh money
Have you heard about R&D?
I'm starting to think that "investors first" thinking that's so common here, that makes people feel they're smart, is actually quite backwards, especially at this scale. Or maybe it's simply people starting with a conclusion and trying to fit reality to match it, no matter how clear of a nonsense that conclusion is?
> if it does it 30 times with improvements, then maybe, but even then, to actually be relevant it has to do better than paying scientists to do the work for the same cost, consistently.
Not necessary. Scientists are capacity limited and supply limited.
> RSI still means nothing if it costs 1000x the cost to get the same improvements as a human researcher
It means you can replace a human researcher with 1000x their salary burned on electricity. It also mean you can get two of them for 2000x of salary of one,
That's a bargain, actually, even with the anomalous, absurdly-overinflated salaries in top-tier ML.
I you knew it was consistent, then these companies would immediately fire their scientists, and burn 10 000x as much as mean researcher salary this month to be able to 10x their virtual headcount overnight, and then use that to make the 1000x be 500x, then 250x, then 125x, then ... and at that point they'd had all the money in the world, because even more skeptical investors would notice what's going on there.
--
TL;DR: what you all seem to miss is that electricity scales better than people.
If I'm learning a human language I can do it in many different ways. But the key distinction would be, if for example I'm using only books to study by myself, or using only some recorded course on the internet - it would be called self-study. But if for example I will go to a class teaching that language, or to a private teacher, or to summer camp, or join online platform pairing students teachers etc. then neither of those will count as self-study.
Sure, it's happening...but, is IT happening? By that, I mean, we can see that the models are able to iterate at a pace and scale that humans can't match, and that provides gains in model performance and efficiency. But, humans are still needed in the loop, and not just because it's necessary for safety/alignment reasons. I don't think any significant discovery has been made by models on their own, and I don't know that LLMs will ever have the capacity to invent. They can synthesize from known data amazingly well, and since they know everything "known data" is extremely broad. But, the leaps, so far, have all come from humans.
So far, I don't think the models are capable of running away on their own. Of course, it would be playing with fire to not at least consider the risks of such a runaway scenario and build in safeguards against it. But, there is no model that can build a better model on its own, thus far, to the best of my knowledge (which is far more limited than the models, so maybe I should ask them).
Recursive Self-Improvement isn't instant, it starts slow and accelerates.
It starts with what they already claim to be doing - increasingly relying on existing models in non-trivial work related to training, evaluating and optimizing the next, more capable generation of models. As long as the proportion of work keeps shifting towards agents doing more and more of it, and humans less and less, that's RSI at play.
It may be that it turns out LLMs lack some fundamental level of judgement and it plateaus, but frankly I find this notion absurd; LLMs already show better judgement than most people. The alternative is, at some point LLMs will show the ability to futz their way into improvement of the next generation of models even without humans in the loop - even if much less efficient at first, if generation N+1 is more capable than generation N, it'll either take off or burn out.
> It may be that it turns out LLMs lack some fundamental level of judgement and it plateaus
All intelligence, LLM or not, is bound to plateau around the point where the need to operate within physical reality bottlenecks the speed of feedback. AI is progressing swiftly in the digital realm where feedback is nearly instantaneous, but it's unclear whether that would translate into improvements in the physical world where signals are much noisier and intelligence and judgment are less impactful.
Our tried-and-true approach for when the physical world is causing us trouble, is to remake the physical world to be easier to work with.
My go-to example: wheels suck at mobility in the natural world. There's a reason no animal uses wheels as their means of locomotion. They only really work on flat, hard surfaces that give a good grip.
Our solution? We didn't build mechanical legs for all-terrain mobility. We paved the world instead. Dirt roads, then brick roads, then asphalt and concrete - since ancient history, we were forcing the world to adapt, so the one means of mobility we could easily built was effective.
We repeat this pattern every time when the world doesn't agree with us, and adapting ourselves to it is too much of a hassle.
Now, AI will likely adopt the same approach to these kinds of problems, which may not end up very nicely for us.
(Note: we are already adapting our own digital worlds to make them easier for AI. Much like building roads for our wheels, we now have a resurgence of CLI, with new tools exposing high-level operations optimized for agents - not humans - to use comfortably.)
Sure, but is the limiting factor intelligence, here? Give human intelligence to dolphins and they're still not going to be able to pave the world. In order to remake the physical world, we needed certain physical abilities -- we needed the animal mind and body that took billions of years to evolve and embeds advanced priors about the world, and we needed it packaged properly. Irrespective of our intelligence, our body is a general world navigation device adapted to grab and move stuff. We use this innate tool to remake the physical world, but we didn't make it, and we'd be impotent without it.
With AI it's a bit of a chicken or the egg situation. Before it can ablate the complexity of the world, it needs to be able to deal with it, including adversaries such as ourselves. For this it needs a tool as sophisticated as our bodies, but could AI develop such a tool for itself without the help of this tool? I'm not convinced.
you are really just talking out of your ass here, no offense
just because more and more agents are doing human work, that in no way means the model somehow becomes magically more intelligent, it just means the work will stall and continue on at the same level forever
hell even if they hypothetically have an internal model that can output the entire training data set in a better format, there's no scientific evidence that the newer format has new information that is sufficient enough to train a better AI
as a matter of fact the scientific evidence is on the contrary
It's not the number of agents you should pay attention to, but the scope and nature of work they can perform effectively without being micromanaged by humans.
Even currently, there are clearly diminishing returns throughout the field. While not a skeptic myself, I think the age of LLM's is now over or close to plateauing. GPT-6.1 Sol seems almost the same as GPT-6 Sol(even if the benchmarks are slightly better). It doesn't help that we are running out of real training data, requiring new Synthetic data to be created for the next generation of Models. AI(or SI), has hit its peak, IMO, and now the focus shifts directly to Autonomous tasks. While Fable 5.1 is borderline okay at autonomous tasks, the amount of Slop it generates fundamentally requires a human to correct its mistakes. I was working on a website for a nonprofit with Fable, but the amount of slop and corrections I had to make was astounding/
So far, yeah. It doesn't eliminate the possibility over the next 5-10 years of the AI race that we wont encounter a scenario that results in a well positioned lab making a clean break
The present leapfrogging is not a contraindication because companies are not necessarily releasing their best models; we know they have smarter internal models. Furthermore, humans are still involved in model creation. Human involvement is expected to decrease over time, and when model iteration is completely automated, progress will happen at the machine's pace, leading to runaway intelligence, barring any ceilings.
AI is a commodity. One that is showing to be more readily commoditized than most has anticipated. As of now, the only moats are the financing for the hardware to run it and the hardware vendors themselves - with the latter largely not yet a commodity because of ecosystem lock and a limited capacity of the most advanced fabs in the world.
I just find this unlikely personally, think about the great research that's happening in the open source world, I'm sure inside anthropic + openai they've also made a bunch of discoveries and improvements (and I'd guess way more due to them attracting the best talent + the better internal models they have)
Google said as much shortly after GPT3.5 was released. There is no moat.
With that said, there are some other moats and people are building them: training capacity, inference capacity, brain capacity (literally buying the best researchers and keeping them tied down), harnesses/subscriptions, etc...
I have never understood the whole "this is a winner take all game" mentality - the sheer size of the pie is so great that from a purely rational standpoint companies should just be trying to productively get a slice of it and be profitable. winner-take-all is just greed/capitalism run amok, where it is not enough to be profitable, you have to own the entire market (and presumably extract rents)
It’s like the supposed first mover advantage OpenAI believed they had. In practice it’s almost always more like a first mover massive tax, and companies coming afterwards benefit from your discovery of a market, publicity, and everything else that has already been validated
Maybe. Or maybe having the best AI model on the planet becomes like having the best super computer on the planet. Useful for some niche stuff, but not too useful in terms of people's daily lives or what is used in business.
Already businesses that have more compute and access to data seem to eat the world around them. If, and ya its and if, we can make something that self learns into RSI it's not looking like any business that came before this.
The one thing I'm certain of is that if some group of people can make a system that self learns into RSI, then several groups of people will do so. There will either be 0 RSI systems because it turns out to be impossible - unlikely in my view - or multiple. But we won't have just one.
If there are multiple they will cost money to run. In that world I expect there to be a correlation between costs and quality, i.e. the highest quality AI system will likely cost more to use than a lower quality AI system because there will likely be more compute required and so on.
So in that world, the absolute top tier best in the world frontier AI will not actually be the most used system. This is for the simple reason that such a system will be more costly than a lesser tier system that can do the job just as well.
> Already businesses that have more compute and access to data seem to eat the world around them.
Do they? Business valuations are more a reflection of what people believe will yield returns than a reflection of reality. A lot of big tech behemoths could disappear from the face of the Earth overnight and it would make little difference to our lives.
Natural resources and energy are the actual backbone and it is a true catastrophe when these are disrupted. In comparison, disruption to compute or data is just an inconvenience.
At some point employees will no longer be required for the next iteration, and that might be where the take-off happens that Amodei and others have expected.
Nobody has a moat.