Google is just scared. Google has been scared since November 30, 2022.
Text search is fundamentally a deterministic problem, and squeezing it through a heavy non-deterministic, stochastic engine -- especially through a CoT LLM -- is inefficient from a first-principles algorithm design pov. Also, Google did great search before AI overviews using probabilistic linking and knowledge graphs.
I've been using DuckDuckGo's NoAI version as a fallback, and I've had great results so far; brings me back to the web I remembered growing up -- personal blogs, random forum discussions, and whatnot -- with just a fraction of the page bloat.
I say all this as a HUGE fan of Gemini, Gemma, and Antigravity.
not really... search was great in 2008 or 2014, but not in 2022. seo spammers won by then, and their writing was even worse than opus 5.
I remember that the sites google shown before were sites you could get lost in. often they were communities. often they were sites actually written by an enthusiast of a subject.
these sites died. I wonder whether google was the victim or the culprit. the death of these sites did end up making their product worse. however, any site that actually hooked you meant less ad revenue for google. google wants you to come back from the result, since that's a second chance for you to click the ad. a result you come back from however is inherently a bad result.
You don't want Google to be a direct keyword search. Well maybe sometimes you do, but not usually. Text search is not fundamental to the Google user experience - DWIM (do what I mean) is fundamental. The ideal user experience is that you type in what you want and the machine finds it or does it. LLMs designed and deployed properly are infinitely better at this than keyword search engines are, despite being less predictable.
Example search: video card game with the robot guy
AI overview [correct]: The video card game you are thinking of is likely Inscryption, which features a mysterious, robotic-looking or masked scrybe (like P03, the robot scrybe of technology)
That's not what I meant at all. I understand no one would want 1:1 text search, and part of the reason why Google won in the 90s was because of PageRank. All I was saying was that basing a 'search engine' on a CoT LLM is very inefficient.
>LLMs designed and deployed properly are infinitely better at this than keyword search engines are, despite being less predictable.
In what way? How are LLMs more efficient and 'infinitely' better at this than what Google had before AI overviews? If anything, I find that search results have gotten worse.
That’s not what they’re doing either though. If I’m correctly informed, they are still running bog-standard Google searches under the hood and letting the LLM find the relevant stuff in them. It’s a bit of a godsend not having to do it yourself since the original results’ quality tanked so much. And kinda elegant from an algorithmic perspective actually.
It’s still not as good as actually searching the web semantically as I used to often find myself wanting to get rid of results that sounded vaguely like my search term but actually completely missed the point, but it makes it somewhat bearable again.
Text search is, but normal people dont want text search. Normies want to concierge fetch them perfect answer on first try. LLMs are really good at convincing you they just did that.
Yes, I'm aware. I meant that deteministic systems should be at the core (PageRank was deterministic), alongside other contextual clues and knowledge graphs -- things Google did before this AI overview crap.
What pisses me off is that you can't turn it off. Like, fine, you may bury and dark pattern the crap out of the disable toggle, but at least give users the choice to opt-out. And no, UDM=14 is not a toggle.
> I've been using DuckDuckGo's NoAI version as a fallback, and I've had great results so far; brings me back to the web I remembered growing up -- personal blogs, random forum discussions, and whatnot -- with just a fraction of the page bloat.
Honestly I don't believe you, despite the desire to. IME, DDG results are even worse than Google's; I've had queries that returned enough AI generated blog spam (and I mean utterly worthless borderline nonsensical drivel) to fill the entire first page of results. Not one organic result.
The fear will again be if the HybridAgent model is being faithful to pseudo text generated, even if you can read it out partially.
Also, I just finished watching a video on AI 2027 scenario. I don't know if AI companies and researchers are using it as a cautious warning or as a target to hit (or if it's genuinely inevitable).
Yes, thinking in latent space might be the ultimate efficiency hack for CoT LxMs, but going down that path will (near certainly) mean we'll lost control, for one simple reason — humanity will have no way to pause and diagnose.
It'll be like a moody chimp with a button to the dead hand system (Systema Perimeter)
I have a question, and perhaps some of the AI/ML infrastructure experts here could answer: Mistral says, "ML4 was trained from scratch on 3,800 NVIDIA Grace Blackwell GPUs in Mistral’s own datacenters in Europe."
If a 1T params model trained on ~4k NVIDIA GB GPUs could almost match the performance of Kimi's K3 (which is on par with top closed source models of OpenAI/Anthropic) while beating/exceeding other leading SOTA models from top Chinese labs, what are we (in the US) even building these super massive data centers for? Just to churn through more backpropagation reps more quickly?
SpaceXAI's Colossus supercluster in Memphis and Colossus 2 in Memphis/Mississippi (Southaven) are supposed to run into hundreds of thousands to a million GPUs. MSFT's Fairwater GPUs are supposed to have hundreds of thousands as well. So, 3800 GB GPUs are an absolute drop in the bucket. I don't understand the strategy of hyperscalers here, especially with edge inference hardware only getting better from here on (Apple, and all).
Distillation explains some of the advances, but doesn't that mean hyperscalers have a ton of deadweight wrt GPUs sitting on their balance sheets? Will all these GPUs be used for inference once a SOTA model's training checkpoint/batch is done? It's bonkers to me.
> Will all these GPUs be used for inference once a SOTA model's training checkpoint/batch is done?
I have no real data to back this up, but that has always been my assumption.
Claude says that K3 can be assumed to have required 10-100M GPU hours. If you have 100k GPUs that would mean like 6 weeks of training. 100k GPU's can serve 3-30 trillion tokens of K3 per day. Google apparently serves ≈100 trillion per day [0].
The big labs probably want to have capacity to fairly quickly train / post train different SOTA models continuously + being able to serve peak inference demand in valuable markets (US daytime?).
Your just missing many things and so have an incorrect picture of the situation
- k3 is not on par with fable or astra. Closed models remain far better than best open weights at least today.
- compute is not just used for training, more and more is inf
- even in training you don’t do one run, you do many. Final run is a small portion of total compute.
The world is extremely compute constrained currently, like extremely.
K3 feels a little under-trained. There's a huge gap between how much it knows when you talk to it, vs how well it performs on coding benchmarks. I'm quietly hopeful for some big post-training gains like we saw with the GLM-5 series, but you're right, the hype about being "Fable at home" was wishful thinking.
Also it's bizarre that K3 already feels old/dated.
It's unclear how big a role distillation plays, but it may be a big one. There's also a law of diminishing returns. To get a meaningful increase in model quality you seemingly need an exponentially larger model. And most people don't think K3 is actually on par with top closed models.
Google says 1 million blackwell gpus are being delivered monthly I'm really curious if it's companies just hoarding chips/memory/servers awaiting to be deployed in data centers not ready yet for months or years, or everything built is actually deployed upon delivery. Plus google and amazon have there own chips in the mix.
> And most people don't think K3 is actually on par with top closed models.
I'm honestly struggling to see the differences between K3 and GLM5.3 and the various Claude models.
Maybe Fable is better (a little), but for me at least, being able to see more of the thinking tokens is extremely valuable as it means I can steer the model quicker and more effectively.
Do you have more details on this claim? I see it a lot, and I'm trying to understand what people are doing that they see these differences.
Personally I feel we're at a point where the harness is more important than the model already. Garbage in, garbage out. Most people are perfectly fine with something as small as qwen 3.8 27b. Qwen 3.8 flash has also good reviews from home labbers. I think the diminishing returns is huge and trillion param models may not be the best use of resources.
> what are we (in the US) even building these super massive data centers for?
Partially to make investors think it's worth giving US companies a lot of money. Also, I think distillation is a significant part of why Chinese models perform as well as they do. I think that's completely fair play (OpenAI/Anthropic/Google/Meta stole a lot of their training data). But I would expect if US model developers stopped right now Chinese development would slow down.
US companies are clearing the path, others follow in their wake.
> But I would expect if US model developers stopped right now Chinese development would slow down.
I would be willing to take this bet, given the nationalities of basically all the ML teams I've seen in the Valley/tech over the past decade or two.
There are lots of Chinese people, and some of them are super smart, so it really should be expected that they would be able to produce good models, particularly in the incredibly competitive environment of mainland China.
Can you help me understand why you think the US labs are in the lead?
> Just to churn through more backpropagation reps more quickly?
"Just" is doing an awful lot of work here. I am a simple man, sitting here, waiting for hours on end for a computer to write code for me, that I would much prefer to get in a fraction of that time.
I assume I am not very special in that way, that there are parties who have a lot more interested in getting their stuff done quicker than I do and have no trouble imagining a world that will remain intensely supply constrained for the coming years.
PS: It's actually a fantastic feature that the stuff that is used to built the machine can then (at least to some degree of efficiency) be used to run the machine.
A 1T model with quite few active parameters though.
A 1T model with more active parameters is obviously more expensive. If it had 1T active parameters it would probably be 20 times more expensive to train.
But does that change the training requirements specifically? You're right of course for inference, unless I understood that part wrong about active parameters (which is entirely possible).
It obviously increases the time by 20 times given fixed hardware, it probably also makes the hyperparameter search more difficult and expensive, depending on how predictable the performance of your architecture is.
This was pretrained for 2 months, so a model with 1T active parameters would take 3.33 years. Maybe a 4 months pretraining run can be acceptable in a future where Euclyd or VSORA machines have been rolled out in quantity. But we don't know how far off that is.
But that (i.e. models made with 4 month pretraining runs) would probably allow an EU-made genuinely China-beating open model, so if VSORA is rolled out in quantity, let's say, tomorrow, then this could be a thing that would happen this very spring.
What do you mean? Is post-training/RL not training? It seems to me that inference is quite stable there and only change being more time on RL which I would still categorize as training.
You got it backwards. RL is long running inference plus training at the end. They let the Agent spin for quite a while, then they calculate the reward afterwards and modify the parameters accordingly.
Within the frame of that discussion, the important distinction is whether the calculation is done for a paying customer, or for model improvement. You can choose to make separate buckets for RL and training (that's interesting to understand the evolution of model creation techniques), but it won't change the end user.
This seems to imply that training will ever be done? But yeah, I think the idea is that the appetite for thinking-on-tap will be enormous.
Even for smaller models, I think they’ve found that training an enormous, inefficient model and then distilling it internally to something much more efficient to serve is the way to go.
Today's data centers are being built for yesterday's inference need. There's a persistent cult belief that ai hasnt found a niche or that companies havent proven utility or use cases or whatever. the demand for ai (internal to hyperscaler, and external for everyone else) simply dwarfs what is available.
Imagine going back in time to late-80s/early-90s, and vividly describing all the tools we have now to folks back then -- coding agents, loops, autonomous multi-hour agentic workflows, automated testing, and PRs, and whatnot. Wonder what their reaction would be.
I sometimes look at my pre-2024 workflows and feel like I'm looking at a caveman's toolchain. Absolutely bonkers as to how quickly we humans adapt to things.
Also, neat bit of trivia: Borland was originally named MIT ("Market In Time"). The OG MIT in Cambridge, MA, sent a trademark letter and they then changed their name to Borland, which was supposedly a client of theirs.
Imagine going back in time to late-80s/early-90s, and vividly describing all the tools we have now to folks back then -- coding agents, loops, autonomous multi-hour agentic workflows, automated testing, and PRs, and whatnot. Wonder what their reaction would be.
I sometimes look at my pre-2024 workflows and feel like I'm looking at a caveman's toolchain. Absolutely bonkers as to how quickly we humans adapt to things.
It’s complicated. Especially because we had a huge regression in programming from roughly the mid-90s until recently.
In the early 90s I used Turbo Pascal as a kid. It was an editor with an awesome built-in debugger, profiler, online help, super fast compiler. This all culminated into Delphi, which had all that and offered drag-and-drop UI building.
Programming regressed a lot after the demise of Borland IDEs. Especially on Linux/Unix the tooling was very primitive in comparison.
In a very weird way, LLMs brought back a bit of these old environments in the sense that you can use one interface for programming, debugging and profiling. It just does more of the boring work for us (cross-referencing code, writing profiling cases, etc.
Other than that, LLMs, etc. would be so foreign for someone from the 80s that their minds would be blown and it would look like alien technology.
They would probably say, "That's nice dear, but we just got rid of the timesharing service and we're not going back to that just because you call it The Cloud. We just got perrsonal computers and we're not going back."
"Where is the menu ?". "Where are the scrollbars ?". "I'm on the 10th page in Google search and still haven't found an example for this function that compiles"
I remember making a comment here on HN a couple weeks ago.[0]
One of the things that was promised in 2024 was a "deflationary spiral" for physical goods and the economy in general because of insane efficiencies that AI would supposedly bring. I believed that whole-heartedly and I was a huge AI fanboi even as my artists friends and Lib Arts friends were complaining.
We're in almost October 2026 now and they're nowhere in sight. We have a bunch of unreliable agents that do clerical and design work here and there, but where are all the industrial productivity boosts? If AI were so good, large industries and factories should be adopting them as a never-before-seen scale. Where are they?
Patience. I know it's hard being a huge AI fanboy among artist and Lib Arts friends who are in a cult of perma doom and hate. But I see artists using AI to make AI art that is fantastic and doesn't waste fossil fuels unlike Christopher Nolan's performative movie, The Odyssey.
Meta Muse, the #1 downloaded iOS app in the world, is only a few weeks old.
Opus 5.5, Sonnet 5.5 and Astra 6 are only a few days or weeks old.
Get new friends. They are holding you back and they will never trust you again anyway because you "betrayed" humanity by being a huge AI fanboi.
How much do you think Meta paid for that top advertising spot, or do you really believe that most normies in day to day life downloaded Muse to their phones?
I bet not a single one of my friends or coworkers, tech savvy or not, have downloaded muse.
> I know it's hard being a huge AI fanboy among artist and Lib Arts friends who are in a cult of perma doom and hate.
aw come on. no need to be insulting ... and I dunno about "doom", but a lot of the hate from artists is actually kind of justified, that didn't go away.
and tbh if you call yourself a "fanboy" unironically, kindly go easy on calling other people's behaviour a "cult".
What bothers me about this whole AI tokenomics situation is the lack of transparency. OpenAI and Anthropic have to perhaps be the most opaque companies in existence wrt their offerings. There's like a thousand variables that they can change on the backend at the push of a button which can wildly swing API spends within the same model (partly also due to the non-deterministic nature of LxMs, but still), and there's no objective way to measure them other than vibes.
When the regulations do arrive, I think they should really focus on AI companies and API providers being more transparent wrt how they're billing their customers. Because right now, it's a totally vibes-dependent and a mess.
And it's all measured in "intelligence", a completely meaningless term. For coding i'd be much more interested in how much context actually works, what the complexity of algorithms it can understand and create is, for what languages. How much it manages to follow existing structures or that is just adds ad-hoc machinery to pass the test, etc etc.
A smaller model in the same generation will never be the same as a bigger one, assuming this is a smaller model, and the same generation, as naming implies, it will not be comparable, it might be on the benchmarks, even on the benchmarks that matter, but the whole story should also give the drawbacks.
It's still insane that they stopped showing you all the tokens you pay for. They could inflate the billed reasoning token amount by a lot before it would raise any eyebrows.
In Search Advertising, the amount you pay (under GSP Auction) is a function of your pCTR. And guess who determines your pCTR? The Search Engine itself! :-D
you REALLY need to start working on your own local models, no not because your 4080 on a pi harness with qwen 3.8 will be mediocre today, but because in a year when OpenAI and Anthropic have fully enshittified there will be NO regulations and they'll have your chestnuts in a vice and you'll be glad you can tell them to rm -rf
I'd wager on the second scenario. Anyone who's been paying attention to the industry knows that most of the 'gains' have come from test-time compute and architecting harnesses in novel ways. In my estimation, capability increases from "pre-training" alone died early last year, and we're now probably seeing test-time and other benchmark hacks approaching their limit as well.
If you zoomed back to late-2024, people in the industry were predicting how we'd have AGI by now and the economy would've already 'taken off' with massive productivity growth and ushering in of great prosperity ('deflationary spiral'). Where is it? Where is the productivity growth? Where is the deflationary spiral?
To be fair, models have gotten better in jagged ways, but reliability is far from usable, especially in long duration tasks, and there has been no effort by the AI companies to address the human brain's bandwidth bottleneck -- they hit the gas like there's no tomorrow and we have enormously capable but jaggedly intelligent multi-modal models with agentic capabilities that are only as effective as the human using it. This whole thing has become a giant mess.
I have been doing some research on the 'productivity paradox'. Fascinating stuff.
Turns out productivity growth stalled after the 1960s and has been very low ever since. No one can properly explain why. One thing stands out: investment in compute drives economic growth, but crucially doesn't demonstrably increase productivity either of labor or capital, or total factor productivity.
What I suspect is happening is: computers and software drive wealth concentration. 60 years on from the first general commercial computer, IBM 360, the entire industry has been driven by redistributing wealth towards an ever diminishing number of public companies.
With that perspective, what is happening with LLMs seems to fall right into the 6 decade pattern. I'm still in the middle of reading the 2 dozen papers or so I found about it so far but it has been fascinating.
How do you explain the fact that Qwen3.8 27B performs vastly better than any open model from even one year ago, if using the same test-time compute and harness?
It's obviously more capable in any task I tried (coding, translation, summarizing, etc). Benchmarks are not the only way to tell if a model is better or not.
I wanna see numbers showing companies and countries having excess growth due to these tools. Where are these data?
It's all vibes, and the numbers contradict the vibes. There's 30 years of literature trying to explain the "productivity paradox" where we can't see any excess productivity driven by computer technology. Lots of FOMO, no hard data. For an entire generation. And people come here every day and say stuff like you just said and they really seem to think that "this time is different".
I even wonder if the frontier AI models are really as capable as they claim or if the companies behind them have just special cases all the “hard” questions.
For example, the earlier generative LLMs couldn’t correctly answer ‘how many r’s in “strawberry”?’ due to the underlying nature of the tokens.
If they get it correct today, how do they do it? It feels like we’re being deceived by the Wizard of Oz…
Imagine you communicated only in Chinese, but external to you there was a rote translation service that translated to and from English. Someone asks you how many r's in strawberry. ... well you've never seen the spelling, as you only have seen chinese. You're going to get it wrong.
How would you get it right? Training with the answer!
You can discount the 'special case the hard questions' by: Observing when top models run entirely locally can solve them [1], or by posing an alternative or cryptic version of the question (but careful, it might bypass the training! -- but if it can solve it then its probably legitimate.)[2]. Best of all is to stick with open (weight) models where such slight of hand is impossible and don't worry about what the closed shops are doing
[1]as they can in this case, GLM-5.3-flash says:
"There are 3 r's in "strawberry":
st r awbe rr y
1 in "straw" (straw)
2 in "berry" (berry)"
[2] E.g. try sending them aG93IG1hbnkgcuKAmXMgaW4g4oCcc3RyYXdyYmVycnnigJ0/Cg== to thwart the benchmaxxing.
I'm with this. My company was absolute chaos at the end of last year. AI this, AI that.
On Productivity:
There ARE improvements. But, these improvements were also essentially people stopping their normal workload, giving that to someone else, and focusing on building an AI tool. Our sales are way down because the VP of sales is now a tech bro.
On Model capabilities:
I think they could substantially improve and still have a similar level of usefulness for my company. I'm not working on Navier-Stokes. I'm building software for a business.
The better models ARE more accurate, handle more of the workload than they could a year ago, and are still incredibly useful tools. But there's only so much I can gain from letting a model work for longer periods of time and doing X+n reviews with X+n subagents. At the end of the day, I need to maintain my personal understanding of the BUSINESS use cases + decisions so that I can make judgements that AI would never be able to make.
And, as SOON as an open model has similar capability to the current SOTA we're probably going to cut ties with our subscriptions.
I guess it would kinda be like driving an F1 car to the grocery store. I think I'd rather have the Toyota Camry of models.
> At the end of the day, I need to maintain my personal understanding of the BUSINESS use cases + decisions so that I can make judgements that AI would never be able to make.
Exactly. There's no point having a model equivalent to a trillion Einsteins at superhuman speed if you can't verify and judge the outputs of the models for your business, else it might just be as useful as comparing water displacement capacity of Niagara falls to your bathtub.
These tools have to be human centered. AI brained tech bros believe that singularity is a good thing — no it isn't. We don't want a world where normal laws and rules and regulations breakdown -- chaos is the word.
Text search is fundamentally a deterministic problem, and squeezing it through a heavy non-deterministic, stochastic engine -- especially through a CoT LLM -- is inefficient from a first-principles algorithm design pov. Also, Google did great search before AI overviews using probabilistic linking and knowledge graphs.
I've been using DuckDuckGo's NoAI version as a fallback, and I've had great results so far; brings me back to the web I remembered growing up -- personal blogs, random forum discussions, and whatnot -- with just a fraction of the page bloat.
I say all this as a HUGE fan of Gemini, Gemma, and Antigravity.
reply