Hacker Newsnew | past | comments | ask | show | jobs | submit | JV00's commentslogin

Personally I found my usage of llms bimodal, I only have uses for the top frontier models (for research and development) or for the cheapest and fastest ones (to use in production for consolidated, simple flows). I am curious what people find the mid tier models useful for.

I feel like people use medium models thinking they are saving money, but in the end, it ends up wasting more time and tokens theough the increased rate of issues/bugs.

If the highest tier model makes even one or two fewer mistakes that require human intervantion, that almost makes it a no-brainer choice for production.


I've been doing some research lately on my own (aside from the major benchmarking going on) and I've definitely found cases where medium models do much poorly and cost more than the higher models. But on the other hand, if I can give the medium models certain skills or other resources to complete a task then they perform better in cost than the higher model (which, given the same resources, sometimes gets more confused than the medium does).

At least for my use case of evaluations of a set of features.


Yup, in my benchmarks usually medium or high do better than xhigh or max

You’re talking about something else (reasoning levels). The parent comment is referring to e.g. GPT Terra as opposed to Luna, Sol or Astra, Claude Sonnet as opposed to Haiku, Opus or Fable, or Gemini Flash/Hydrogen as opposed to Flash-Lite, Argon or Carbon.

Oh, good catch. I do use Sol in production, for cases where I need quite good results, but can't afford to run in production the highest tier model due to pricing (so it's like the highest tier model I can afford for that use-case).

To be more specific, this is for a support automation, where the model has to be quite accurate in following rules and generate responses, and Luna models or lower-tier models often make bad mistakes and generate wrong responses, so are not really usable for sending data directly to the user.


Even better at the local LLM stage, where QFN can catch a mistake in prior work and patch it without any subscription costs at all. then I can use a cloud model for polish and finish and move on.

I'm building stuff at home with my work focused on managing infra. If anyone wants to sponsor a SOTA model subscription I will check it out. Until then I'll see how flexible a Strix Halo setup and my homelab can be.


I also started with Qwen-3.8-27b and used for a while, I still use it so Claude can delegate tasks to it, but the SOTA models are still a lot better than local models, so much that I don't see much sense to use local models as long as I have a paid subscription.

Even if the difference is a single extra error caught early, that can end up saving me hours of work later.


Asking people to guess beforehand how much intelligence they require for a task will hopefully be a relic soon. We’re lacking useful routing techniques which accurately pick the right model for the job. Or a high intelligence model which completes simple tasks fast and cheap. Or something else.

Yeah, an xhigh models should know a task is easy and respond quicker.

I have some tasks that deserve a bit more reasoning but for a long time most of my needs have been met by mid-range models, rather than the big ones. There's barely any motivation for me these days to use something more advanced than Terra on low/medium for everyday work. For a personal agent doing occasional coding, I'm still on Mimo flash.

I think many people still expect their tasks to need something more fancy, but could do with a reset of expectations and try to go much lower.


Production pipelines for non-coding usage. Jagged edge is very real. Just one example, loads of tasks where Gemma 4 26ba4b wipes the floor with Deepseek Flash v4.1. On Artificial Analysis-like benchmarks, the latter is many times more "intelligent". Not everything in the world is coding and math, but those are easy to "measure". Things that are hard to measure aren't part of their benchmarks.

Same, I either use xhigh on best model, or use cheapest and fastest models (mercury/jev/haiku/luna)

In the Opus 4.6 days xhigh was a meaningful improvement. These days with Opus 5.5 I just don’t see a meaningful quality increase.

In executing tasks no, but in generating assets, art, UI, etc, higher reasoning leads to drastic improvements.

But children implies they will learn and replace you, We will be more like elders with dementia to them.


It's more like running a daycare. The stream of children never ends and they never grow up. If you get really sick of it though you can always kick out the kids (responsibly) and leave.


Universities, at least, should be given access


Disagree on the slop point. By now the code of frontier models works well, and the need to read it is gone. The Claude word salad problem was pretty much solved with the 5.5 opus release.

The other 3 points made in article make sense though.


> By now the code of frontier models works well, and the need to read it is gone.

Like anything else, it depends on what you're working on. For small personal projects I agree, but in maintaining large scale business applications with millions of lines of code, I am still having to guide the models to reuse existing code and to be more flexible with their data structures to accommodate changing business requirements in the future.

They only see a snapshot of the world the application exists in due to their ephemeral nature, and until that is overcome, they will only be able to learn general principles and not the particular nuances of your codebase that only arise from observing how the users use it on a daily basis.


If you don't need to read the code at all, what are the engineers doing to make the other issues matter? Deskilling and alienation are still about the relationship between engineers and the code.


You still have to orchestrate these things and the wholistic system architectures, pull the slot machine handle many times to get the right thing, and review/verify the working output constantly.

That said, I think we need to be prepared for how a simple prompt like "make it faster" will work in a relatively short timeframe - and where the AI is able to adequately refactor/re-platform/be done in a way where even the architecture becomes obscure to us.


I often use simple prompts like “make it faster” and get back functionally correct commits. When I review them, I often discover that they didn’t do what I really wanted - sometimes Claude made it faster by incurring expensive costs, or doing a migration that would be risky in prod, or making a tradeoff that’s not worth it.

Perhaps I could set up a workflow to handle this without reading the code. I’ve seen people who make Claude fill out arcane templates, and then spend their time reviewing reports and tinkering with loops or whatever. But why would I want to read a TPS report about the code rather than just reading the code?


Really seems more like you got used to their verbal tics and unmonitored quality, to be honest.


DHH, antirez, steipete all said something along the same line, about not needing to read all code anymore. Are they used to unmonitored quality too?


The Claude word salad was certainly dialed back somewhat, but there are still a few flies in the soup, so it’s far from solved.


Ok, if when it happens you just tell it, it does rewrite it better. Not always human coworkers can do that :)


Yes, there’s always flies in the soup, and sometimes they just serve you a whole plate of insects, but you can’t beat the price. Just ask the waiter to get you a soup with less flies, and keep doing that until the number of flies is acceptable to you.

How about, no thanks, I’ll go to a real restaurant.


“Solved” = roughly back on par with 4.8. This is like Apple rolling back the butterfly keyboard and telling you it’s the best keyboard yet


No come on. 4.8 was borderline unusable. I get intelligible results from 5.5


I don't know why people are downvoting this. It depends on what you're building.

AI build me a cost overview of xyz: works perfectly without ever looking at the code

AI build me unreal engine 5 level graphics, or a database, or infra at high scale: this will need some active steering for now :)


They will do these prompt just fine but the result will be a near identical copy of an open source version. And it won't tell you it cloned some existing engine, because it does not know, it just pulled it out from its model. Now you have an AI slopped 3d engine or DBMS no-one cares about that will just rot.


And even for active steering, does it require reading the code? Iteration can be done on the result of the work itself, abstracting code away


It used to be true before Luna but right now flash and flash lite are not competitive at all


Capex on data centers can be for additional gcp capacity for their customers, not necessarily spend for internal r&d.


When Meta, OpenAI, or Anthropic expand capacity, it's for their services. When GCP does this, it's for ALL services.


If anybody from Apple is reading, just give this article link to claude or codex and have it fixed! The work has already being done for you!


It could be that those bugs are Claude or Codex works of art.


In the not so distant future we'll just ask Claude to rewrite the entire MacOS from scratch without those bugs :)


It's a totally different technique though than what parent is referring to. The one you are referring to is used to take advantage of image and video compression algorithms


Maybe not them, but their children could do it. Italy's capability to assimilate is underrated.


I think there will be a difference between foundational software and application software. The more you go to the outer layer the more code will be generated on the spot, throw-away, single usage. On the other extreme, foundational software that is battle tested, security hardened, has survived waves of AI-hacking, will become irreplaceable and crystallise as a semi-immutable layer. Open source is in the best position to win foundational software while agents will dominate the outer layer of disposable code.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: