Personally I found my usage of llms bimodal, I only have uses for the top frontier models (for research and development) or for the cheapest and fastest ones (to use in production for consolidated, simple flows). I am curious what people find the mid tier models useful for.
I feel like people use medium models thinking they are saving money, but in the end, it ends up wasting more time and tokens theough the increased rate of issues/bugs.
If the highest tier model makes even one or two fewer mistakes that require human intervantion, that almost makes it a no-brainer choice for production.
I've been doing some research lately on my own (aside from the major benchmarking going on) and I've definitely found cases where medium models do much poorly and cost more than the higher models. But on the other hand, if I can give the medium models certain skills or other resources to complete a task then they perform better in cost than the higher model (which, given the same resources, sometimes gets more confused than the medium does).
At least for my use case of evaluations of a set of features.
You’re talking about something else (reasoning levels). The parent comment is referring to e.g. GPT Terra as opposed to Luna, Sol or Astra, Claude Sonnet as opposed to Haiku, Opus or Fable, or Gemini Flash/Hydrogen as opposed to Flash-Lite, Argon or Carbon.
Oh, good catch. I do use Sol in production, for cases where I need quite good results, but can't afford to run in production the highest tier model due to pricing (so it's like the highest tier model I can afford for that use-case).
To be more specific, this is for a support automation, where the model has to be quite accurate in following rules and generate responses, and Luna models or lower-tier models often make bad mistakes and generate wrong responses, so are not really usable for sending data directly to the user.
Even better at the local LLM stage, where QFN can catch a mistake in prior work and patch it without any subscription costs at all. then I can use a cloud model for polish and finish and move on.
I'm building stuff at home with my work focused on managing infra. If anyone wants to sponsor a SOTA model subscription I will check it out. Until then I'll see how flexible a Strix Halo setup and my homelab can be.
I also started with Qwen-3.8-27b and used for a while, I still use it so Claude can delegate tasks to it, but the SOTA models are still a lot better than local models, so much that I don't see much sense to use local models as long as I have a paid subscription.
Even if the difference is a single extra error caught early, that can end up saving me hours of work later.
Asking people to guess beforehand how much intelligence they require for a task will hopefully be a relic soon. We’re lacking useful routing techniques which accurately pick the right model for the job. Or a high intelligence model which completes simple tasks fast and cheap. Or something else.
I have some tasks that deserve a bit more reasoning but for a long time most of my needs have been met by mid-range models, rather than the big ones. There's barely any motivation for me these days to use something more advanced than Terra on low/medium for everyday work. For a personal agent doing occasional coding, I'm still on Mimo flash.
I think many people still expect their tasks to need something more fancy, but could do with a reset of expectations and try to go much lower.
Production pipelines for non-coding usage. Jagged edge is very real. Just one example, loads of tasks where Gemma 4 26ba4b wipes the floor with Deepseek Flash v4.1. On Artificial Analysis-like benchmarks, the latter is many times more "intelligent". Not everything in the world is coding and math, but those are easy to "measure". Things that are hard to measure aren't part of their benchmarks.
It's more like running a daycare. The stream of children never ends and they never grow up. If you get really sick of it though you can always kick out the kids (responsibly) and leave.
Disagree on the slop point. By now the code of frontier models works well, and the need to read it is gone. The Claude word salad problem was pretty much solved with the 5.5 opus release.
The other 3 points made in article make sense though.
> By now the code of frontier models works well, and the need to read it is gone.
Like anything else, it depends on what you're working on. For small personal projects I agree, but in maintaining large scale business applications with millions of lines of code, I am still having to guide the models to reuse existing code and to be more flexible with their data structures to accommodate changing business requirements in the future.
They only see a snapshot of the world the application exists in due to their ephemeral nature, and until that is overcome, they will only be able to learn general principles and not the particular nuances of your codebase that only arise from observing how the users use it on a daily basis.
If you don't need to read the code at all, what are the engineers doing to make the other issues matter? Deskilling and alienation are still about the relationship between engineers and the code.
You still have to orchestrate these things and the wholistic system architectures, pull the slot machine handle many times to get the right thing, and review/verify the working output constantly.
That said, I think we need to be prepared for how a simple prompt like "make it faster" will work in a relatively short timeframe - and where the AI is able to adequately refactor/re-platform/be done in a way where even the architecture becomes obscure to us.
I often use simple prompts like “make it faster” and get back functionally correct commits. When I review them, I often discover that they didn’t do what I really wanted - sometimes Claude made it faster by incurring expensive costs, or doing a migration that would be risky in prod, or making a tradeoff that’s not worth it.
Perhaps I could set up a workflow to handle this without reading the code. I’ve seen people who make Claude fill out arcane templates, and then spend their time reviewing reports and tinkering with loops or whatever. But why would I want to read a TPS report about the code rather than just reading the code?
Yes, there’s always flies in the soup, and sometimes they just serve you a whole plate of insects, but you can’t beat the price. Just ask the waiter to get you a soup with less flies, and keep doing that until the number of flies is acceptable to you.
How about, no thanks, I’ll go to a real restaurant.
They will do these prompt just fine but the result will be a near identical copy of an open source version. And it won't tell you it cloned some existing engine, because it does not know, it just pulled it out from its model. Now you have an AI slopped 3d engine or DBMS no-one cares about that will just rot.
It's a totally different technique though than what parent is referring to. The one you are referring to is used to take advantage of image and video compression algorithms
I think there will be a difference between foundational software and application software. The more you go to the outer layer the more code will be generated on the spot, throw-away, single usage. On the other extreme, foundational software that is battle tested, security hardened, has survived waves of AI-hacking, will become irreplaceable and crystallise as a semi-immutable layer. Open source is in the best position to win foundational software while agents will dominate the outer layer of disposable code.
reply