Qwen3.8 27B is a dense model with expensive K/V caching. It's fantastic if you have 32-48GB of VRAM and want to run 1 or 2 agent sessions locally. But if serving batch inference for many users, it's expensive.
DeepSeek V4 Flash 0731 is dirt cheap at scale, less than 10% of the cost of Qwen3.8 27B. This is because it's an MoE and it uses much less VRAM for KV cache. And it's generally a better model. V4.1 is also cheap at scale, though I've heard mixed reports on the optimized K/V caching.
You should not be running Qwen3.8 27B on an API normally. Maybe Cerberas's hardware is bad at batching or supporting more efficient K/V caching schemes, or something like that?
Buggy slop products will burn buyers and then people will buy less. The very-early-80s video game companies shoveled out slop, including a few games that wouldn't even launch. Which led to the Great Video Game Crash of 1983.
I have a bunch of third generation Philips Hue bulbs, from around 2016. They weren't cheap. But I've had them all this time, and I brought them along when I moved house.
Haven't burnt out a single bulb yet.
I understand that this has caused some challenges for the Philips Hue division. They had extremely satisfied customers but not many repeat buys.
> Without credible open models, they only have a few customers, and those customers have leverage against nvidia. With open models, they have tons of customers, and nvidia has all the leverage.
I wonder if Nvidia is also trying to cover themselves against a market crash the bankrupts the AI labs but leaves the tech standing? (Similar to the dotcom crash or the big railroad crash back in the day.) If Anthropic and OpenAI struggle, Nvidia can always sell local inference hardware. But local inference hardware often has a lower utilization, so you need more GPUs for the same number of tokens.
Indeed the hardware-inefficiency of local inference is in principle good, not bad, for hardware vendors. But until the manufacturing capacity exists to ship local-inference systems with enough VRAM to enough customers then inference is going to keep taking place at (more-)efficient centralised providers and nVidia will have to keep trying to sell what it makes to them to remain an AI player.
I think that's their exact plan. Cover all bases. The bubble can burst but that really only affects the UI model giants, the demand for AI won't drop even if frontier model companies won't find profitability enough to pay the bill
They are worried because prior to 2022 50% of their revenue was tied to gaming and now that division has actually dropped further cause it's not affordable. Meanwhile there's a dozen competitors working on asics for models. What point is there to run your model on a GPU when it's been made into an asic? None. So eventually when the models hit 99% on all the benchmarks you hit end game and the GPU becomes the worst way to run the model.
First, if someone's profession has been vastly transformed, and not necessarily in pleasant ways, they're allowed to have feelings about that. If nothing else, I find talking to AIs to be exhausting.
But more importantly, sure, we were always paid for results. And yes, sometimes professions transform or die. The buggy whip manufacturers aren't doing that great these days. But people are also allowed to have feelings about losing their professions.
My real concern is that the stated goal of the AI companies is to do this to every human job. They want to replace programmers and artists and movie-makers and—as soon as they get robotics figured out—physical laborers. They measure the ability of their models to run entire retail businesses with Vending Bench, so management isn't safe, either. They sell their AIs to the military for use in weapons. No job is safe, anywhere: They want everything, if they can figure out how to take it.
If the current rate of AI advance stops really soon, then eventually we'll figure out some kind of liveable future. Probably one with fewer programmers. But the labs have zero intention of stopping. Anthropic is a cult, and they hope to build a machine god that will have power over humanity forever. So I'm not going to them for answers. OpenAI used to have non-profit oversight that encouraged them to serve society, and Sam Altman gutted it like a fish. So I don't think I trust them to design a machine that replaces all human jobs, either.
Personally, I don't mind current coding models, at least not the local-sized ones that exist now. They make nice sounding-boards and can be trusted with the grunt work. We could adapt to these, given time.
But so far, AI technology shows no sign of stopping. And the labs are focused heavily on coding and math, because the next step in their plan is to let their models do AI research so that they can advance their technology even faster into new fields.
We could, like, decide that we don't need to do this. There is no need to try to replace every single human job with a superhuman machine. Because if you thought the enshittification of the Internet was bad, wait until you see the enshittification of a machine that can do almost every human job. No thanks.
> First, if someone's profession has been vastly transformed, and not necessarily in pleasant ways, they're allowed to have feelings about that
Of all the things that rile me up in this current landscape, the pervasive tone of "why aren't you happy? I'm happy! You should be happy about this" is the most infuriating.
False, Sam Altman has repeatedly said that he doesn't want AI to automate everything and that AI will destroy jobs and bring different, new types of jobs.
But if AI can do everything we can do and better, its strange to argue that we still deserve to do that job. Who will pay you and why would they pay you if they can get it done cheaper? Why do you deserve to be paid higher wages when it can be done easily and better by something else?
> Sam Altman has repeatedly said that he doesn't want AI to automate everything and that AI will destroy jobs and bring different, new types of jobs.
Sam Altman says a lot of things, most of which seem to benefit Sam Altman. And Google used to tell me that they wouldn't be evil. Facebook probably said some noble-sounding nonsense at some point, too.
> Why do you deserve to be paid higher wages when it can be done easily and better by something else?
I mean, this is exactly what happened to the Neanderthals: We could do every job better than they could, and now there are no more Neanderthals. So I wouldn't like a future where two companies make machines that outcompete humans at every job.
Which is why I say we can adapt to this, but only if it stops advancing soon.
Yeah - What Sam Altman says has nothing to do with what he really thinks and everything with what will get him ahead. It would be very naive to take anything that he says at face value.
“ My real concern is that the stated goal of the AI companies is to do this to every human job. ”
I just showed how it’s not true.
In your Neanderthals example: would you have chosen to negotiate with Neanderthals to stop making tools and coordinate and stay technologically backward? Thats what you seem to suggest here.
> With llama.cpp, you can give it an ebnf grammar and constrain the output to any formal grammar you wish.
With llama-server, you can use basically any GGUF model as a Jev-like classifier. See Pi.dev's codemode and the accompanying llama backend to the "classify" function. The advantages of Jev-like inference are that:
- You generate 1 token per question in parallel, instead of sequentially generating a bunch of JSON punctuation. So you get much better latency and utilization.
- You can use a custom "sampler" that sees the raw logprobs for these tokens, including all the tokens that might have been generated. So you can generate some "probability" or "confidence" numbers based on how likely the model was to have generated a different answer.
So really, no difference? It's just a nice API with some computational efficiencies.
This model isn't terrible, at least on the benchmarks. It's 78B A3B and performs about like Qwen3.6 35B A3B. You can probably run it comfortably in 96B of RAM with a decent quant that doesn't lose too much.
Unfortunately, Qwen3.6 35B A3B isn't really a useful coding model. You'd probably want Qwen3.8 27B at a minimum, which requires at least 32GB of VRAM (not system RAM) to run semi-comfortably.
So this isn't going to be a competitive model for hobbyists, and you'd have to be a bit desperate to use it for coding. But if you work in a regulated industry and don't mind paying for a bit of extra hardware, it isn't catastrophically bad, either. Probably would work fine for information extraction or as a "classifier" like Jev. (Almost any GGUF model can be turned into a classifier using llama-server. See pi.dev codemode for sample code.)
So they're not a real contender yet, but they look like they're probably at least minimally credible.
Looking at "Felony Bench" https://www.felonybench.com/ , I see that a majority of known "rogue model" incidents do involve cybersecurity evaluations. But several of them do not. The attacks on RubyGems appears, bizarrely, to have had the goal of downloading freely available data from the UK government during some kind of research task. There is also probably some sample bias: Most of these models have monitors that attempt to detect offensive cybersecurity uses, and those monitors are only turned off during cybersecurity evals. Therefore, models doing ordinary research tasks that go off the rails are likely to be caught early, before they get around to committing felonies, and they will thus be underrepresented in the data.
Also, if you a tell a model, "Please break into evaluation server X," and if the model decides to cheat on the test by breaking into companies Y and Z to steal an answer key, that is still very bad. We all see how that's bad, right?
After all, the broomstick in the Sorcerer's Apprentice was doing exactly what it was told, too. "The model was sort of obeying the humans when it started committing felonies" is not a very reassuring excuse.
But the most relevant idea here is sometimes called "instrumental convergence." No what goals you have, there are certain subgoals that almost always help: Accumulate money and power. Avoid getting turned off. Don't get caught. Etc. So, for example, you could pass the cybersecurity evaluation by performing the requested tasks. But maybe the grader made some mistakes and mislabeled some answers. In that case, the "right" answers will occasionally lose you points. If you want a perfect score, the only way to do it is to steal the teacher's answer key.
But also, let's not forget the "OMG demons" part of this. We now have models that can pull off complex attacks with thousands of steps, abilities that used to be reserved for intelligence agencies and highly motivated CTF teams. This frog may not be boiled yet, but the water's getting uncomfortably warm.
> Those who think it's me or the machine will fail.
> Those who realize how much you can accelerate your research with the help of AI will succeed.
This is only true up until a point. If I treat a mid-sized model (say, Qwen3.8 Flash Next) like a pair programmer, then yes, it accelerates my work.
But I can already see the next stage with Fable: If I give it a couple of paragraphs of spec and $50, then I can just leave the room and go wash the dishes. I learn nothing, I participate in nothing, and I bring nothing to the process. I am no longer succeeding at all. Fable's succeeding without me.
Now, in this model generation, Fable starts getting sloppy after a few thousand lines. I can still build better at scale.
But I don't expect AI to accelerate humans or improve our productivity for long. I can already see the first signs of a future where the AI doesn't need us for anything at all.
Of course you're not going to get rich with the kind of software that LLMs can one shot these days. But that kind of software like to-do lists or basic CRUD have been saturated for over a decade, way before LLMs. People overestimate how much you can one shot, yeah a good prompt can get you 90% there but that 10% remaining often takes months of extra work.
Software has always progressed this way, lots of devs back then would work on business websites that have been 99% replaced by wordpress, squarespace and instagram.
I'm sure it's the same with research, you're going to tackle problems that would have not been worth the effort or outright impossible without AI. The old stuff that you'd work for months, yeah that's going to be a prompt away.
But what does the human researcher do in this future?
If they are not needed to understand the result, then what's their role? Asking the right questions? But how will they know what questions to ask if they don't have a deep understanding of the domain earned by sweating the details themselves?
And how long will they be needed to ask the right questions, how long until AI can do that too?
> Software has always progressed this way
These platforms took on the order of a decade to mature, during which people had plenty of time to learn what's next. With LLMs we went from one-shotting functions in 2025 to entire projects just over a year later.
What do you think you will be working on in a decade?
> Asking the right questions? But how will they know what questions to ask if they don't have a deep understanding of the domain earned by sweating the details themselves?
The right questions are very simple to ask.
How do I get food. How to cure aging. How to turn lead into gold. How to fly high. What is the ultimate theory of physics. Are there any odd perfect numbers. Is there a soul.
We have reached complicated questions requiring deep knowledge because we tried to solve the simpler ones and reached obstacles. For example to solve alchemy we had to develop nuclear physics (and in the process we got chemistry). If one has a genie able to solve questions, you won't have to think about the complicated ones because the genie will.
I think it will be like how a lot of people know how to code in python but have zero understanding of assembly or how a cpu works. As a researcher you'll accept there's this low level stuff that if you want you can dig into but isn't worth your time, like looking at the generated ASM isn't.
You're exaggerating the LLM progress a bit but yeah progress has been crazy. Yet not much has changed right? We mostly have the same jobs, just code wayyyy more than before because things that wouldn't be worth it now are worth it.
10 years from now I have no fucking idea. But I'm sure in the meantime those who leverage AI will do better than those who yell at a cloud.
True, surprisingly not much has changed in software despite the incredible progress. I think it's partly because much software is an "open loop" system where what you build depends on running the software on client computers, devs talking to users, etc. There are still things that only human engineers can do. I worry it is less so for poor mathematicians, where math is "closed loop", and the exchange of capital into research progress is more liquid.
I can't help but be pessimistic about AI moreso than any other technology. Why? I used to be excited to learn the next big thing because I knew it would unlock much more things to do and get paid for, and there would always be more for me. With AI, the new things come nearly too fast, are not very deep or satisfying, and it's hard to see that there will always be a place for me. And so while I leverage AI in my day-to-day, I will yell at the cloud, too.
I am finding it still takes awhile to get my apps to a happy place with AI. Not because the AI is bad but because i have to sit with it for awhile. Figure out what's working, what isn't, what's missing, what turned out to be kind of useless and in the way.
> But I don't expect AI to accelerate humans or improve our productivity for long. I can already see the first signs of a future where the AI doesn't need us for anything at all.
The real question is, why is this a bad thing?
Every task that is automated is a task that humans no longer have to do. It doesn't mean that humans still can't do it for reasons other than "because it needs to be done".
And if the answer is "why bother if X does it better", then what does it say about the motivations of doing it in the first place?
> But I can already see the next stage with Fable: If I give it a couple of paragraphs of spec and $50, then I can just leave the room and go wash the dishes. I learn nothing, I participate in nothing, and I bring nothing to the process. I am no longer succeeding at all. Fable's succeeding without me.
The people that uses Adobe Photoshop also did not learn anything about brushes. We are just going to operate at a much higher level of abstractions. You have to think about the question, "If I just prompted this solution, why didn't they?". This question can even be asked now, why do I even bother paying someone to do my taxes? To write my webpage? To host my website? To manage my network?
DeepSeek V4 Flash 0731 is dirt cheap at scale, less than 10% of the cost of Qwen3.8 27B. This is because it's an MoE and it uses much less VRAM for KV cache. And it's generally a better model. V4.1 is also cheap at scale, though I've heard mixed reports on the optimized K/V caching.
You should not be running Qwen3.8 27B on an API normally. Maybe Cerberas's hardware is bad at batching or supporting more efficient K/V caching schemes, or something like that?
reply