Simon Willison's post about this gives a good context on why exactly this is happening. While it doesn't mention this in the Artificial Analysis page, this is likely with Max reasoning, which has extremely long reasoning traces:
I'm curious if they can make up for this with insanely high tokens-per-second especially when served from hosted providers, though, given how tiny it is (37B!)
Qwen models are slower in tokens/s, compared to similarly sized gemma4 and others, and they use more tokens per task, in part thanks to that xhigh default.
On the other hand, there are some of us who are stuck with hardware that has plenty of compute, but limited (V)RAM. The new 27B is just perfect for that.
What configuration are you using? On both vllm and llama-cpp, I get significantly higher speeds from gemma4 than qwen3.6 (with their respective speculative decoding methods).
Output TPS in vllm for instance:
- Gemma4 26B-A4B: 200-300TPS
- Qwen3.6 35B-A3B: 120-180TPS
- Gemma4 31B: 80-120TPS
- Qwen3.6 27B: 60-80TPS
This is for a first request on a dual 5090 setup, with their respective speculative decoding methods.
> with their respective speculative decoding methods
You're benchmarking drafter acceptance rate, then. Which is real life values, yes, but attributing worse drafter performance to the other 95% of the model being inherently slower.
Use claude/codex/whatever with /goal to optimize params for you.
IMHO draft model support on dense models is great alternative to MoE on GPUs (high bandwidth, less memory) – more intelligence, speed somewhere mid way there which is usually sufficient.
I took the liberty of adding your reasoning effort chat template to my setup. You can play around with the last few parameters. In generall VLLM will be better in higher concurrency scenarios, so if you only use it for a personal vibe coding assistant and less as a general home model for task execution llama.cpp may be better.
yes, you know, personal use doesn't necessary mean no concurrency.
it's good to play with harness setup where you fan out multiple concurrent branches that share non trivial amount of prefix then reduce their output/summary back into main agent.
ie. instead of serially reading further skills/relevant source files for planning/thinking, you can branch and read them in parallel reusing prefix / or use to to approach request from different angles in parallel - to map-reduce result onto main context of what's actually relevant. branching subagents has benefits of not polluting main context, shared prefix prefill is close to free on a cache hit and with concurrent decoding/continuous batching you can utilize gpu well to get good speedups.
ie. what's relevant is number of active concurrent sequences (and their shape, ie. shared prefix), not so much number of users.
i'm not sure with llama.cpp vs vllm regarding concurrency – llama server has multiple server slots, continuous/dynamic batching enabled by default, prompt caching (also on by default), ram prompt cache, context checkpoints, unified kv buffer across sequences etc. so shouldn't be bad, i guess would be good to actually benchmark. personally i'm happy with llama.cpp.
Yes, I mentioned the setup, but on vllm you can only use TP with speculative decoding or pipeline parallelism without, so there's tradeoff to both.
I gave general numbers of what I'm getting above, the performance ratios seemed similar regardless of setup (eg. getting a AWQ-in4 quant on a single GPU vs PP without speculative decoding vs TP with speculative decoding).
Overall single GPU is fastest, and TP+speculative decoding is still faster than PP, but for fp8 models you need dual GPUs whether you want it or not.
I benched Qwen 3.6 35B-A3B against Qwen 3.8 27B with the same parameters, thinking set to low. Despite 35B having 9x fewer active parameters, it benched only 2.34x slower. The 35B got only 50% more agentic tasks done per hour.
i feel like Simon omitted an important part of how Qwen's "reasoning" levels work. they are just one sentence additions/omissions to the system prompt
xhigh -> "Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer."
medium -> no mention of effort (sentence omitted)
low -> "Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration."
in my testing this doesn't seem to produce exactly deterministic thinking levels, because it's just a system prompt nudge. i had instances where medium thought longer than xhigh
The models are post-trained on these prompt additions so they’re more structural than thinking of them as “system prompts” suggests. (All LLMs ever see is tokens going in, so even the concept of a system prompt is just formatting they’ve seen in post-training.)
You can also apply fixed token budgets for the reasoning blocks, though it will decrease quality in some cases.
Why not invent a few magic token values for reasoning level instead? It would be like 4 out of a vocabulary of 200k and save like 30 tokens in every prompt
yes of course, I understand that. but I feel like it would've been nice to include in the article because the main point of it is the effort and overthinking
I'm running 3.8 27B locally, and the results from the past few days have been excellent. I find raw speed is less of an issue when you can trust the model more to reach the right result.
> insanely high tokens-per-second especially when served from hosted providers, though, given how tiny it is (37B!)
It's a dense model so it will use all of its parameters per token. 37B active parameters isn't tiny at all, it's almost what Deepseek R1 had, and it's 2/3 of what Kimi k3 uses, so it's not going to be “insanely high” tps: it's going to be three times slower than Deepseek Flash (Prefil speed is going to be quite high though, but not token generation).
Its 27B not 37B and having just 27B in total and 3T and like 30B active of those is still totally different. A 120B with 5B active is still much slower than a proper 5B. Just like the new Ling 3.0 Tiny with 8B and 1B active only gets around 120tk/s compared to 250tk/s which a real 1B one gets on my hardware.
On the other hand, it used about the same tokens as GLM 5.2 and got 1 point lower score.
The fact that we have a GLM 5.2-class model that can run on two 3090's comfortably at Q8 is absolutely insane. It wasn't long ago that GLM 5.2 was considered amazing for open weight models.
There is some special sauce that they have. It’s not just a simple quant of another release. Or so they imply. I don’t have any insight into how it works or what the Bonsai special sauce is.
It’s still going to chew up context quickly. Surely, some of the added tokens are helping the model, but does it require as many as it generates? What happens on long, multi step tasks as it pushes old tokens out of context? I’m not sure we know the answers to those.
Personally, with our company on Cursor, I can see why model makers are not the best people to go all the way down the stack. Using the right model for the situation will continue to be important, and model makers, by design, do not want to give you the choice to run different models.
Right now, we use:
- Kimi K2.5 for easy fixes, asking about the code, various agentic commands (e.g., summarizing Loom videos for Slack messages)
- Opus 4.8, Sonnet, or Kimi for planning (we find GPT-5.5 to have too terse outputs for plans)
- Kimi K2.5, Composer 2.5, GPT-5.4 mini, etc. for faster implementation (i.e. we don't have to wait around for the slower tokens-per-second generation on Sonnet, etc.)
If we had to only use Opus, Sonnet, and Haiku, I'd definitely be looking to switch harnesses
This is so neat! I really think AI turbocharges this kind of personal project way more than it speeds up programming for work:
> I was curious about the possibility of doing this myself, and I asked ChatGPT. Not surprisingly, it knew a lot of the various tapes, file formats, sizes, processing, storage, and after it asked some clarifying questions, it was quite optimistic about me being able to do this myself
Between this, it seems like it helped with so many different parts of the process:
1. Asking for how to do technical things, like transfer video from these old VHS to a newer computer.
2. Writing code for the web portal to host the videos.
3. Writing VLC plugins to help with data entry.
4. Transcribe audio into text.
Similarly, a coworker recently made a website that imitates what Alpha School does to incentivize his own kids to finish their homework all in the span of a weekend, and it's cool to think of the kinds of projects that less or minimally technical people can do with the help of ChatGPT to guide them.
Of course, the debugging techniques and the debugging and problem-solving techniques that you get from being a professional programmer helps a lot with taking what LLMs give you with a grain of salt, and knowing what they're good at and what they're not. But it is a superpower for sure.
> Of course, the debugging techniques and the debugging and problem-solving techniques that you get from being a professional programmer helps a lot with taking what LLMs give you with a grain of salt, and knowing what they're good at and what they're not. But it is a superpower for sure.
I'm really coming around to the idea for the lucky of us (and I'm assuming a lot about the average HN poster) AI really is a force-multiplying tool
>the part that surprised me most was how much context you lose if you don't capture it alongside the media
Who are these people?! [not recognizing faces], I kept asking myself while sorting through my mother's most-sacred of possessions...
This past Christmas I finally sat down and watched VHS of early "celebrations," and it's scary when the rose-colored-glasses have lost decades de-tinting.
Who are these people?! [not recognizing our- younger -selves]
To scan photo albums I found that the book scanner at the local library worked very well. You probably take a slight hit in quality but the overall result is still very good I find.
Out of curiosity, what do you think contributed to this working better than even OpenAI agent or some of the other tools out there?
I'm not that familiar with how OpenAI and other agents like Browser Use currently work, but is this, in your opinion, the most important factor?
> An infrastructure provider that exposes OS-level controls, not just a browser layer with Playwright screenshots. This is important for performance as a number of common web elements are rendered at the system level, invisible to the browser page
IMO, the combination of having an "evaluator model" at the end to verify if the intent of the task was complete, and using multiple models that look over each other's work in every step was helpful - lots of human organization analogies there, like "trust but verify" and pair programming. Memory management was also very key.
Hey figmert -- this is Peter, one of the co-founders of Wanderlog. I'm actually on a trip to Italy right now and definitely feel your pain with some performance issues, and we've been working hard to improve this.
If you haven't tried the app in the last few months, can you try it again and let me know what parts are feeling slow for you by emailing me directly at [email protected]? I'd love to take a closer look, and especially if you've got specifics with screenshots/videos, I can try to fix some of these myself too.
We've had to go through this process for the app I have, and it definitely was cumbersome and makes the process a huge pain. Fortunately, after a while Google often lets you switch to a Tier 1 assessment, which involves using various tools to analyze your code and make improvements without shelling out a ton of money.
At the same time, Google is in a tough spot here. The files and documents in your Google Drive (or Gmail) are incredibly sensitive. One possible solution is using the https://www.googleapis.com/auth/drive.file OAuth scope, which only lets you access files a user has explicitly shared with the app. I'm curious if iA Writer has limitations that makes this a bad user experience, but from a user security point of view, I can see why I want the apps that get to see my whole Google Drive audited too.
As a user of Google drive, I’m so glad it works like this. I have a ton of random apps that store stuff in my drive that I don’t fully trust, and it’s very reassuring that they only have permission to read the files that they created.
I’m certain that if the full drive access was easy to get, they would all use that as the path of least resistance. And some of those apps would be sucking all of my data out to some random server.
I'm very sympathetic to that approach. But I think it has to be tempered at least a little bit with reputation. iA has been making Writer for 12 years now and it's always been a premium, highly user-respecting app. If they can't get through that bureaucracy, it probably can't be done.
Granted, past performance doesn't mean they'll be perfect forever. It's not a guarantee. It should carry some weight, though. I can't think of many devs I'd trust with my data as much as iA. Omni Group, I guess. Agile Tortoise. There's a set of devs who stake their business on their sterling reputations. It should be possible for that gang to at least contact a human to answer their questions.
It's not clear why they even need full access to users drives without the users input. Drive offers plenty of apis that let you store and access files that don't require these hoops. There is no security audit required if you pick the scope that only lets you open files the app created. You can also let the user use the OS file picker to open any file.
I get that it's a pain for them to rewrite the integration to use these new scopes, but it's ultimately a huge win that this free for all access has been locked down.
It feels like a situation where we just need laws to make it illegal to do a data grab like this and apps in country's without those laws should get the scrutiny.
I think a random phone app WOULD do that because there are no repercussions for doing so. Facebook, LinkedIn, and then late comers ruined the phone ecosystem by doing all the shady things they did when you wanted to do one simple useful thing. I should be able to grant contact information to an app so that it can connect me with my friends on the service. I should not have to worry about all of my contact information being harvested for spam and sold to anyone the company thinks they can make a buck from.
But I also can't imagine using a program on my computer that was prevented from having full access to my file system if I wanted it to have it. MacOS slowly killing the system is making me considering switching to a different OS for the first time in over a decade
It already is illegal to write malware that steals your files. But software is global. Anon individuals in shitty countries don't care about your countries privacy laws.
So we get both privacy laws, and technical restrictions that put the user in control of their files.
Yup. And it needs to be something that has to be done regularly, either every time the app updates or on a fixed schedule. Otherwise you would get a similar ecosystem that happened with some browser extensions, where a benign developer goes, writes an useful app, gets the permissions for that and a user base, then some shady company comes and acquires the app and updates it to use the permission to suck up all data.
Sure it's an annoying process for developers, but Google has to think of the user privacy when creating the policies around these kind of permissions.
This is a bit of an unusual choice, but I recently was trying to use Webflow, and its videos are both highly informative and funny! Engineers often thumb their noses at videos, but they really show how to do it well:
OpenLLM in comparison focuses more on building LLM apps for production. For example, the integration with LangChain + BentoML makes it easy to run multiple LLMs in parallel across multiple GPUs/Nodes, or chain LLMs with other type of AI/ML models, and deploy the entire pipeline on Kubernete (via Yatai or BentoCloud).
Just to be clear, the "capital" here is not really money, but instead, machines and equipment. Think a car company (e.g., Tesla) buying a used car factory (i.e. the NUMMI plant [1]) or a biotech company buying equipment cast off from Genentech or Pfizer.
> To document the interaction between firm and machine age, we lean on 1.56 million transactions covering 70,000 models of machines. Across a wide range of industries and equipment types, young firms acquire older capital, whereas older firms are more likely to buy new capital
This seems to line up with what I've heard from friends in biotech and mechanical companies. One friend lamented that "at Apple, we had the best of everything: suppliers, equipment, machinery. Here, we make do with what we've got and find creative ways to work with them." Similarly, on the YC forums, we often see used lab equipment listed for sale, and there are whole industries around this.
Pretty neat! Definitely a part of the startup or industry network effect I haven't thought of before. I wonder what other examples of startups working with super-janky second-hand machinery are out there.
I'm sad to see all the hate this article's getting in the comments, but also can't say I expected anything else.
Most of the hate I'm seeing here is focused around work-life balance, but even the author acknowledges that "I don’t think people should cry or feel like impostors or skip their vacations regularly." There's so much more here! Kudos to Brie too for really putting herself out there like this.
Taking apart what the article mentions, the fulfillment she felt at work came from:
- A sense of shared mission: of the mission statement, "it was a little abstract, but we believed in it enough to recite it with pride"
- Being pushed to do better: "My work was meticulously but warmly critiqued by my peers and leaders alike, and my work got better and better because of it"
- A community and culture: "It felt like magic, but there was deep thought, care, and intention behind everything. I had a tingly feeling that I was part of an organization that had cracked something about creating a great culture"
Forget work -- think about a side project, a hobby, a sport -- anything that you've applied yourself to. Does it feel good to hold yourself to a high standard, in the company of other peers who are into the same things?
That sounds like something we can all get behind!
Stripe may not have been perfect, but let's not throw the baby out with the bathwater here.
> "it was a little abstract, but we believed in it enough to recite it with pride"
I have worked for companies that had a mission I believed in, although from my position I was far from moving the needle in any direction.
Maybe that gave me an extra kick at times; maybe I stayed up late a few times without complaining because it was necessary to finish off some crucial work.
But rest assured that I have never proudly recited a shared mission statement written by someone (the "company"-- its executives and vice presidents and so on) who would have no hesitation to let me go--with great sadness--because there are hard times coming. "But," they would say, "it has been great working with you, we wish you the very best."
Everyone is focused on work-life balance because so much of the article praises the idea that dedicating your life to work is what makes companies "incredible sources of community and self-actualization."
Imagine you are a young adult with a young child, and your partner also works. How is any of the following tenable?
"Everyone stayed for dinner every night [..] there was no way I was going home before my neighbor was."
What about a family?
"my manager asked me to reconsider the vacation I had been planning because my team needed me. “If you go, who will cover your work?” I looked around at my colleagues who were also regularly working 15-hour days and decided to stay put."
What about your family? What about your colleagues' families?
"Call me masochistic, but I have to admit that it felt good to care about anything that much."
But this only works if you don't have other pursuits in life that you care about that much.
"But I am still nostalgic for a time when the gravitational pull of work was strong. For me and everyone around me."
As a husband and a father, as a manager, I want to create a place that allows people to do their best work, and then go home and give their best to their families without feeling pulled constantly back to work.
"It’s more about missing that universal agreement that it’s really, really cool to devote yourself fully to your work."
10 years from now most of us are not going to be working for the same company. 10 years from now most of my kids will be grown and I'll have lost the opportunity to invest in their lives and our relationship. I'm devoted to my work, but not at the expense of my wife who I promised to share my life with, or my children who I literally brought into this world. If I'm not there for them, present in their lives, who will be? I want to build that community. I want children who take to me when I'm older. I want a family that shares values. That only happens if I surge enough time with them, over time, to know what they value, to show what I value, to grow together as they change.
How many of those coworkers will we be sharing and creating close community with ten years from now? Are we really building a lasting community around this mission?
You say:
Forget work -- think about a side project, a hobby, a sport -- anything that you've applied yourself to.
And I agree with you about the value of having a performance oriented community focused on a meaningful mission,
but if we work the way the author is advocating through her examples we won't have time/energy to apply yourself to any other pursuit.
https://simonwillison.net/2026/Aug/16/qwen-38-27b/
It seems like the token usage is 2.3x GPT Luna Max and almost 2x Kimi K3!
https://imgur.com/a/dDSyhr2
I'm curious if they can make up for this with insanely high tokens-per-second especially when served from hosted providers, though, given how tiny it is (37B!)