What exactly do you mean when you say llm generated code? Are people prompting llms for changes and features without reviewing the code or iterating on it and then comparing that to what human writes? Because if so it's not surprising that you're getting worse results. Humans also write code through iteration. You can definitely get llms to write good code by enforcing guardrails and constraints through tooling and agent.md, and iterative reviews to nudge towards what you want. The first pass will look nothing like the committed code. I don't expect the llm to one shot anything.
I don't understand what you mean by novel intelligence or what people actually expect from these kinds of "Ai" but what novel intelligence can humans claim? Everything we know or learn is based on what someone else figured out. How are llms any different in that respect?
I am interested, so please link it. I agree that our judgment is important, but I think these skills are themselves a form of judgment. The agent uses them to make decisions. It will never be perfect, but it is already pretty good.
My issue with this observation is that encoding ourselves into a codebase in the way I described is something our employers did not really get in the past. When you built something for an employer, they got the code and the end product. Sometimes that was a lot, but when an engineer left, they still took their skill set, experience, and judgment with them. Replacing that person was never guaranteed to work out.
Now, though, we are starting to leave behind much more of what made us capable of doing that job or building that product in a particular way. The code is no longer the only artifact. We are also encoding parts of our decision-making process, standards, patterns, and judgment into the systems surrounding it.
That’s determined automatically by the agent and the harness. Certain skills trigger on their own depending on the context and what the agent is doing.
For example, I have a '$rest-api-design' skill that acts as a guardrail whenever work involves the REST API. That same skill gets used at multiple stages of the process: when writing the ticket, when designing the feature, during implementation, and again as part of the final review.
A typical workflow might start with a meeting with a stakeholder. The meeting transcript becomes part of the context, and I have a skill that can pull that meeting directly into the harness. I may also provide some additional context about the feature that was not captured in the meeting and ask the agent to write the feature request.
From there, the agent may trigger other skills automatically. One of those gives it access to the project's .NET library. This is a full compiled .NET library, not just a collection of scripts. Since PowerShell itself runs on .NET, the harness can load the library DLLs directly into the shell and invoke their types and methods interactively. In practice, that lets the agent use the library almost like a CLI without me having to build a separate command-line interface for every operation.
That is useful because the agent can call into the same underlying code and abstractions that the application uses. It can use that capability to verify claims made in the meeting transcript or in my prompt, inspect existing behavior, query underlying systems, validate assumptions, retrieve data, or even put together a small proof of concept before it writes the ticket.
Once it has enough confidence in what is being requested and how the system currently behaves, it produces a Jira ticket for me to review. If I am satisfied with it, it can submit the ticket to Jira.
At a later point, I can pull that same Jira ticket back into the harness and ask the agent to implement it. All of the context captured in the ticket comes with it. At that point, the '$rest-api-design' skill triggers again and acts as a guardrail during implementation.
While it is writing the code, it also knows to call another skill I wrote called '$simplify'. That skill encodes a lot of my preferences around code structure, simplicity, readability, and style, so it further constrains how the implementation is produced.
Once the agent believes the work is complete, I can deploy it to test and go through the normal testing process. If everything looks good there, I run the implementation through a review process. That review invokes many of those same skills again to verify that the code follows the architectural, API design, and implementation constraints I have defined. By the end of that process, I usually have something I am pretty comfortable with.
Getting to this point took a lot of time and iteration. I have spent a significant amount of effort refining the skills, the 'AGENTS.md' instructions, the tooling available to the agent, and the context it can access. It also helps considerably that I have deep domain knowledge of the codebase because I built it from the ground up.
The system is not perfect, and I do not expect it to be. When I notice that the agent did something strange or violated one of the constraints, I usually go back and ask it something like, "Why did you do X, and why didn’t skill Y prevent it?"
It can usually give me a useful explanation of how it arrived at that decision and why the existing guardrail did not catch it. If I agree with that reasoning, I update the skill, the instructions, or the guardrail to cover that edge case.
So the process is iterative. When something gets through that I do not like I try to understand why the system allowed it to happen and then improve the harness so that the same class of problem is less likely to happen again. Over time, those failures effectively become new constraints and additional institutional knowledge encoded into the system.
In principle I’d agree usually, but I think this specific case, it’s reasonable to highlight that the “experts get value” argument doesn’t really add much to the discourse anymore.
We know this and you just need to look at what’s happening with LLMs in maths to see it’s true. It doesn’t add anything to the verification point however.
While this may be a common understanding in this community, it's worth repeating so that this community can tell everyone else, as I don't think most people understand this currently.
But also, it is a post that is self promoting their personal twitter page; there should be a high threshold of doing this, and regurgitating some trite analogy should fall below the bar.
When I read stuff like this people I feel like people haven't yet accepted reality. I think knowing how to write code made a lot of people feel very special. Like the could do something magical, and now feel like that's being taken a way from them. I've been writing code since I was a child and spent my entire career as a software engineer, so I understand where these people are coming from, but the reality is AI has changed this profession and this skill set forever. People need to deal with this and move. You can still do very special things with AI that a "normal" person cannot do, so focus your energy there.
What is your point? That coding was easy and nothing special? It wasn't. That programming changed forever? No one arguing. That all of us should enjoy new world order? Nah, this work isn't fun for me anymore. I will do something else for the living
I never said it was easy but it's clear that you're not coping well with new world order. Unfortunately for a lot people, it's here to stay do get used to it.
Very similar performance to 4.6 and codex 5.3 but slow and token inefficient. Still wildly impressive. Initially I didn't believe the results because 3.6 27b couldn't complete the benchmark so this is a massive leap in capability.
I do agree that Qwen 3.8 27B is excellent but slow and very token inefficient. My benchmark places it near opus 4.6 and codex 5.3 performance. 3.6 27B couldn't even complete the benchmark. Please see below for details:
I was genuinely surprised because it's quite a leap from where 3.6 was an as far as I understand this isn't a new model, it's the same model that's been post trained, so I don't quite understand what they did to improve it so substantially. The previous model couldnt get through the benchmark at all. Though it remains terribly inefficient and slow. The hardware will have to get substantially faster for these kinds of models to be daily drivers. I think I forgot to mention in the bench that I ran it on an m5 max mac book
reply