Hacker Newsnew | past | comments | ask | show | jobs | submit | pegasus's commentslogin

Judging by the examples you gave, I suspect that your positive experience is mostly with small(ish) greenfield projects (which we know AI excels at), whereas parent is most likely employing agents in a large codebase with lot of preexisting architectural commitments.

How did you conclude

> whereas parent is most likely employing agents in a large codebase with lot of preexisting architectural commitments.

From this?

>> I have to beat the GPT and Claude models over the head with a brick every ten seconds for it to even partially code or design as well as me. Despite precise and generous specification, they frequently make horrible architectural decisions.

>> When I read claims like yours, I want to withhold judgement and understand, because it 100% does not line up with my experience and my gut reaction is to question your own capability to either write, read or understand quality code. I code all day long with AI models but it's an extremely frustrating experience at times. And don't even get me started on all the annoying meta behavior outside of coding, such as Sol's obsession with "tests" or competing for world's busiest-yet-least-functional UI

That doesn't follow at all.

Perhaps I should have expanded, since putting a number of examples doesn't tell people how large those codebases are. It's true that some of them are greenfield, but some are coded within large codebases that existed before LLMs arrived. You're also dismissing the smaller projects as if it's a given that LLMs should work for those.

What I'm finding interesting is there seem to be disconnected camps of people who either can get it to work, or not. People seem to have either integrated the LLMs into their daily coding, big or small, and then there's people giving shallow dismissals of its efficacy.


The user you are replying to is completely correct. I am discussing load-bearing codebases with real users and real consequences and architectural commitments, not hobby projects for my friends and I, or "massively" multiplayer online games that I probably haven't actually properly benchmarked for being truly massively multiplayer at scale.

You listed some really cool projects and you're exactly the kind of person I think of when I think of a bright future empowered by AI, where developers still have fun making cool things and technology enhances our lives at the snap of a finger, so I don't mean anything negatively.

You might have a great eye for architecture and foresight for keeping bugs and complexity at a minimum, and might just have a good amount of free time, motivation and creativity. I'm not calling any of that into question, true or not. But I really was discussing a different kind of work, more mission-critical or performance-critical stuff which I work on both professionally and in my free time.

I've also cooked up some really fun little games and apps this year and we'd probably have fun using each other's tools, and little usability complaints could be dealt with quickly. But I wouldn't consider most of those polished, bug-free, safe to use, or even finished. So what you're saying does have merit, I was just discussing a different class of work. And I also recognize that AI is rapidly getting better and in a few years, many of our current concerns might no longer be relevant for a number of reasons.


Yes, but I also have two decades of experience writing substantial code for commercial organizations, writing mission-critical and performance critical code. I've got plenty of contacts in a similar situation.

When I talk to those contacts, they are not so dismissive. They don't think that the AI is making lots of mistakes, despite working on massive code bases that are old and complicated.


AI can actually be a godsend for old, complicated codebases and greatly increase velocity there, and existing architecture can help to guide them. My problem is that the 6-series was a major step backwards and I am back to arguing with these models instead of just working with them. They are really bad at adherence.

Anyway, I'd categorize my feelings not as dismissive, but frustrated. OP said their models code just as well as them, and while there are many parts of my work I'm increasingly deferring to AI, actual programming and architectural design are still lacking heavily for me.

I can trust my AI to research and suggest solutions I'd never even considered. I can't trust it to say, "Finished," not even just because there may be bugs, but because many systems will often be categorically missing, mocked, stubbed, TK'd, etc. or it will invent and bake in bad assumptions we'd never discussed. And I can't trust it to adhere to best practices. This is fine, I can review the code and iterate, but there are some things I still just find these models cannot get a grip on at all.


Smallish greenfield projects where op clearly doesn’t look at the code closely either but only evaluates happy paths of the resulting behavior. Godspeed to op.

In an important way bridges and buildings are indeed proven to be sound by applying the mathematical tools of structural engineering, materials science, etc. Software Engineering also has analogous formal tools at its disposal (Lean is one gaining a lot of exposure these days), but many were being neglected because it wasn't worth the bother. In this new adversarial world, we'll be forced to reach for them more often. Luckily the models are also a big help in making these tools more approachable.


This is a terrible article, both poorly argued and ideologically biased. The other story presented, about a peasant named Peter Stubbe who turns into a werewolf undermines its central premise, tying the phenomenon of imagined werewolfs to certain feminist ideas, or at least shows them to not be gender-specific in any meaningful way. Peter was said to have especially preyed on women. He was a "bad guy" in the same way that the she-werewolves were "bad girls". Plus it's his story which includes, as presented, accusations of sexual (shall we say) misbehaviour, so whence the conclusion at the end?...


GP already mentioned the EU as an example.


Yes or Canada

And also, do we really trust the US government on this after Snowden?


> Spam killed email.

Sure, but it didn't kill human culture. As writing didn't either. Nor will AI do it. Technology changes, human culture does the same.


The most powerful people in silicon valley are openly aiming to replace the entire species with robots in space. This is their genocidal aim. The fact that bad things haven't happened yet doesn't mean that directionally, they aren't attempting to replace as much human culture as possible.

It sounds silly! That's because they are completely divorced from the human experience, and they have enough power to really make us all suffer.


But those other ones at least have the room to potentially do that, even if most won't. So what GP is saying is that this is a magical stock indeed, but the other way around, in that it's magically capped.


The bugs will only disappear until the model that was used to security test the generated code becomes obsoleted. So, yes, this is the new cyber normal. Unless we replace the tech stack with a formally verified one. Which will need to happen, one way or another, seems to me. But cheap & sloppy generated code won't be feasible in this paradigm, so I believe we'll need a much more thoughtful approach.


> The bugs will only disappear until the model that was used to security test the generated code becomes obsoleted.

I think you're ignoring the distinct possibility that even if the code is not formally verified, LLMs could simply stop writing bugs, or this kind of bugs. No matter how smart an attacker AI, it cannot exploit bugs that are simply not there.

But hell, even formally verified ones can only verify back to the spec. There are plenty of examples of bugs in formally verified software.

I place a higher likelihood on LLMs winning (producing fewer bugs than formally verified) over the long run. Mostly because verified software can, IIUC, only verify back to the spec. Whereas LLMs can go "uh, your spec itself has a bug, you really want me to codify that?".


> you're ignoring the distinct possibility that [...] LLMs could simply stop writing bugs

That just doesn't seem like a reasonable possibility to me, given what we're seeing from and learning about these models. They have no capacity to be aware and adhere to an overarching vision, which is why they inevitably corrode architectural coherence over time [1]. The only reason these agents are able to still create usable code is because harnesses take advantage of model's ability to find bugs, giving us the famous loop, adding duct tape on top of duct tape, ratcheting up the complexity, until the adversarial model isn't able to find any more issues. Thus, the bugs are pushed out into higher complexity dimensions, temporarily invisible until a stronger model comes along which can detect them.

[1] see for example the "Decoherence Trap" article discussed here yesterday: https://maevyn.codes/decoherence

> There are plenty of examples of bugs in formally verified software.

Of course. Which is why humans will always need to be involved in the software creation process. We have to specify what we want. It's true that we ourselves often don't know what we want, in fact this is a big part of what software development is about: requirements gathering and continuously recondisering and reworking them. But it would be beyond naive to imagine that software might one day know better than us humans what it is we actually want, no?


It could lead to a mess, sure. But my favourite prompt on an LLM-untouched codebase is along the lines of "find UB and other bugs. If the fix is less than 5 lines, fix it and commit it. One change per commit". Then I of course check the commit history before pushing.

My experience over the last couple of years with this type of prompt is that eventually you run out of bugs to find (meaning you reach "I see what you mean, but I disagree").

And those fixes don't cause messes.

And a new model may find something new, but not even remotely the volume seen the first time. This indicates to me that the supply of bugs is finite, and after this period of house cleaning, it should go down.

But anyway, a mess (decoherence) doesn't necessarily mean an exploitable mess.

Reading that good article, I find myself thinking that we did not need LLMs to get decoherence in complex software or systems. At least LLMs don't have personal fiefdoms, egos, and company politics adding to encouraging decoherence.


I had in mind living systems, as most software systems of interest are: a perpetually moving target, continuously modified, expanded and refactored. Many of these systems are apparently now being polluted via careless and massive vibe-coding.

One thing to keep in mind is the logic of technological evolution and market competition will ensure that we always push the complexity of our systems to the edge of what is possible using current tools.

I don't deny that coding agents can be (and are) very useful for software engineering, as you say they have a lot going for them, and I'm pretty sure that once the dust has settled, they will be seen as an intrinsic part of the process, but I also am convinced that skilled humans will still very much be needed to direct and supervise them.


Vibe coding may produce a lot of code, but I'd say LLMs can already find more than 99% of bugs, and I extrapolate that they therefore won't add them to living systems.

But even if new code gets bugs, we run a bug finding pass on the delta. So the vulnporalypse will end.

And in my prompts I've seen LLMs find complex interaction bugs that a human would basically never find (and the bug was added by a human).

You say polluted by vibe coding. But in terms of bugs, especially in the last like 6 months, you could argue that humans are the polluters.

E.g. while I've seen "ai deleted our database", but if what you're saying is true, we should be seeing "AI added this buffer overflow". But we don't, because humans are the polluters of that.

Yes, there have been "ai coded up client side password checking", but I've not seen that recently. Only like 6 generations and three years ago.

At some point it's like telling the cab driver to just follow Google maps directions. Like... sorry, cabbie (Black Cabs in London with their "the Knowledge" is what I'm thinking), your memorization, while impressive, is worse than the app.

The painter, while impressively skilled, is a worse documentarian than the camera.

And that point is probably behind us. At least for typing. And it's climbing up in abstractions.


"Incoherent" has a static connotation, "decoherent" OTOH is dynamic and has an associated verb, which is very handy when describing a gradual process which (almost) inevitably occurs as features are added to a system. Since confusion with the usage from quantum physics is unlikely, I too think we can safely and fittingly overload the term like the article does.


Yes, this is the thinking behind using the term. It’s also different from incoherence because decoherent systems can still make sense in some ways. They still work, they can still be iterated upon.

They are decoherent because the work to change them is extremely difficult. This is a different, and much more costly, state to exist within.


I think we can safely overload the term since the domains are separated enough. No one will think you're referring to quantum states when complaining that the system has been decohering too much lately. I can see why the author choose the term: "decohere" is a verb, which is fitting since it denotes a process, something that happens to systems gradually as change accumulates. There's no such connotation and associated verb to "incoherence".


But math is not formalizable (ask Goedel why). And no, computers have not decisively beat humans at maths, that's a laughable assertion. Sure, laymen always imagined math as being about proving theorems, but there's a saying among mathematicians: good mathematicians prove theorems, great mathematicians make conjunctures, the greatest mathematicians write definitions. But maybe you already thought they've solved math when they were able to calculate a zillion digits of pi, lol.

It sounds like you're a bit like that AI stuck in the datacenter, confirming their hypothesis without needing to actually check whether it's true or not in the real world. Then proceeding to imagine how the rest of the world is wondering at your imagined magical capacity of prediction.


(Sighs)

Goedel's theorems don't stop AI from finding and formalizing math results. They certainly don't stop AI from making conjectures or writing definitions. They have nothing to do with any of that.

The theorems are about inability of formal theories to prove certain sentences involving provability (the first is about "this sentence is not provable", the second is about "this theory is consistent"). Most mathematicians never touch any such sentences in their entire career. All the life's work of someone like Gauss, an all-time great, can be formalized just fine.


I'm not surprised your understanding of Gödel's findings and their implications is just as superficial as that of maths and philosophy of mathematics in general. Gödel's work has implications way beyond self-referential logic games, and cannot be swept in a corner where it can be safely ignored. For example, there are certain diophantine equations that have no general solution due to Gödel. Also, worth discussing are the Continuum Hypothesis, Whitehead's Problem and stuff like Paris–Harrington Theorem, Goodstein’s Theorem, etc. Not to mention the huge impact on computer science with the Halting Problem, Rice's Theorem and others. In fact Gödel's work actually spawned CS as a field of study.

The idea that maths is just the study of arbitrarily thought-up formal systems is just a very naive idea that doesn't actually reflect reality, once investigated in depth. Anyway, I have feeling you will let nothing rob you of that sweet feeling of having "magical predictive capacities", so I'll be on my way. I've already dropped some bread crumbs if you're interested in actually expanding your horizons and getting in the weeds, where things are not as straightforward anymore. But beware, "here there be monsters". Godspeed!


This is obviously pasted from an LLM, so yeah, bye.


Anything I don't agree is pasted from an LLM


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: