Is it even worth pointing out that the author misunderstands how UB works in Rust, given that at this point such a misunderstanding has to be willful than otherwise? There is _no_ UB in the safe subset of Rust, and in mixed safe-unsafe rust, the UB can only originate in the unsafe blocks.
In modern C++ (i.e. with smart pointers) something similar is true in that the UB can only occur in code dealing with certain types and functions (e.g. raw pointers), and not other types and functions. It's really the same as rust, just without the compiler support and the explicit unsafe blocks.
I think you are entirely missing the author's point. The author is generalizing from the specific technicalities of C/Rust/etc UB, to the problem with UB which is that should it be triggered, then you can't know what the program will do. This does not have to be the result of language specification. If writing safe Rust yourself, yes no UB will occur usually, and you can know what will happen based off of what code you wrote. The author extends UB to vibecoding where there is no specification to understand the translation of prompts to code. Without thorough review, you are unable to be sure that the output code matches the intent of your prompting, which is analagous to writing code with UB. The issue the author has with vibecoded Rust is not that the code can trigger undefined behavior at the language layer, but that the perfectly "safe" code generated may not at all match the intended semantics.
The problem with the author's argument is the inductions don't follow from the premise. With defined C, you can in principle look at a piece of code and know what it will do in the abstract machine (or at least build a model dependent on assumptions about things like unspecified behavior). Actually doing this may be practically impossible, but that's not the point. It's not possible in the presence of UB. You can't know what a piece of code containing UB will do, even in principle.
You can in principle read the LLM's output and know that it won't put your credentials on the net, so it's not the same as UB. Maybe there are practical similarities to UB in how LLM bugs present, but I'm not sure it's a useful comparison and it's not the argument the author made.
> The problem with the author's argument is the inductions don't follow from the premise.
That's possible. No one ever accused me of sound arguments :-)
I would still like to address your comment anyway.
Lets call this assertion #1:
> With defined C, you can in principle look at a piece of code and know what it will do in the abstract machine ... It's not possible in the presence of UB.
And lets call this assertion #2:
> You can in principle read the LLM's output and know that it won't put your credentials on the net, so it's not the same as UB.
With Assertion #1 you state you are not examining the output of the compiler, you are only examining the input (i.e. the source code).
With Assertion #2, you state you are examining the output of the LLM, and you are not examining the input.
IOW, these two actions are not comparable because in one you examine only the input while in the other you examine only the output.
In short: you are comparing analysing the input in one case with analysing the output in another case.
For the case of accidentally doing $FOO when trying to do $BAR:
1. No amount of input-analysis on LLM prompts will ever reveal to you if it generated code that will do $FOO - you have to analyse the output. There is a zero percent chance that examining the prompt "Do $BAR" will reveal to the examiner that their credentials will be leaked by the generated code.
2. There is a large number of automated input-analysis for C that will catch a large number of UB that prevents $FOO, when the code implements "Do $BAR". Additionally, while a lot of UB gets through, a great deal are actually caught during review.
Think of the case: "I wrote code to add two numbers, but UB caused files to get deleted off my computer"
In C, this was always possible (and C programmers acted accordingly). In Java, C#, Rust, etc this was never possible. Unless your code was generated by an LLM.
That's a good point, I didn't realize I was implicitly mixing up inputs and outputs.
I think you're imagining a very particular way of using LLMs though. The source code is the source of truth in traditional development. It's the artifact we preserve long term and the one that's used to regenerate ephemeral artifacts like binaries. When you regenerate binaries from source code containing UB, the result may not behave the same as before. Each binary's semantics can be individually understood, but not the semantics of future translations.
If you treat the entire LLM->binary system as a black box, then yeah. I agree there's no reasonable way to go from input to output semantics, much as there isn't if you ask a human. But people generally aren't using the prompt as the source of truth. They're using the code that's produced, which (in the absence of traditional UB) will have the same semantics every time it's used even if the initial LLM doesn't.
The practical impossibility is a real issue, see the recent post about booleans in Doom where the author knew what the problem was, where it was, and what it was, but after reading through the standards still couldn't really find where in the standards it was forbidden, eventually saying "it's probably this bit because I can't find anything else that fits".
And when the author of the current post says:
Turn on all linting, all warnings,
this doesn't help. I've seen code compiled with -Wall -Wextra -Wtf that produces zero warnings but for which gcc happily outputs code that segfaults, crashes, or otherwise breaks catastrophically when run. So the compiler is saying "I've found UB here, I'm not going to say anything despite maximum warnings being turned on, I'm just going to output code that I know will fail when run".
If that's the author's point then the article needs a rewrite. I suspect that was _not_ the author's point and it's offered as a good faith but misplaced post-hoc justification.
>> Without thorough review, you are unable to be sure that the output code matches the intent of your prompting, which is analagous to writing code with UB.
> If that's the author's point then the article needs a rewrite. I suspect that was _not_ the author's point and it's offered as a good faith but misplaced post-hoc justification.
I am the author (thanks for giving some of your valuable attention to my post; much appreciated :-), and I can confirm that the `>> ...` quoted bit above is my point, and this bit of my blog-post is where I made that specific point
> As of today 2, there is a large and persistent drive to not just incorporate LLM assistance into coding, but to (in the words of the pro-LLM-coding group) “Move to a higher level of abstraction”.
> What this means is that the AI writes the code for you, you “review” (or not, as stated by Microsoft, Anthropic, etc), and then push to prod.
> Brilliant! Now EVERY language can exhibit UB.
Okay, fair enough, I'm not the worlds best writer, but I thought that bit was pretty clear when I wrote it. I still think it's clear. Especially the "Now EVERY language can exhibit UB" bit.
I'm now half inclined to paste the entire blog into a ChatAI somewhere and see what it thinks my conclusion is...
> Also, we've realized the scientific reality that traveling faster than light is likely impossible
I'm confused at what speculative fiction exists from before the 1910s (or thereabouts) which involves FTL? I've no doubt there's a handful of works but this is hardly a plausible explanation for a _recent_ decline in these topics.
> I'm confused at what speculative fiction exists from before the 1910s (or thereabouts) which involves FTL?
I bet very little. We didn’t really know how enormous the universe is (https://en.wikipedia.org/wiki/Stellar_parallax: “Being very difficult to measure, only about 60 stellar parallaxes had been obtained by the end of the 19th century”), and we hadn’t known for long that the speed of light is an absolute speed limit.
There are no types whose fields "live on the heap", nor are there types whose fields "live on the stack"; these are simply not properties of types. Values always live exactly where you put them and you can put values anywhere you want, thanks to Rust's "all types are moveable" rule. Now something like a `Vec` or a `Box` _owns_ some data strictly on the heap, but that data is not _part of_ (i.e. a field of) the `Vec` or `Box` value.
As a counter-example to your idea, it's theoretically possible for a type to have a `?Sized` field (at the end), although this idea was never completely fleshed out in the language. A value of such a type could be constructed on the stack.
Now in practice, if you encounter a type with an unsized type parameter, it's probably a smart pointer. It may have an ownership relation to some data which lives on the heap. That may be what you're referring to. But such heuristics are going to be more confusing than helpful for anyone who doesn't understand the basic premise. The location of data in rust is actually quite simple, but sometimes beginners make it more complicated than it really is somehow.
The federal circuit COA has never met a patent they don't like. That's all that matters. Statute text is irrelevant. The federal circuit openly ignores SCOTUS precedent about patents (e.g. Alice).
That's the traditional answer parroted in the Wireguard documentation but a few hours' serious thought and design is enough to reveal the fatal flaw: any encapsulating protocol will have to reinvent and duplicatively implement all of the routing logic. Perr-based routing is at least 50% of wireguard's value proposition. Having to reimplement it at the higher level defeats the purpose. No, obfuscation _has_ to be part of the same protocol as routing.
(Btw, same sort of thing occurs with zfs combining raid and filesystem to close the parity raid write hole. Often strictly layered systems with separation of concerns are less than the sum of their parts.)
In this case with the, I believe it’s called quantum tunneling by mullvad, it’s actually a good thing. Because the encapsulation protocol is just UDP/IP, a well established existing protocol that can masquerade as any kind of internet traffic easily.
There's a counterintuitive pricing aspect of Opus-sized LLMs in that they're so much smarter that in some cases, it can solve the problem faster and with much fewer tokens that it can end up being cheaper.
God bless these generously benevolent corporations, giving us such amazing services for the low low price of only $200 per month. I'm going to subscribe right now! I almost feel bad, it's like I'm stealing from them.
That $200 a month is getting me $2000 a month in API equivalent tokens.
I used to spend $200+ an hour on a single developer. I'm quite sure that benevolence was a factor when they submitted me an invoice, since there is no real transparency if I was being overbilled or not or that the developer acted in my best interest rather than theirs.
I'll never forget that one contractor who told me he took a whole 40 hours to do something he could have done in less than that, specifically because I allocated that as an upperbound weekly budget to him.
> That $200 a month is getting me $2000 a month in API equivalent tokens.
Do you ever feel bad for basically robbing these poor people blind? They're clearly losing so much money by giving you $1800 in FREE tokens every month. Their business can't be profitable like this, but thankfully they're doing it out of the goodness of their hearts.
I'm not sure that you actually expect to be taken seriously if you're going to assert that these companies don't have costs themselves to deliver their services.
Unfortunately during coding I have found many LLMs like to encode their beliefs and assumptions into comments; and even when they don't, they're unavoidably feeding them into the code. Then future sessions pick up on these.
In modern C++ (i.e. with smart pointers) something similar is true in that the UB can only occur in code dealing with certain types and functions (e.g. raw pointers), and not other types and functions. It's really the same as rust, just without the compiler support and the explicit unsafe blocks.
reply