That reads to me as if the GP confabulated a different exploit target than what the article mentioned (or the article changed), but it's also a threat that should be taken seriously.
_you_ wouldn't. The blog post is about vulnerabilities that happen inside the inference engine, and not via external interfaces accessible via HTTP. The token parser is involved both on the way in (where yes, you would as user supply it likely via HTTP), as well as the way out of the LLM (where there are very few, less hardened layers when interfacing with the parser).
e.g. for a vulnerability exploit in the parser, an LLM can be trained with a specific input sequence that would output a fixed malicious payload to output eval-able content, as outlined in the article.
The attacker supplies the context that triggers these bugs via the http interface. There is no other way to supply it with context.
Vulnerable code could exist in token generation and in hook recognition and tool call parsing [1]. There is also significant scope for mischief in tool ID mapping between models and harnesses, as these are done by untyped numeric IDs, with varying schema[2]. Routers also introduce vulnerability paths as they inspect these tokenized (json) sequences and act in them, e.g. to match models with stricter call signature regex. [3] Parallel tool calling is also an interesting surface for exploits.
I'm not saying that the HTTP interface is not an attack surface, but that securing that attack surface (as one may assume in a simple security model) is not enough to protect against that class of attacks, as no level of introspecting the input will help predict and secure against malicous output.
This is misunderstanding the attack. The potentially malicious output tokens are processed entirely by classical text processing techniques, i.e. parsing, in order to enable tool calling on the client and via MCP servers. Garden variety vulnerabilities can occur on the inference server, but are amenable to modest levels of software engineering competency.
well, yes, exactly. I'm not trying to claim that old code is more reliable just because it was written a long time ago, I'm making the claim that old code is more reliable because of the survivorship bias. If code was first written 20 years ago and is still in production, unchanged, I can be relatively certain there's no stop-the-world bugs in those lines. (this says nothing about how pretty the code is, though).
I was maybe a little unclear with the phrasing, I meant to say "software that's been around for 20 years is more reliable than software that's been around for 2 months"
Yeah this is my mistake, the phrase "can't you just turn it off" was in several drafts but got edited out, and I missed that during the publishing of the essay.
Say more? HTTP isn't mentioned in the article. The article talks about bugs in the parser that ingests token sequences.