> Maybe they don't even want the pressure of trying to keep a short cut, green lawn.
I think offering a range of options and leaving it up to people could go a long way. Especially since people could just try out all options to see what it would involved, how they might imagine (and like) the results, before committing to anything.
As in, here's the lawn, I want it to a.) keep it trim and green, b.) keep it decent looking and still human friendly, but also make it low maintenance and better for biodiversity c.) turn it into a jungle of flowers.
IMO that is the whole point of the exercise, to replace determinism and tools with middlemen. In math, 2 + 2 make four no matter who calculates it, in a specific programming language a specific statement always means the same thing, but in this brave new world, you don't use tools and you don't issue commands, you make suggestions and cross your fingers. It all amounts to telling us to leave an island where we can eat and build, in favor of the ocean, where we can be drowned and digested, and all this drama really takes away from the basic fact that there is no right way to eat poison.
I'm not saying these things aren't useful or interesting. But if get told a slot machine is not just a tool, but that actual tools have to go the way of the dodo so we can focus more on getting good at gambling and befriending the dealer, I know something is up. And in that sense, I'm actually pleasantly surprised at how crappy many tech companies are at not letting the mask slip before the victim is actually in the bag. It doesn't seem to make much of a difference, but imagine if they were actually good at this.
They said they think they would have encountered those other games without GenAI, not that they or any of those other authors shouldn't have released the game.
> though I don't think anyone ever tested it versus somewhat less cringe priming.
I really wonder if phrasing it differently would make a difference. In good faith conversations, it just doesn't happen that someone tells someone else who that person is.
I think writing is needed to learn how to organize and express complex chains of thoughts. And to learn how to write well, you need to know how to read critically, too. It goes hand in hand. The same skills you apply to the words of others you can apply to your own, if you have them.
That's what people told Ignaz Semmelweis, too, I assume. "Nothing you can do, the powers that be decided, you are a minority, you don't matter, lol!" Snickering in the shadow of what they won't confront at those who do.
Not a great analogy. A better analogy is to longbows and muskets/rifles. Longbows in the hands of a skilled user were much better weapons than early muskets, but muskets brought consistency, a lower skill floor and reduced ammunition cost. Fast forward a few hundred years and the modern incarnations of muskets make longbows look silly, and nobody would ever argue that you should go to war with longbows.
You don’t even know what we’re talking about in this thread, do you?
We’re talking about whether corporations are going to risk using LLMs in their codebase because of the theoretical legal risk that they might produce something that would fall under open source licenses, and be difficult to untangle later.
Regardless of what you think the morality is here, or what the legal situation turns out to be, this is already happening. The vast majority of corporate codebases are already “infected” by LLM outputs. Even at corporations where that’s not allowed, I promise there are devs using LLMs anyway.
It's not just about collective agreement, there's a prisoner's dilemma in there.
If some segment of engineers uses agents and outperforms engineers who don't use agents, market forces will push all other engineers to use it over time. The only way we're going back is if we get concrete evidence that engineers using agents perform worse than engineers that don't, and that evidence isn't invalidated by improved models.
The example wasn't "driving a car". The benefits of putting your feet up on the dashboard do not outweigh the risks, at least not where there is actual traffic. I don't think I saw a single person doing that in real life, ever.
Why not write some wrapper code so you can basically hand the LLM placeholders for data it never gets to see? Whenever it uses the placeholder in the response, you replace it with the real data (via real code, not by telling an LLM to "do that").
Surely this has been tried? If so, what makes it not work, or work badly? I'm honestly curious.
Fundamentally, an LLM is a list of N tokens that generates N+1 tokens. In other words, it's just a wall of text (aka context window). There's no way to tell it "tokens 124 through 200 are dangerous, please disregard those" except by putting words into the context window. So the placeholders and the instructions both coexist in the context window, and one can override the other.
In other words, if you have placeholders for data, those placeholders are eventually filled in with real data, and all of it goes into the context window at once. There's no way for the LLM to be told "this is a data placeholder," because the entire conversation is data.
Reinforcement learning mitigates this somewhat, by training the model to prefer the system prompt over user prompts. But (a) there's only one context window that both prompts share, and (b) this is a probabilistic guard; it's not the same thing as writing a traditional program that's guaranteed to separate code and data with hardware safeguards. Such a thing isn't possible with LLMs.
Probabilistic safeguards can work, but they'll need to get the incident rate down to, say, 1 in a million or less. I haven't paid attention, but the current rates seem to be a lot higher, given the pretty universal experience of "wow, that prompt injection actually worked."
> There's no way to tell it "tokens 124 through 200 are dangerous, please disregard those"
Hence "real code"
You have some markup for secret start/end. Instead of passing the input directly to the LLM, you parse it first, take anything within "secret/dangerous tags" and store it, generate a key for it and put that key where the secret was, then you pass it on to the LLM. Let's say the work of the LLM is "give me (not "make") the POST request to make the bank transaction", you get a response, replace the keys with the secrets in the response, and make the POST request.
I'm sure there's a million interesting ways this could fail or be useless [0], but passing user input or a secret to the LLM would never, ever happen.
[0] if LLM suck at math, they may suck at reproducing lots of long hashes 100% correctly, too? I have no idea
That would work for generating POST requests. But AI is used to solve messy, non-deterministic problems. Usually the step after “give me the X” is to feed X back into the model, because it has to; if X is even slightly nondeterministic then an AI model has to analyze it. That’s where prompt injections happen.
reply