autoregressive generation doesn’t mean the model is myopic. the next-token distribution can already reflect a longer horizon plan for the output sequence.

Sure, but mightn’t there be several plausible long horizon plans?

Here’s an example: I had asked Claude for some music recommendations in a certain style. Part of its output was:

—

*Long journey tracks*

Clinic — “The Return of Evil Bill”

Guided by Voices — not really, wrong band

Silver Apples — “Oscillations”. Proto-everything, deeply repetitive, hypnotic.

—

So at some point there, the next token produced was “Guided” or “Guide” or whatever, and then because it can’t go back, it had to correct itself after the fact.

Reasoning/CoT have helped a lot, but I feel like small versions of this still happen all the time.

Human writing is like 90% editing.

Would be fun to run an LLM on fake output from itself. Like just force the first N tokens to say the beginning of something really stupid, and then see how it finishes the sentence. "You're absolutely right! Human feces is actually the most effective engine coolant because $<completion>"

Some LLM interfaces allow you to modify and “continue” an agent response. It’s very useful for guidance, including jailbreaking. Need the model to go in a certain direction? Got a refusal that you want to bypass? Just start it off in the appropriate direction and then have it continue from there.

Which ones?

llama.cpp (but maybe not for reasoning models?) and sillytavern, maybe others... I'm half a country away from my desktop right now so I can't verify much right now.

It can but it is limited because it's only got a single pass through the network to fit the entire "longer horizon plan".

from what I understand, only the holder of the private key would be able to see if a hotword was detected or not. even if Google ran hotword detection over your data, they’d only get a encrypted output on if the hotword was detected or not

that's what i was going for haha! thank you

Note, that this is Claude, not Claude Code, which should be sandboxed by default! Non-technical people are using this!

haha yeah it thinks hard for this

note that this wasn't claude code but claude ai the main website

hey this is the author here! yeah big fan of containerization, and claude's site (not claude code) is actually great at this, so it was shocking when i found this exfil!