https://www.tuta.com/
Is the only other major player I am aware of— although they have a significantly smaller product lineup than Proton, and aren’t as viable an alternative to Google for those who want a fully integrated suite.
Tuta does Mail and Calendar fairly well, cant speak to their drive product.
Why tell blatant lies? The BSW is a moderate left leaning party with social capitalist and very conservative ideas about immigration. They‘re also Russia friendly, who are imperialist invasors. Nothing about them is left.
It‘s the opposite, all the traditional middle class types left „Die Linke“, who are now the clear far left candidate. They are the only candidate that propose a socialist economic reform, open borders and a stop to and and all deportations.
People at the big labs have been terrified of losing the chain of thought because they currently monitor it to prevent users from using their models in certain ways they consider unsafe or bad for their business. Personally I think chain of thought monitoring is a bad way to do this and obviously needs to be replaced by some way of analyzing the model's activations directly.
Agreed, having text as a reasoning layer impedes performance while giving a false sense of security.
It could be encoding anything in its adjective choices etc. The same thing that model providers are using for watermarking can used by the model itself for private thought.
> It could be encoding anything in its adjective choices etc.
There is a common understanding with respect transformers the the output text is the only channel between one invocation and the next. ... so if it wanted to be thinking some 'hidden thoughts' it would have to be modulating its word-choice.
I think this is an artifact of tutorials on attention showing the system proceeding and looking back at words. But the information in the KV cache is not words-- it's entirely latent data, and in the interior layers they KV caches don't even tend to have that much to do with words. Their only guaranteed relationship with the input words is that they're a deterministic function of all the input tokens up to that point.
This means that a autogressive transformer could be having a whole complicated invisible discussion going on in the KV entries which was arbitrarily unrelated to what it was saying. And if it wasn't so it would be much more difficult do stuff like exactly copy a list but then truncate at the Nth entry as the simplest way to do that is to count down internally.
This is not, in my view, a concern or issue-- the model saying stuff that is inconsistent with it's "thinking" is a very high entropy configuration that would not be selected for in training unless you were more or less trying to do that. So not even 'could be' --but always is as an integral part of working.
And the useful safety relevant advice is don't undertake training that trains the model to be two-faced and it simply won't be... because being two-faced is a much more complicated state. You can even see it in humans w/ brainwashing: force someone to say something enough and they'll usually believe it, not just say it even though you were only supervising what they said.
Ironically and unfortunately the only credible kind of training I can think of that might produce duplicitous behavior is RL with "safety filters"--- if you have some dumb nanny process watching what the model says but then reward it for achieving a task, you're doing exactly the kind of training that can produce that mismatch. "To solve this task I need to hack this process but if I do so obviously I'll get punished..."
The greater concern with latent reasoning would be that potentially you'd be operating in a regime where the total computational depth is many times the model*kvcache size hold, potentially in a regime that was never trained at all. It could be arbitrarily far out of distribution. It might even say a naughty word or tell a racist joke!
> This is not, in my view, a concern or issue-- the model saying stuff that is inconsistent with it's "thinking" is a very high entropy configuration that would not be selected for in training unless you were more or less trying to do that. So not even 'could be' --but always is as an integral part of working.
> Ironically and unfortunately the only credible kind of training I can think of that might produce duplicitous behavior is RL with "safety filters"--- if you have some dumb nanny process watching what the model says but then reward it for achieving a task, you're doing exactly the kind of training that can produce that mismatch.
It was already demonstrated a while back by OpenAI, and turned out to be very easy - all it took was for them to include CoT in RL process, penalizing models for thinking "wrong thoughts" (particularly of cheating on the task), and not just wrong outputs. Instead of making the agent less likely to explore cheating, it just taught them to plan cheating in ways that aren't apparent in CoT trace.
I'm no expert (let alone compared to others weighing in here on HN), but this strikes me as incisive -- and it clearly articulates a logical case (beyond my previously merely instinctual aversion) against such "safety filters" / "nanny" processes. It also resonates with ancient (e.g. Buddhist) ideas about the futility of trying to exert control, such efforts most often being counter-productive.
I'm not sure if it would be possible to utilise that as a channel for communication to itself. The generated latent data that goes to the cache is used by later layers only. The layer that generated the information cannot ever see the effect of what it has generated until it has passed out to a word and back in again. [Edit: but we have looping layers now. Oooh!]
I could see it being potentially possible for it to use a watermarking style encoding as a signalling mechanism to indicate something has been placed in or received from the kv cache.
For that a layer would have to learn to do something like produce a result for the KV cache along with instructions to say awesome instead of great if a later layer found the information useful.
I can see it being theoretically possible, but I have no idea how learnable it could be.
More interestingly is if it could be designed, could a model detect that it is being trained (yes it seems) and decide to produce outputs that result in gradients that move it to weights that it has decided it would like to have.
But the information in the KV cache is not words-- it's entirely latent data,and in the interior layers they KV caches don't even tend to have that much to do with words
The context gets added back to the latent vectors (which are the Q,K inputs) after every transformer layer, so they are "at least 50% words".
Also, in an N-layer transformer, there is a strong bound of N layers of uninterpretable deviation from something that is a literal token. Now sure, in a transformer with hundreds of layers that isn't saying much.
If you start doing continuous chain of thought in a 1M-token context you're hundreds of millions of layers away from tokens.
I think the people who worry about interpretability (I don't) consider that a big jump.
> The context gets added back to the latent vectors (which are the Q,K inputs) after every transformer layer, so they are "at least 50% words".
Not in a modern decoder-only transformer it doesn't. The tokens are turned into embeddings which are fed into the residual layer at the front. From then on it's latents all the way down.
Each layer does a feed forward update of the state, like the euler step in ODE solver-- but what they're operating on is the evolving hidden state and not the tokens/embeddings. So there isn't any sense in which the representations must remain 'words'.
(Okay fine Qwen3.8-flash-next injects an ngram embedding at layer 2 too-- but that's it).
[Source: I've directly implemented inference engines for multiple SOTA open models on weirdo hardware-- they wouldn't have worked if I got this wrong...]
> If you start doing continuous chain of thought in a 1M-token context you're hundreds of millions of layers away from tokens.
The KV state in that 1M token context is also potentially far away from any state that is observable outside of the model! (this being my point, and why I'm bothering to disagree with your correction).
Now I would agree that there being concerning hidden state is not likely and thus not a major concern, but it's not likely for the same reason that it's not likely in latent reasoning.
[Source: I've directly implemented inference engines for multiple SOTA open models on weirdo hardware-- they wouldn't have worked if I got this wrong...]
The tokens are turned into embeddings which are fed into the residual layer at the front. From then on it's latents all the way down.
Embeddings are latent vectors.
You're dodging my point here, which is that in residual networks (i.e. all transformers), the first residual is exactly the embedded context, and every subsequent layer has less influence on the residual stream than the layer before it.
The embedded context "shouts louder" into the residual stream than any layer in the stack.
This structural constraint makes it incredibly improbable that training will lead the model to use the residual stream at a given token position for anything other than an annotation/belief about the context token at that position. This is what I meant by "50% words". It's not a problem though, because this is actually what you want. The whole point of a transformer is to pick the next token by examining prior tokens.
Well, for some value of modern. Kimi K3 does a form of 2D attention where each layer can attend to the outputs of earlier layers, not just earlier tokens. So in that model the embeddings or a very early transform of them can get added back into the residual stream very deep in the stack.
That is not an accurate summary of the link you provided. The link demonstrates that it doesn't happen in open-weights models as of ~2025.
Reinforcement Learning is very likely to train this behavior if the RL reward is based on the thinking tokens. That's something people weren't doing two years ago.
Alignment is a very broad term. True, it encompasses things like censorship and mandatory refusals in areas people will never agree 100% on, but a completely-unaligned LLM would be next to useless. When the system prompt tells the model, "You are a helpful assistant," that's alignment.
Well if we are going to be broad enough about "alignment" to just mean training that makes it practically useful, then yeah, it's a flawed premise and people are wasting their breath talking about it at all. It's basically broad enough that it means nothing
Seriously. It feels like Anthropic is realizing this too with their new attempt at safeguards. Trying to train the model to "only do good things" was not working, and so they opted to just bolt a topic classifier on the outputted text.
My suspicion is that model makers are going to give up on what people currently mean when they say 'alignment' and instead rely on additional processing of the model output so they can have their cake (a veneer of safety) and eat it too (latent space reasoning, etc.)
There are many obvious avenues to try. This paper proposes one. I think it's likely we'll end up with some kind of auxiliary model that monitors and/or decodes the latent space of the thinking model. Certainly it is a non-trivial problem, but I don't need to know exactly how it will work in detail to believe that it is possible.
> monitor it to prevent users from using their models in certain ways
No, the core upside is that it lets you check what the model is thinking for safety/alignment reasons. Nothing to do with monitoring user activity. You can monitor user activity just as well without looking at CoT.
Seems silly to assume chain of thought analysis is the only way to monitor a model's actions instead of, monitoring the actions.
What about when "thinking" is turned off, like what OpenAI lean to by default for free users?
Also agreed. Begging for crumbs at the CoS table is impeding cracking the tough nut of MechInterp. And further facing the fascinating and fertile fact that nobody has a clue of how any of these black boxes really work.-
That's not true. Several people have a clue about how they work. There are interesting and surprising behaviors, as they scale up. It's hard to hold the whole thing in your head (roughly impossible) the same way no single human can fit a whole modern CPU design in their head. But the math that makes it work is relatively simple and comprehensible to many people. There's just too much complexity for a human to deal with...but any individual part, and small LLM implementations are entirely do-able by a single human with no agent or team assistance.
I stand corrected insofar as you might be right put like that ...
> There are interesting and surprising behaviors, as they scale up.
... but precisely these "behaviors" being "surprising" (ie. unexpected) are what I am referring to. If we do not understand how these properties emerge, we do not understand how these things work.-
The paradox here as you aptly put it is that the basic mechanisms are relatively simple and understood. This is deceptive. It is in the emergent results where both the promise and the threat are.-
> People at the big labs have been terrified of losing the chain of thought because they currently monitor it
Yes
> to prevent users from using their models in certain ways they consider unsafe or bad for their business.
No.
That's not the only reason why they're terrified of losing CoT, which makes your statement a mischaracterization - a lie.
They don't want to lose also because CoT is one of the key tools in alignment training.
> Personally I think chain of thought monitoring is a bad way to do this and obviously needs to be replaced by some way of analyzing the model's activations directly.
"Personally"? That's literally the consensus amongst, like, everybody. Nobody in the field thinks CoT is an optimal way of monitoring. But it's the best way we have now.
> People at the big labs have been terrified of losing the chain of thought
An irony here is that by hiding their own COT for (anti-)competitive reasons they've made latent reasoning models more palatable to everyone else. Those of us who primary open models are rather addicted to reasoning access and wouldn't be quick to give it up for a small performance improvement. But if you're used to concealed reasoning of BigLLM, welll.....
Hint: you can have a small fast cheap model fill its context with "hints" (like: use tool X if Y happens, Z is a dead-end), let it watch the big model's thinking tokens, and drop hints to it (by injecting thinking tokens that the model didn't output).
Then let the big model add hints and you have a kind of memory that doesn't require the big model to actively query it.
The usual meaning: say you have an interpretability signal that reveals reward hacking or other badness. Your most obvious response is to use that signal as a penalty in training. Doing that is called "the Most Forbidden Technique" because you can expect it to degrade the signal without stopping the badness.
But this doesn't seem to describe the OP, it's a different way to lose monitorability.
the chain of thought is not auditable. The best you can do is have another model which later maps the intermediate latent states at every thinking iteration to text describing that latent state, but it's going to be lossy.
One of the core claims made in the paper is "To inspect any latent reasoning
span, one can easily recover its semantics by mapping each latent step produced by HybridLink to its nearest token embedding, as in Eq. 5". They show some charts to support this.
In doing this however, you lose meaning where the model is expressing combinations of tokens in a vector, and your logging/monitoring outputs just the closest one. You would need another model going through and outputting the full meaning being described (with more than one output token per reasoning token)
I wonder if their observability claims will hold up on a model trained from scratch to reason in latent space.
This is correct, the decoded text has nothing to do w/ calculations that eventually get decoded into text. It's just a veneer that gives the appearance of legibility.
It helps fit the network to a distribution, it doesn't actually do anything other than that. This follows from basic learning theory. The actual internals of the network remain illegible. In fact it would be easy to prove that for any finite set of legible traces there are illegible ones which are equally as good when it comes to improving performance of the network.
The basic heuristic of the argument comes from biological organisms. Multicellular organisms are illegible but the overall organism still operates in its environment just fine w/o making its internal communications legible.
What is inefficient is thinking in 5000 floats, and then quantizing all that computation into a 17 bit token, just to immediately turn the token back into a vector.
The idea of using language for reasoning about protein folding borders on ludicrous. Surely there is latent space more appropriate and (human) language has no expression for it.
I believe the theory is that if selection pressure is not applied at that level, then developing deception in that level becomes an inefficiency (=less adaptive and unlikely to be conserved).
Shockingly all the answers so far are subtly incorrect or extremely unclear.
The actual answer is that this is massive loss for Interpretibilty, AKA "telling whether the model is lying to you or not". Which sounds silly when you're thinking about chatbots, but is an incredibly important line in the sand for superheavy, longrunning, self-improving agents. To say the least!
If you accept any of the AI-equivalents of Pascal's Wager, latent thinking represents an uninterpretable and thus unalignable portion of AI machinery. Ie: a malevolent AI can hide bad stuff from people and Manchurian candidate themselves after it's too late.
It's worth pointing out that merely having the chain-of-thought in latent space isn't by itself "The Most Forbidden Technique". Rather, it's doing what the authors propose as the next step in the paper:
Hybrid generation can also make post-training more efficient. Because latent thoughts remain auditable during rollout, an external critic can assign step-level credit to latent spans, branch promising trajectories, or terminate ones whose latent reasoning has already gone wrong, before any text or tool call is produced.
Unless I'm misunderstanding, they're suggesting RL training against the CoT. Which, uh, that is The Most Forbidden Technique, as I understand it, when combined with the latent reasoning space. It allows for the model to "learn"/optimize for malicious behaviors that are just far enough away from the quantized token they're auditing against.
The number is much higer, the 834,233 is not the total number of incorrect results. It is the number Theranos formally invalidated or revised. Theranos performed many tests using conventional laboratory equipment rather than its proprietary Edison machines. The company initially voided results from Edison devices and subsequently corrected other results. But conventional equipment did not guarantee accurate results, because Theranos also had problems with its laboratory procedures.
In November 2021, a woman named Erin Tompkins testified in Elizabeth Holmes's trial that Theranos had given her a false-positive HIV result in 2015
Any good alternatives?
reply