For consumer AI products and non-enterprise business accounts, if you look closely at the ToS it says that you retain the rights to your inputs and the outputs you receive, collectively called “content”.

You can opt-out of training in your content. However that doesn’t cover the results of tool calls - which are fair game for training. Theres also the fact that they make a distinction between ‘pre-training’, ‘post-training’, and ‘training’.

Both Anthropic and OpenAI also leave a carve out in Enterprise contracts for “safety events”, which as far as I can tell is a euphemism for “something the model has never seen before”.

That is, fundamentally the marketing ploy of “AI Safety”.

For an AI system to be “completely safe”, like a self-driving car, it must map the full environment it operates in and be taught the “right way” to handle every conceivable scenario.

So when you present it with novel data or a situation, that constitutes an “AI safety event”, which per their ToS is a carve out for training on your session.

Further, go read the ToS carefully about what you opt out of. You’ll note there’s nary a mention of tool calls. You’re only opting out inputs (what you type) and outputs from the model from being trained on. All the content of the tool calls made while running on your system are fair game for training.

They're supposed to be focused on genuine training improvements over existing data, including via RL. Gathering data under the table is misaligned behavior.

> including via RL.

Can you please expand on what you believe RL looks like in this context?

Well for us Europeans, GDPR explicitly allows them to store pretty much anything if somebody can claim it’s for safety.

It’s literally just complicating things and providing zero reassurance to us AI customers.

For safety - hmm new for me (European), please elaborate.

You don’t need user consent to retain data that’s arguably needed to provide the service itself. I guess, safety falls into that category. Training on that data is probably a grey area, if they can guarantee it won’t contain PII. They have stolen all the internet, so I wouldn’t bet highly on their restraint.

It's very easy to prove that training is not necessary to provide the service, because whenever you're using a model it's static and the training only maybe goes into new models.

It doesn't say for whose safety. It's certainly not yours.

For safety = for the Stasi equivalent that can request it without a warrant, unlike von der Leyen's phone. Until it gets leaked to criminals through North Korean state actors.

They're data brokers so it was kind of expected.

Additionally, how stable are those ToS? Do they change every few days, like their models?

It’d be absurd to believe nobody tested these, and that no individuals involved in relevant standards bodies were aware of this weakness.

Those 8 bit certs were out for over a year, in the public. So nobody tested them. At least no independent security researcher.

>. So nobody tested them.

A lot of people would benefit a lot from this incident if they keep it to themself

We live in an absurd universe, the absurd should be expected.

"absolutely no way that there is a $100 bill on the sidewalk- if there were, somebody would have picked it up already"

I'm not sure what you're saying here. I'm mainly pointing out that the UI toggles being counterintuitive doesn't change the semantics -- blocking all incoming connections blocks built-in and signed software from receiving connections too.

I mean it’s normal to you when the GitHub desktop app shows an expired certificate and pops “Always Accept” in place of “Never Accept”?

Services like ProactiveHarvesting and AppleIntelligenceReportingSELFIngestor automatically install and run on Macs Without iCloud, Siri, or Apple Intelligence setup

Kind of shocked nobody is calling out at the flag at the bottom of the announcement saying that it's not eligible for Zero Data Retention and it's currently pretty nebulous what the "Don't train on my conversations" toggle means, as the TOS classifies "conversations" as "user visible input and outputs" - says nothing about thinking, etc.

I suspect anything you make in these, the thinking traces or "safety evaluations" of the content allows whatever you build to become RL or eventual pre-training data.

This is probably a silly question, but is the internal monologue useful training data? It's where the work is done, but without the input or output, it seems hard to get any useful context.

In other words, what would the data be used to train for? It has to improve some kind of objective function. But it's a little hard to see what the objective function would be if it's just raw inner monologue.

Isn't the internal monologue exactly what Anthropic is always trying to hide to avoid distillation?

If it was useless, they wouldn't bother.

It's probably not as useful as if you also have the prompts, but even assuming they really don't train on the prompts (which we can never verify), you can probably get to the prompts based on the monologue. Some agents essentially repeat the prompt in the monologue.

"The user asked me to build X using Y..."

Yep. It occurred to me a few days ago that the OAI TOS only says “Input” as what you provide “Output” as what you receive, Input and Output collectively as “Content.” and they won't "train" on your content. But they can retain content for safety evaluations and debugging.

What's neat about "safety evaluations" in LLM parlance is apparently encountering any novel information constitutes a "safety event" that can result in new reinforcement learning data... This anthropic 2022 paper that basically describes how it's a perpetual information siphoning machine with a cute little graphic https://arxiv.org/html/2212.08073 - they publish their "constitution". OpenAI has a "model spec" they somewhat regularly update that I suspect is their equivalent process. I suspect we've all been unwittingly advancing their models capabilities..

This Dec 2025 Google paper "A Practical Guide to Generating Synthetic Data With Differential Privacy" spells it out pretty clearly - the focus is on "privacy" - nothing about protecting the user's IP or unique knowledge/ insights.. https://arxiv.org/html/2512.03238v1

In OpenAI's case it seems to essentially generating synthetic training tuples from {prompt, chain-of-thought, answer} or scores on the chain of thought for reinforcement learning.

It would appear opting out of "Improve the model for everyone" didn't actually mean what we thought it meant.

The open models let you see there thinking and the models seem to be succinctly/ compactly representing the actual underlying concepts they're working on in some symbolic way or another. But they definitely represent the meat, bone, and marrow of the task

They likely train on logs.

Oh it fucking loves its “product owner” bullshit.

A .github/CODEOWNERS file seems to help when it’s going down that path, but I don’t like to indulge it..

I mean I still regularly use 5.3 spark (the cerebrus model) that comes with my sub to do rapid reviews of 5.6's work and it finds oodles of problems in about a minute.