One great thing of the LLM age is being able to task Opus to do a security audit of a random open source tool. Or indeed a closed source one with the help of Ghidra. I can audit every Chrome plugin every time there's a new version. Finally, enough eyes to catch the shallow bugs.
What this incident highlights is you can now poke around the insides of MacOS, Android, and Windows to see what awful collection and telemetry has slipped in. Modifying what happens is a bigger challenge. Trusted boot and system integrity protection also protects the surveillance code.
It is almost certainly on a disk with full disk encryption turned on, and password/biometric unlock. Unlike iOS, MacOD hasn't gone down the path of separate encryption keys, secured by the secure coprocessor, for each category of secrets.
The best solution to not losing confidentiality is not to collect it, and not to process it or store it. Relatedly, why does this daemon have access without permission being granted? If it was a third party tool it would not.
It's illustrative that there are even errors in how the humans have formulated the task and documented the errors.
They propose:
Valentino → Female Child
Yet this is obviously a boy name, else it would be Valentina. Is this model going to return confidence values? No, they want decisions. This is an inherent problem in AI with reductive logic for these tasks.
Likewise, dates. Is 17/08/22:
- 20170822 (Chinese write the year first, and in speech and chat they will say '17' not '2017'.)
- 20220817 (Everywhere But North America.)
Is 04/06/2009:
- 20090604 (Non-North American)
- 20090406 (American)
Without knowing the cultural context, the location, the document type, the document author, etc, you could be getting it wrong.
It’s an interesting problem. And i have following thoughts to it?
- Are all cases even decidable?
- Shouldn’t one false positive rank higher than a hundred rank negative.
-Shouldnt’t the AI just abstain from decisions to avoid false positives.
- Is it smarter to seperate this in a lot of small sub problems and to define this as a rule set of small yes no decicions, why does this need to be solved agentically?
- You should definitely clean up deterministically first.
I feel like if you really want to solve this problem, you would approach it differently. This seems like a wrong approach and a bad article on top?
> -Shouldnt’t the AI just abstain from decisions to avoid false positives.
Yes, and I think regulation should be quite strong in this regard.
> - Is it smarter to seperate this in a lot of small sub problems and to define this as a rule set of small yes no decicions, why does this need to be solved agentically?
For larger contexts I worry about agentic confabulation, but humans unquestionably look at the gestalt when doing matching. I don't want idiot clankers making enforced decisions because they have misunderstood one field.
> - You should definitely clean up deterministically first.
If you can't do field validation on entry then I'm not sure deterministic rules work better, they may be worse, although easier to reason about.
But surely there are some deterministic reasons to clean up.
I personally wouldn’t lean towards LLMs on identity resolution. I would like give it the dataset and then ask me questions about specs? I really struggle to find the LLM part reasonable here. I mean potentially some typos, or nicknames, but those could be potentially resolved deterministically also.
I would probably have reviewer running that suggests new rules that I could implement. To me this seems more like compiler work than LLM work.
That is not what they say. There are errors in the data, the data does not have the real name it has “female child” in instead.
Perhaps there’s something else in the post that you’re talking about but please do take the time to actually read what’s there, I’m only telling you what’s written in the text around the problem you think you’ve found.
>Is this model going to return confidence values? No, they want decisions.
The system doesn't return confidence values because that's not what they are designing. The models, however, do and to me it's interesting that Jev's measure of confidence is pretty well calibrated to its correctness.
>Without knowing the cultural context, the location, the document type, the document author, etc, you could be getting it wrong.
And? There's no perfect process, just ones that asymptotically approach correctness at exponentially increasing cost. They own the system and get to decide what works for them.
Exactly, there is no perfect process for such simplistic binary decision making. But we are seeing pipelines for tasks, like document processing, where it is becoming accepted that the machine is right.
Rather than binary classification we should be locating document fragments in a high dimensional feature space, and with all those fragments coming to a confidence bounded guess.
Additionally, when someone submits documents that can't be recognised as valid, the response should not be INVALID, it should be UNDECIDED. In that sense, passing to Opus is great, but passing a tiny fragment of the problem is a bad way to do it.
There needs to be CVEs/KEV for public misinformation vulnerabilities. APTs have been using social media channels to spread disinformation, the CDC has been disabled by another APT, and we're failing to immunize people against health misinformation.
You mean decently-funded students. The education system compared to pretty much anywhere else on Earth is extremely well-funded, it just goes to administrative parasites.
There are a few Nordic countries that are at about 80% of California. California students have an average of $27,418 per student projected for 2026-2027, and $25k in 2025. I'm finding nearly half of that for UK, Europe. Perhaps you could peruse some statistics and cite yours? Taking into account PPP Norway and Austria get close (22k and 20k respectively). Germany and France after taking into account PPP still are around 15-17k.
Far from from 'most', we find literally none except for Luxembourg ($33k), which being an extremely wealthy and tiny nation is not that much of a surprise; and is otherwise an extremely outlier.
On the other hand, the UK, France, and Germany are all beaten by Alabama [yes, that bastion of education], at $15k PPP (you can't, of course, forget to apply PPP calculations to the US states you are looking at if you're going to apply them to European countries). This is, btw, very consistent with the average income in these areas, too - Alabama is around or wealthier in PPP terms than the UK, France, and Germany as well.
Once upon a time when a document or posting was written, the first use of a TLA (Three Letter Acryonym) or ETLA (Extended Three Letter Acryonym) would result in its expansion immediately following in parenthesis.
Despite with the 'no need for TLS, apt packages are signed' clowns would have you believe, you can usually point to a package mirror with https. This has the added advantage of preventing a mitm from knowing what packages you install or blocking selected packages.
Apt uses persistent connections and has multiple requests in flight at once (Acquire::http::Pipeline-Depth). If you acquired the entire history of downloads then you could match it against software updates, and possibly identify by-choice dependency sets, but it would be quite a project.
What this incident highlights is you can now poke around the insides of MacOS, Android, and Windows to see what awful collection and telemetry has slipped in. Modifying what happens is a bigger challenge. Trusted boot and system integrity protection also protects the surveillance code.
reply