All of human knowledge, experience, distilled down to idempotent instructions and code for specific tasks, functions, situations is fully testable against a near infinite amount of fuzz and attacks.
We do not question the strength of steel if properly treated.
Code is no different. We will enter the point where code is finally trusted as a carabiner and rope for the life of a climber.
Code needs to run on physical devices, which are prone to undervolting, bit flips, hardware bugs in specific batches, radiation, deterioration, and the like.
Preferably, I wouldn’t trust my life to a calculating device running on AA batteries while dangling from a rope.
We also have fully redundant systems. The elevator for example required Otis to do this fantastic demonstration 150 years ago and now we don't think twice about it.
Do you know how elevators work now? Their safety mechanisms? Planes are 10x more likely to cause death.
> Do you know how elevators work now? Their safety mechanisms? Planes are 10x more likely to cause death.
There's something called safety regulation. An installed elevator is very likely to have passed all the verification necessary for it to clear the safety bar. Same with the other devices you listed.
As for general code, it's mostly based on trust. But I see practices endorsed by the AI providers and trust is very low on my end.
This doesn't follow. We've had code as trusted as that for _decades_. Common criteria and EAL was certified at the end of the 20th century. Before that there was the rainbow series in the US dating back to the 1980s [1].
It's absolutely cool to see this sort of validation process get cheaper and easier, but it means nothing if no one cares enough to do it. EAL1 certification can be as cheap as a beefy server, but it's not commonly done because effectively no one cares.
As microsoft learned with the zune just because it's cheaper or better at the same price doesn't mean anyone will use it. Culturally, we have to make the effort of verifying code worth it.
I am curious, not that I agree it exists or understand the reality of izzat, how does it differ in saving face provided by gentlemen who allow the loser to have a graceful defeat?
In martial arts the winner will often say "I got lucky" or "you almost had me" to allow the loser to save face, at least publicly, when in reality the skill gap is evident.
if it only affected the company itself rather than ~the entire world, there may be face to be saved. trusted authorities have to be held to a MUCH higher standard, for good reason.
no grace if you use 4 as the private key for a trusted authority.
I discovered a large security concern that has existed for a decade, since the code was originally written. I was working on an unrelated task when my AI agent discovered an oversight because the agent happen to examine that file.
Everything was sound and every test passed for years along with every security scan because intent was not validated.
I would expect an unending source of exploits in the future until AI rewrites everything.
You learn along the way by finding something you want done, never giving up.
I want to build a robot that does one simple thing - follow my voice and deliver something to it's original location. That means I need to learn batteries and power loads, electronics, audio processing, directional acoustics, more programming, soldering, and a half dozen other things.
AI + deep thinking about the problem.
The goal is not the objective but to learn everything along the way.
All of human knowledge, experience, distilled down to idempotent instructions and code for specific tasks, functions, situations is fully testable against a near infinite amount of fuzz and attacks.
We do not question the strength of steel if properly treated.
Code is no different. We will enter the point where code is finally trusted as a carabiner and rope for the life of a climber.
reply