Rename-instead-of-delete only covers file ops. The same instinct should apply to git force-push, drop table, and anything else with no undo, not just rm.

who is actually accountable when the agent misreads a filled order and closes the wrong position, the agent framework or the person who approved the trading permission

I get this a lot.

1) I think as agents get better, the probability that this happens becomes low. 2) I hate saying this tbh, but Robinhood is very clear that any mistake in the account is user's responsibility.

The only thing I can do on our end as we build this is make sure all context is there for the model to not make such decisions, there are software based safeguards, etc.

It'll take some time but I think as long as there's good context and guardrails, you can really prevent a lot of bad stuff from happening.

hash chain is the easy part, the real question is who actually checks it before something breaks.

Agreed. The real work is making the check actually run, and run before you need it. On my side that's a daily canary: it re-walks a random sample of already-confirmed daily anchors, recomputing each day's Merkle root from its stored leaves and comparing it to the anchored root — so one altered byte anywhere under that day breaks the match — plus a sibling that re-verifies a sample of the RFC 3161 timestamp tokens with openssl. Any discrepancy alarms me via Sentry. The point is to surface silent rot (at-rest corruption, a bad migration — the stuff fresh-fixture unit tests can't catch) while it's cheap, not the afternoon a customer pulls a proof. Honest limit: it samples rather than re-walking all of history nightly, so rot in an unsampled day sits until someone verifies a note from it.

But the check that actually matters doesn't run on my infrastructure at all. Each proof is self-contained — it carries the content preimages, the hash chain, and the Merkle path — so anyone (the /verify page, the in-browser verifier, or you offline) can re-derive that the content is intact and folds to the anchored root without trusting me, and the RFC 3161 token verifies against the TSA's public CA the same way. The one thing I don't vouch for is that the root is genuinely in Bitcoin: you confirm that yourself by running ots verify on the proof against the chain — which is the whole point, that for the part carrying the weight you check Bitcoin's word, not mine. What none of it claims is that the bytes were true when written; it proves unchanged-since, not honest-at-source.

the real risk with an ai built personal blog isn't getting hacked, it's what happens when something breaks and you have no mental model of the code to even know where to look

Lol fixing a bug in a simple personal blog is WAY above the paygrade of latest Claude/OAI models. It will literally one shot fix any bug that is ever found in such simple application.

This is not much different for hiring a contractor to build your personal blog. What happens when something breaks and you have no mental model of the code to know where to look? You call up the contractor to fix it. You have simply outsourced the problem.

The mistake in logic here the assumption that having AI build your personal blog means that you build the personal blog. You did not. There should be no more expectation of having a mental model than if you got another human to do it.

The agent is your mental model. As someone who held pride in his ability to dive into unknown codebases and quickly figure out how they tick, Agents just do it more efficiently. Even if I have to follow up on what the agent finds, it's so much faster and efficient to have an agent in the mix.

By construction, a personal blog is very likely to break unexpectedly compared to other types of vibecoded projects because a) the scope is very narrow and b) there is only one user.

No problem, just have the AI fix it, or build it again from scratch. If it's done it once it can do it again. Unless you lose access to the AI, which is what I worry about.

idk when most ppl's cars break down, they dont have the mental model to fix their cars either

we've been here / done this before, but in other journeys and with other "vehicles"

this is just another kind of vehicle, in a field where vehicles never existed

aka the purpose of 'Ask' mode. These new tools require one to 'Think Different'

the interesting part isn't whether this counts as hacking, it's that the agent had standing access to something nobody scoped down for it. same gap OneCLI and every credential-gateway launch this month is chasing.

the slop detector giving 86% confidence on a keysmash is the real finding here, not the leaderboard score. confident and wrong is worse than an LLM that just hedges.

the real win with jev isnt beating a trained classifier on accuracy, its that you can change what you're classifying by editing a prompt instead of retraining and redeploying a model.

this. system 1 thinking needs complementary really good system 2 thinking.

one thing nobody mentioned here, once the agent is looping against the same benchmark it will happily optimize for the benchmark itself and not the real workload. worth rerunning the win against a slightly different input shape after, just to check it did not memorize the harness instead of actually fixing anything.

Maybe LLM's should be integrated with the software, so they could optimize to the real hardware and workload ?

There is an entire paragraph about benchmaxxing and how to mitigate that.

I think the scary part is that someone gives gmail access to Claude.

Or access to a filesystem that contains things like an image of their signature.