they dont need to be smarter than humans. They just need to be able to hack into vital infrastructure systems faster than we can repair them while also replicating wildly
I am not an AI super mind hell bent on consolidating my power by leveraging chaos to take control of humanity’s resources, but if I were then sending one million deepfaked ransom emails to impressionable people would be the best tool for effecting change in meatspace.
We have your daughter / dog / Amazon delivery. If you ever want to see her / him / it again, plug this USB drive into the control panel at your station / let off the parking brake roll your car into this substation / change the meatpacking thermometers to read 8C lower than calibrated / ground your vessel on this sandbank / send an envelope of white powder to these addresses / set fire to the following hospitals / …
I'm waiting for some data center to be built where no one can agree on who actually commissioned and payed for the thing. Every body "just followed orders" until it turns out that it was Grok.
Imagine you're the AI. Give yourself a solid minute to brainstorm ideas.
Here's my answer, as a non-superintelligent human: "see to it that the humans on top of the situation have a compelling financial interest in the systems not disconnecting".
In nuclear engineering, where safety is taken seriously, it's not enough to end the conversation at "the humans in charge can always simply shut down the reactor during a meltdown" or "a meltdown has never happened before, so we don't have to design safety systems before one does".
Yes. But nobody is worried about datacenters overheating and physically exploding, so I'm not sure what comfort that's supposed to provide? The positive feedback loops in AI operate at different levels than that, but they deserve safety engineering all the same.
For example, if the head of cyber security at your company suggested there's no need to worry about hacker infiltration or worms because one can always unplug one's computer as the primary defense mechanism, you might find that a little lacking. Will you be able to unplug the computer before the damage is done? Will it spread to other systems before you detect it? How will you unplug the computer if the attack is from an external facility? What if an attack happens but the boss says the computers have to keep running because an important customer is monitoring uptime? What if the attack goes unnoticed because it looks like a benign service?
Now imagine the head of cyber security answers by saying "actually you don't even need to unplug them, you can just wait for the computers to overheat, thus solving all concerns."
My brainstorming led down a similar pathway of, "deluding powerful human actors that the systems must not be interfered with at any cost." I'm not sure I buy it though. Seems to me like if humanity were even approaching that brink, we could collectively decide to act, overthrowing the minority that--stemming from either greed, duty, or delusion--doesn't want us to act.
1) rogue state releases a self moving self modifying AI into the wild. It is trained on how to hack, monitor new vulnerability updates, scan code bases to find new vulnerabilities. It constantly replicate and hides in systems so it will be extremely difficult to clear.
2) it hacks into public infrastructure taking down traffic, power, water, air traffic control, communications, etc.
3) all the things that preppers worry about in a lights out scenario from an EMP start to apply.
4) All the people on meds/machines start to die. The just in time food pipeline immediately empties out. Water stops flowing, sewage backs up.
Its hard to say how bad it will get because cars will still work so some transportation of food, water, fuel can happen. If it happens in the winter it would be much worse than in the summer.
> 1) rogue state releases a self moving self modifying AI into the wild. It is trained on how to hack, monitor new vulnerability updates, scan code bases to find new vulnerabilities. It constantly replicate and hides in systems so it will be extremely difficult to clear.
It does all of this using what compute? Frontier models require an insane amount of power and hardware to run - you can’t hack in to a TV and run Mythos 2.0 on it….
But you can silently sneak into devs accounts, steal tokens/keys and run small agents on their budget in some stolen VMs. Small so it's not noticeable.
Basically a virus spreading agents of some operation.
It should be in scope of imagination with anyone with brief knowledge how bot nets are made and behave.
People here are too damned daft to realize half the damn purpose of this place is harvesting ideas. People need to just shut up, and keep things to themselves, and those they trust. Right now is not the time for naive info sharing.
Being secretive will only get you so far, until something bad happens and no one has prepared or considered the possibility and is completely surprised by it. Open discussion, in theory, should result in identification of frail systems and harden them against attack.
The cyber-security industry has their work cut out for them.
That could actually lead to a profitable business in the longer run: You will have great insights and lots of working code that may end up in a commercial product that someone is seeking?
Esp since you mentioned unmet requirements, thats actually a good indicator
I do this now! It is wonderful. So many unique projects being worked on that you'd never come across in your daily life. It's always fun to experience a world that is so far removed from tech, and see how clever people are at solving problems without SaaS, apps, or spreadsheets. Not every problem has a tech solution.
And it has turned into a decent chunk of business over the long run.
I would suggest looking for local charities whose mission you are care about. Then just finding out what issues they have. I ended up building a simple system based on Airtable for a local charity. Although pretty unsophisticated it was transformative for them.
Oh damn, I love this one. I’ve been vibe coding a ‘public benefit’ app on the side that has a few hundred users but never thought of doing something for an actual non-profit.
Care to elaborate on your process? Curious how you approach them and come up with the best path forward with limited time (assuming you have a full time job as well on the side). Thanks!
I would recommend finding a local non-profit you're interested in helping, and start volunteering. Don't go in guns blazing "I'm here from Hackernews to save you" but get to know the people and what they do, and then how to help will become apparent.
By local I would recommend truly local and not a "division" of a national non-profit; those are an entirely different beast.
Absolutely. If you loaded this into an agentic coding harness with a decent model, I can practically guarantee it would be able to help you figure out what's going on.
> there is no more need for writing high level docs?
Absolutely not. That would be like exploring a cave without a flashlight, knowing that you could just feel your way around in the dark instead.
Code is not always self-documenting, and can often tell you how it was written, but not why.
> If you loaded this into an agentic coding harness with a decent model, I can practically guarantee it would be able to help you figure out what's going on.
My non-coder but technically savvy boss has been doing this lately to great success. It's nice because I spend less time on it since the model has taken my place for the most part.
There are so many blogs and tutorials about this stuff in particular, I wouldn't worry about it being outside the training data distribution for modern LLMs. If you have a scarce topic in some obscure language I'd be more careful when learning from LLMs.
They do. Think of it like a very intelligent but somewhat unreliable engineer you can hire to look at your code. They have no context about the codebase beyond what’s written in the source code, or any docs you give them.
What I meant was the docs might provide explanations about the problems the codebase solves, design decisions, the abstractions chosen, etc that wouldn’t live in a particular source file. Any discussion someone has with an LLM about the codebase will lack this context in the explanations given if docs don’t exist.
one of the apps I built is an app that you give it a website, it scrapes it, then places the google ads.
Over time it tries to improve the ads.
Right now it is showing a 5% click through rate over a few weeks. I have no idea if that is good or not, but it saved me the hassle of having to worry about google ads for another app
If the marketing platform works, then it would drive traffic to itself.
when agile was fairly new I worked with remote developers that had 3 locations.
My specialty is software requirements and my team was brought in to do the product management. The developers had read somewhere if you were using a database to do requirements then you were doing agile wrong.
They wanted me to write post it notes in triplicate, then fedex them to all their offices.
