My agentic workloads happen on two physical machines dedicated only to agentic workloads that share a VLAN with only eachother, with single token access to the specific repositories they are meant to push to and limited scope, hard limited credentials pulled from the hardware store on hardware token/fingerprint request only.
I view the safety around unmonitored agentic work in a similar way to how I view working with viruses or bacteria in a disease lab.
This stuff isn’t magic, it isn’t intelligent, but it is dangerous and needs to be handled with careful intent.
I haven't gone to network filter and logger yet, but I'm running mine in a VM, using lima (https://lima-vm.io/) to make it painless. Works on Mac and Linux, has an untested Windows version.
smolvm has better bootup time (<200ms), secrets handling, and egress control compared to lima. Among other things like native branching/forking of the vm itself.
As for the rug pull concern, I'm committed to open source with 2 pieces of evidence.
1. oss license
2. Business model is accruing revenue from offering hosted service + inbound due to the open source project. The business model + open source is aligned.
vz limits how much you can control the guest memory(doesnt expose a way to do free-page-reporting), has pretty terrible dis/virtiofs performance, and i've had issue with high host cpu usage. that said, iirc colima can switch to a krunkit backend so it can avoid most of these. ymmv but on benchmarks vz was 2-5x worse on a bunch of different ops
So far I've settled on one jumbo VM with many Incus containers inside. Nothing goes inside the VM that I can't afford for agents to exfiltrate. I've been really happy with this setup.
I think there’s something to be said for distributed systems that work equally well online and offline. I’ve built an issue tracker that uses Git as its transport, with no central broker or service and eventual consistency.
That means agents can collaborate on issues locally without needing network access to an external tracker, which could help with the sandboxing problem. You can then sync when it is deemed safe.
And most people running containers are running a default docker/containerd setup, which does not actually isolate you from anything at all in the first place anyways.
But then it decides to exfiltrate all your shit even inside the container. Which is something I'm trying to lock down at work atm.
Everything needs to go through a proxy, whitelisted domains only, plus credential swapping/re-signing so that the agent never has the real credentials. Docker sandbox looks quite good (September) but I've been using iron proxy (if the author is reading this, fucking thank you bro!) to great effect.
I still haven't cracked "what if the agent decides to drop parts of your codebase somewhere it shouldn't" yet, but I'll figure it out.
Been thinking about this a lot lately, and it’s a surprisingly difficult problem.
All it needs is public access to anything it can have the tiniest effect on, and it can exfiltrate data. Something as simple as a read count on some obscure forum post would be enough, albeit very slow.
Yeah, exactly. I'd say it borders on almost unsolvable for any agent of reasonable complexity.
You see some of the vulnerability chains humans come up with and they're just insane.
Even scanning network data for sensitive strings (say matching against a codebase or data set it works on) is tough because you can only check so granularly before you get loads of false positives.
But maybe like a "rolling" sort of check would help with that, % of outbound data matching private dataset (which should be operated on but not transmitted) over time. That way it minimises what gets out while dealing with false positives. Some biz thresholds might be >20% of data others might be >0.5%. Hmm.
I am tackling this using the Lethal Trifecta principle.
The idea is that agents never have all parts of the trifecta, maximum 2: untrusted content, private data or network access.
Harness have network access by default since they are driven by their provider API, unless you use offline models. So that's a problem.
I am then passing around messages that are tagged trusted/untrusted and the engine can monitor if any agent in the project ever completes the Lethal Trifecta and thus, is vulnerable.
Usually this means for most workflows, the "work" gets done by an offline LLM after some input/classification stages. There's not many ways around it.
> Harness have network access by default since they are driven by their provider API
The harness needs network access, but the shell spawned by the harness that runs the agent's commands does not necessarily need it.
Having a bwrap etc managed by the harness (DSH does this, among others) lets you have tighter restrictions on the agent's shell than the harness. Then run the whole thing inside a VM with a private network, so you can add nft rules to let it dial out to your inference machine but not to the internet at large. It takes an afternoon to set up but gives some reasonable defence in depth.
In terms of agent architecture I'm initially trying to minimise specialisation to make deployment easier on us. Infrastructure is becoming our largest blocker because it's otherwise so quick to get an agent up and running.
With your trifecta, let's say the three musketeers, how do you prevent sensitive data holding A from passing said data to B or C if they all need to collaborate to get a task done?
The data is passed as a message and the message is tagged as untrusted, so the engine knows it.
Agent A has internet access and the dangerous message, but no access to the secret company data, so nothing can get exfiltrated.
Agent B gets a tainted message from Agent A. Agent B can process the message using the secret company data. Agent B has no internet access and can only receive/send messages.
You can imagine an agent C that moves the work along or replies to an external user with the outcome.
Any workflow that has both an input and an output will complete the Lethal Trifecta as a whole, but no single agent does by itself. It raises the bar quite a lot, you need to confuse multiple agents in the loop to exfiltrate data.
Fun research, but it's a bit hard to see all my coworkers right now installing Claude desktop and moving on with no afterthought whatsoever...
Regarding infrastructure: my toy sandbox is just a docker-compose inspired YAML project file. So it's only containers. See my profile. The principle can be implemented anywhere you can run agents and cut network/data access though.
Yeah so ultimately it's still possible for bad things to happen but a lot harder, I agree with your point about having to somehow prompt inject multiple agents vs one (which I imagine is a nonlinear difficulty).
I'm using compose atm as well, mostly just because I need to prove a few concepts and committing to real (well cloud) infra is the slowest part of our dev cycle atm.
And yeah, many of my colleagues (and myself, I'm guilty too) run local code agents for development and we're just kind of...trusting them to do their thing.
But for autonomous stuff I'm assigned to atm I want to avoid any mishaps as much as I can, hence the effort to try and lock all this stuff down.
What's crazy is every day I'm finding solutions (not just to this problem, but in the AI space in general) that were released just a month or two before more often than not. It feels like 2010s node.js ecosystem all over again at the moment. Hopefully we settle into a few nice easy panopticon options that we can throw arbitrary agents into without a care in the world.
It’s difficult in theory, impossible in practice. For example: any external code dependency could be a prompt injection, and lock files are only TOFU protection.
If you can make any request to the outside world you can exfiltrate data, period. You can even include data in the uri or headers and they will appear in logs.
That obviously includes talking with the provider.
If you want it secure, it has to be off the internet and on your own hardware.
I agree of course, but those things are all the more obvious ones you can start to attempt to prevent.
Stripping headers, not allowing other than GET requests. Only allows HTTPS etc, the list goes on.
My point was that even if you locked down an agent to only ever read a single forum, and only to be able to access other post URLs that are from that forum, and not allow any query parameters that aren’t necessary etc.
Even then you can be having the slightest public effect that can exfiltrate data - a read count on some old post. I know it’s wildly impractical, it’s more a thought experiment on how hard it it.
You would need a human reviewing every single action of the agent to prevent them from exfiltartimg data. There are just too many possible side channels for a predefined solution. I believe currently what you want is a locally hosted decision model scoring every agent action againt a set of rules with a human in the loop once the decision model isnt 'confident' the actions are harmless.
I've been thinking a lot about this problem. Also, I think like the idea might be to look at the issue with code mode. Like basically, we know that all agents can write bash and execute it and that is what agents default code mode is.
But like if you only allowed for example JavaScript code that could be executed in a WASM sandbox, then you would have better control instead of just locking down with proxies. Basically there would be a very large API, basically tools, that the agent would codemode execute. My guess is that this would need a tool router to optimize the loading of these into the context. But it would give you very fine control over what code execution can and cannot be done.
Depends what you mean with "agents"... LLMs in coding harnesses, or fully autonomous agents?
If you mean the former, I think you'll find probably 90% of devs are "crazy", running Claude Code or Codex on their bare metal using the auto-classifier.
I used to always run claude with `--dangerously-skip-permissions` in a dev container, but I had to rebuild my container 10 times a day to update the firewall so it can access documentation.
Now I find the included auto-classifier is very reliable. If you keep tasks contained instead of open-ended long-horizon work, it doesn't seem much goes wrong with frontier models. The worst unintended action claude did for me in the past months is that it once discarded an unstaged file which wasn't committed yet, without asking.
But I realize it's still playing with fire, no matter how good the auto-classifier is.
As an example I run opencode and anything related agent-related in its own dedicated KVM VM with its own LVM logical volumes for storage on a headless server. This has no access whatsoever to the personal data that lives on my laptop and workstation desktop PC. The VM gets periodically imaged and backed up, and is relatively easy to move between host machines if I need to.
One benefit of doing this is that I can leave it doing long term slow things with slow but capable LLMs on a truly 24x7 basis without worrying about it being interrupted by a laptop being closed/going to sleep, or workstation level software updates/reboots on my primary desktop PC.
Thanks for sharing this - I've been trying to do something similar but have not had success. Can you say a bit more about the details? What is your headless server running (Ubuntu Server?), how do you connect, how did you set up the KVM VM, how do you move it if necessary, etc? Thanks!
In my case it's a fairly boring proxmox server. The KVM VM started as a bare bones debian install and then I added xorg and xfce4 and the tiger vnc server. I have a GUI on it for when I want to use it that way (in screen dimensions that are a 1:1 mapping of the native resolution of my laptop for full screen VNC session), or of course I also talk to it by regular SSH session, and screen or tmux. The VNC server listens on 127.0.0.1:5901 and then I use SSH port forwarding from my workstation machine to connect the macos tigervnc client.
For people who don't want to do it with proxmox you can also do the same with manual KVM command line tools (virsh, etc) with no GUI. Or as something like a Xen dom0 running the guest as a xen HVM or PVH guest. Lots of ways to do it.
If I need to move it I can shutdown the VM and take an image of its LVM logical volume storage device (by something as rudimentary as like dd into an xz file), move the file, then create a new identically sized logical volume and do the reverse using dd to write from the file into the LVM logical volume location on the destination.
Edit: It would also be possible to do this all in one machine by something as basic as running virtualbox on your laptop, make a disk image file, installing the VM in that, give it bridge mode access to the physical network adapter to get its own DHCP lease in your same LAN segment, and treat it as a logically separate computer. There's minor risks of theoretical hypervisor escapes from the VM on the same machine, but such things tend to get patched fairly quick.
sort of. It does a lot less than Roam, etc., which is good because Roam is complicated. In my experience the simple model of obsidian is just right, and the implementation is excellent.
It was only created as a workaround to Google killing proper adblock related API support for manifest v3, it's definitely worse than regular ublock origin.
The article doesn’t mention any of these namespaces failing as the major issue… he is referring to the challenge of transferring hosting providers, which could be done for various reasons.
That is a failure; the old namespace no longer meets your needs. My point is abstracted over all of that. There is no namespace solution where that can't possibly happen. Switching from one solution that has problem X (and more generally than the article describes) to another that still has problem X isn't a solution, and given that problem X can't be solved, that means there is no solution. In the interests of not spamming replies, this is also my reply to
dcreager.
Yes, but the solution differs. As the author says — updating DNS entries can be considerably less complicated than migrating libraries and their references
In this example, the old namespace is perfectly fine. The issue is the hosting provider. The point is that you can (and probably should) decouple the namespace from the hosting provider.
I've sent in corrections to Apple Maps in Portugal (shopping mall is 1km away from the location on Apple Maps) and Canada (speed camera does not exist). Both were taken care of within a couple of days.
but then again Apple Maps seems (certainly here in London) to be absolutely full of geofraud entries and out of date slop, such as small businesses with linked websites that have been shuttered for years, and strange businesses located in the middle of playing fields, parks and lakes.
Even assuming the end-state of all this is Trekonomics -- consider the amount of retired folks that generally just sit around and watch TV. I think a massive culture-shift is in order before we can even consider a healthy society consisting of 0 essential employment.
reply