If there is very little benefit to them by releasing it, that's a lack of imagination on their part. Being able to disrupt Microsofts, dare I say, number one beachhead into developers and their development ecosystem and bringing AI first developers into Anthropics own vertical AI development ecosystem is pretty damn core to winning the future potential avenue for the software industry at large. It won't pay off in the next five years, but it could pay off big in ten.
Imagine waking up next week and Claude Code now ships free with builtin source control management via this tech with a better user experience than workspaces. Github and Microsoft would take an instant market share hit in the traffic that they swear is causing all of their outages. And then that cuts off Microsofts access to code to train their own LLMs on in the background, further hampering an Anthropic competitor in the AI race.
Agreed, they probably don't. But the financials of other companies clearly say that they are not above leasing further compute capacity from Oracle and X and would be part of a narrative that they need to IPO to raise money for such projects rather than it just being a naked payday.
I think honestly they should focus on their core competency. Microsoft’s biggest problem is the total lack of focus. GitHub is just a great example of that total lack of coherent direction or product vision. I hope Anthropic can avoid becoming everything and nothing at once as Microsoft has done.
I think Anthropic, a company with no historical skin in the game can come at the problem clean slate and only need to take on the core enterprise coding component. They don't need to worry about being production infrastructure, the don't need to be a connector to their own cloud service like Microsoft needed to feed into Azure. AI isn't something to attach onto source control/cloud, rather the source control is something to attach onto AI development. I don't think Anthropic needs to become a Microsoft competitor, I think they're going to need to become an Nvidia competitor in building a vertical enterprise AI company. Not a cloud, desktop OS, search, gaming or browser company. Just deliver the end to end code tooling cycle. Which probably means in the future they're also going to need to make inroads on Linear/Atlassian, but that's a separate conversation.
I have a theory (loosely held, it's just an idea, not a belief) that this could lead to something I call Evolutionary Architecture. That instead of waiting for vulnerabilities to be discovered in code, that systems could end up with code being changed and re-implemented by agents so fast that attacks aren't able to probe for attacks because they end up probing code that by the time the answer is come it, in production that code no longer exists because it was moved to another language or something. The change to prevent vulnerabilities isn't the reason for change, but the change has defensive side effects.
All of this may be slightly inspired by the Militech neural matrix in Cyberpunk 2077
I've been developing a weaker version of this line of thought for code reviews for large architectural features.
A big motivator of thinking architecture through really well is that although the software may work just fine now, a good architecture will make it easy to add a feature a year from now, whereas a bad architecture will necessitate significant rewriting.
But significant rewriting is cheap nowadays, so in code reviews I put a lot more weight on whether it works today, and a lot less weight on whether we'll need to change it when making a different change.
Entire classes of things like "should this be in this component or split out into its own" just don't matter anymore because no matter how tedious changing that might be once there's a ton of things depending on it, that's the kind of thing AI can do no problem.
I think this works well until you get into data, state, history, external interfaces etc. Large scale data migrations are still risky, anything with external consumers you can't quickly change is risky, etc. Internals inside a service I agree, yeah you can let it slide a lot.
I also found though, with agents you can quickly create large amounts of code, when you go to refactor / fix some design aspect that touches large parts of the system this consumes a lot of tokens and actually still takes meaningful time.
It's like, with a digger you can quickly create a hole! Digger can quickly clean up the work of 10 people with shovels! But if you use a digger for 1 week and you were putting the dirt in the wrong place, you find you have a HUGE pile of dirt to move and will still take a long time to clean up.
Directionally I agree with you, but I think design still matters.
It seems those classes of things are more important than ever. It's not about how tedious it is - it's about the models being able to build accurate context and understanding. All the same things that made it better for humans also makes it better for models. More specifically in this context, well-maintained code results in fewer bugs and lower token usage.
I tend in the other direction when I can (for components and microservices - harder for monorepos): I encourage refactors on a continual basis, as it prevents the models from getting bogged down with the way the system used to behave across changes.
TIL! Close, just taken up a notch that wasn't feasible before where MTD was more configuration focused. As CPUs are to FPGAs, MTD would be to this. Rather than telling the config to change on the fly, the code changes on they fly. And rather than having multiple Binaries prepared, the pipeline of changing binaries are more just-in-time than batched.
The Military has the "Survivability Onion", Security through Obscurity is the "Don't be seen" layer, while "Security by Velocity" would be the "Don't be acquired" layer.
It's like when Tony Hawk did a 900 for the first time. Now 900s in skateboarding aren't a big deal, kids can do it now. It was proving to the world what was possible was the mental hurdle that inspires others to actually try at the problem harder.
If AI development is like management, harnesses will totally end up orchestrating people like traditional management does. The end state of the Jev type model hype is that a few years from now decision models replace management decision. Disconnected human managers five levels away from the work make suboptimal decisions all the time, the same logic applied to self driving cars can play out here. The model doesn't need to be perfect, it just needs to make better informed decisions than your fallible VP.
This is great, more access did provide more eyes on these problems.
But, does that all of these being found now call into question, not the open source model logic itself, but the ability of human eyes to find security issues? These vulnerabilities have been sitting here for however long, but how many thousands of humans did not find them before AI?
The threshold for Microsoft and Apple to actually report vulnerabilities is MUCH MUCH higher than for Linux, and open source as a whole. They generally only disclose issues in Windows and macOS that are quite serious and impactful.
For Linux, the threshold is nearer to the point of it being questionable whether a bug is even exploitable on a real production distro, compiled and run with any sort of sane configuration.
Even before AI we have known that no one is smart enough to write bug free C. And with every bug being a launch platform for a full exploit it's become a big deal.
This is the reason why performant and "safe" systems languages are on the rise and being accepted into foundational areas of our operating systems (such as the kernel). When the attack defense surface is tighter (language, tooling, compiler instead of the actual code itself), the smart folks can stay at that layer, while the masses can write more code at a level of abstraction that nullifies many of these vulnerabilities by default.
I can't find it now, but I heard that NASA developed processes to produce completely error-free code (for the moon landings iirc). The problem is that it's incredibly time consuming and expensive to do.
Like everything in CS, apparently this is a trade-off, not an absolute. You can get bug-free code, but it's not commercially viable and is extremely tedious to do.
It’s probably about how they write the space shuttle software, and it’s quite a famous article. The original is now paywalled but there are many copies.
This is example of what I call "the tipping paradox", and I'm almost sure that this phenomenon has its own proper scientific name.
When asked, people prefer €15 burger no tip, but when actually making a choice, they prefer €10 burger with €5 tip. Similarly, companies state "bug-free code" as a goal or requirement, but then they prioritize other goals over code correctness. My workplace is in the process of completely removing code reviews. And actually, I don't disagree with the decision - my career is short, but I have never seen reviews fulfill any purpose other than to share the blame in case of an incident.
They are removing reviews or just human reviews? It’s interesting and I kind of agree to a certain point that human review is not the best way to find and fix bugs. We’ve had so many bugs in code that was fully reviewed by at least 2 humans. Ai review is clearly superior. Human reviews main purpose to me right now is to spread know of what is being worked on.
Some vulnerabilities are incredibly complex and are identified not through the code itself but through various attack chains being strung together.
Also there are likely a lot of vulnerabilities identified but the work required to fix them vs the complexity to exploit them means they don't get fixed.
I'd wager we don't have an issue with identifying vulnerabilities but the ability to fix them.
I work in security consulting and identifying vulnerabilities isn't the difficult part its actually fixing them and fixing the ones that have valid exploitable attack chains that matter
Also, while it's clearly got a lot of training on some use cases, others that probably weren't in the training set have worse good decision rates than a random number generator. If you can rebuild the architecture, you can train it on your use case.
reply