>Couldn't this just be a result of Astra training on massive amounts of circuit theory or mechanistic interpretability literature or other related topics?
Maybe, but it's more like Astra has been trained on enough practical linear algebra problems to be good enough at "eyeballing" how adjusting some weights will result in different outcomes (among other things).
To put it another way: a human can learn to do what Astra is doing here with enough time and assuming the weights are small enough to be practically comprehensible. I know I've gone through the practice of whiteboarding the training of simple neural networks on a whiteboard to understand how they work, so it's not an absurd concept in itself.
The scale of this, though, is certainly extreme and definitely not something that I would have thought any publically available model could handle at this point. It's worth noting, though, that there's virtually no way these frontier labs haven't been exploring this exact concept for a while now.
>In other words, not black magic?
It's just math. Extremely complex and impressive math, but we're still just talking about matrix multiplication under the hood, and none of this suggests anything otherwise.
>It just doesn’t line up with my understanding of how these models work. Is there really enough training data on direct weight editing to give a model this skill?
I mean, this question indicates that your understanding of how these models work is, indeed, lacking.
Despite what some people having being hollering about, treating LLMs as just "stochastic parrots" isn't doing their capabilities any justice. They are constantly exhibiting emergent behavior which wouldn't be found directly in their training data. They are capable of combining different ideas that they are explicitly taught to perform novel tasks.
I'm not saying you're guilty of this, but the people claiming LLMs just regurgitate existing data don't understand how LLMs, and likely their own cognitive experience, actually works.
This hand-wavy phrase has applied to neural networks in general since it was possible to train them. That was more than half a century ago. It means both nothing and everything simultaneously.
Every learned ability of any neural network is an emergent property.
>It has no place in a progressive society where being duped with propaganda into believing falsehoods should not be something to look up to.
I think this really misses the nuance of all of this. To put it bluntly: there's too many dumb people who need to believe in this kind of "magic" to be able to reasonably ignore it on a political strategy level. Just trying to treat everyone as intellectually-capable of being presented facts and making a logical choice clearly doesn't work, and the opposition is perfectly willing to exploit this dynamic.
This post is funny to me, because I feel it falls well short of what a proper, forward-thinking "AI-native" engineering org would actually look like. That is, the author is presenting this as kind of a radical departure from normal engineering, but I just see it as normal engineering with more AI. I don't see that as AI-native.
To me, true "AI-native" engineering will be one where questions regarding how humans will do things like review code are irrelevant. You explicitly throw away any concept of typical software engineering and, instead, differ to the AI. Your job, as a human, is solely to do what the AI can't physically do. That's not the same as closing gaps between what AI is currently capable of doing and what you'd need it to be capable of doing. It's literally enforcing that anything an AI is able to do is treated as correct in itself, and you, as a human, only support it by being a human.
If that sounds outlandish, that's because it is. We don't have such capable AI systems yet. It's possible we never will. But that's beyond the point. The reality is trying to make an "AI-native" engineering org by just layering AI on top of existing processes and systems is futile. If your expectation is that AI will continue to improve in capabilities at an accelerating rate, then these stop-gaps are pointless. You might as well just ignore AI altogether until it is able to completely take over, because that's the inevitable goal, right?
If you're willing to buy that will be possible, then my guess is that it will occur in the form of small scale startups and actual mavericks rather than someone trying to run an existing business who will simply be too risk adverse to go all in when the time comes. Some guy in his basement is more likely to leverage AI "properly" to build an actual "AI-native" engineering org because that person won't have much to lose.
> If you're willing to buy that will be possible, then my guess is that it will occur in the form of small scale startups and actual mavericks rather than someone trying to run an existing business who will simply be too risk adverse to go all in when the time comes. Some guy in his basement is more likely to leverage AI "properly" to build an actual "AI-native" engineering org because that person won't have much to lose.
FWIW I think the company writing this post is one of those small scale startups. But that being said, you're not the only one who thinks this. A decent amount of VCs and funds think that only smaller, newer companies have the potential to be truly AI native because there's too much organizational inertia in an existing business in keeping its existing human processes and so only a newer company can achieve the true "scaling potential" or using AI to scale without human labor.
IMO most modern engineering practices are cargo culture. I’m sure in 10 years we look back on today the same way we’d look back on the days before version control was common. We are in the emailing files around stage of development
> I feel it falls well short of what a proper, forward-thinking "AI-native" engineering org would actually look like. That is, the author is presenting this as kind of a radical departure from normal engineering, but I just see it as normal engineering with more AI. I don't see that as AI-native.
A company which proclaims they are "AI-native" isn't actually interested in engineering, they are trying to ride the hype wave. Companies who care about engineering just... do engineering, and don't make a big deal out of whether they are chasing trends.
I think this is all fair, but somewhat just semantics.
Most engineers are working in "normal" engineering orgs, but can feel the ground underneath them shifting, and companies believe that if they are caught flat footed, they will be out competed in the marketplace, so they are all trying to make sure they are adopting it at least in a way that lets them keep pace if not accelerate revenue.
If you are not impacted by economic incentives, you can continue to not care about this transitional state, but most people do.
Are there any companies outside of the "AI shovels" business that make more money because of AI? Every time I ask someone why they went so hard on adopting AI, I hear the same story about the fear of losing the market share or becoming less competitive. Honest question, who is benefitting from all of this? The revenues at majority of companies seem flat at best. There is a growing concern about professionals getting deskilled, the job satisfaction is not improving, wages are stagnant. Mind-boggling that we are actively engaged and often push for this experiment that hurts our own profession. Cheering for our own demise.
Not keen on outing myself by providing evidence, but I think we do generate more revenue by using AI. We launched new products with a small team and also optimized our outcome-pricing based product using AI several times. We also reduced costs by self-hosting more software.
But we're a tiny baby startup, so things like "spin up a new product with 1 engineer and get to $1m in revenue on it" or "reduce our IaaS bill by 300k/yr" is a meaningful contribution for us.
We're also building new AI-enabled software that was not possible before.
The big breakout hits have been AI Infra, but that makes sense from a technological perspective, Cisco made a lot of money well before Google did, but Cisco is not where the money concentrated in the end.
We don't really know what the equivalent of Google as an application business will be in the AI era, but there's a pretty big chance it's not actual Google. But Google still has to do it's best to hold on.
Engineering’s job has always been to deal with hard technical challenges and apply new technologies.
Writing code was never the point. Predictably creating specific outcomes was. Wrangling AI in a way that multiplies productivity is the new job.
I won’t lie, it kind of sucks to be young right now, learn about punch cards and assembly in school, and then graduate into a market that’s moved on to Fortran. Sorry, I meant high level languages and AI. To say otherwise might imply the industry has seen a similar shift in the past where engineers made the exact same complaints practically verbatim.
Regardless, the reality is that college teaches very little about 90% of real engineering work anyway, and we still need to hire smart people for this new type of work.
I use AI compatibility as a metric when evaluating new software to add to my stack. If an AI agent isn’t able to do most of what you can do in the app by opening it, I prefer to look for other options.
Does not apply to everything though. E.G: I don’t care about AI support for my non work chats/email, since I’d rather not give it access.
This seems really really easy to do even if your org isn't using AI at all. You certainly don't need to be "AI Native" to implement an MCP server for your webapp.
Reminds me of a few things, like being a meat proxy, as well as books like Company of One or Million Dollar One Person Business, and these were published before AI so right now we will see companies with only one founder and hundreds to thousands of AI agents. There already was that drug delivery company with the one founder but apparently a lot of their revenue was fraudulent. Still, it proves the point that it is possible.
I was misremembering, they were sued for spam laws, not for their revenue being fraudulent so it seems it's legitimate, making a billion in revenue with one person (and later on his brother who was hired), the rest is all AI.
The reality here is even more idiotic: someone criticizing a company for a terrible strategy they aren’t using that wouldn’t work if they did. According to the critic!
If you follow this logic, we get to the true "AI-native" engineer which is ecstatically floating in a bath of dopamine-stimulating liquid, while AI reads their brain pattern and instantly builds everything they didn't even form a full thought about.
"If that sounds outlandish, that's because it is. We don't have such capable AI systems yet."
Duh, the idea that those capable of actually utilizing these models for these purposes weren't aware that they could use these models for these purposes up until Anthropic released a blog post about it is silly. Anybody who is even a bit tech literate has been well aware of this inevitability for basically as long as open-weight models have been a thing.
The timing of all of this is much easier to pin to the creation and proliferation of capable of open weight models. Anthropic calling this out is not the cause for it; it's just an obvious correlation.
I got more utility from AI within the first week of ChatGPT being originally released than I've ever gotten from crypto, and I've had bitcoin since they were giving it away for free on Tor lol
They're not even remotely close to being comparable.
Honestly fascinating. I almost feel like you'd be better off avoiding any discussion of the fact that this runs on a Raspberry Pi Pico 2 W because it takes away from the crazy world model capabilities (of course, the fact that this does run on a Pico 2 makes the whole thing much more impressive).
There's gotta be a good way to incorporate this logic into a larger, more "traditional" game in a meaningful way. Seems like an approach for procedural generation that could solve some of the issues usually found with actual procedural generation.
Even more importantly: it's not an either/or situation.
You can love to cook and enjoy every second of it, but sometimes you just need fuel and that $3 microwave burrito is sufficient. Just the same, 99% of your means could be frozen TV dinners, but that doesn't mean cooking a romantic meal for Valentine's Day is off the table.
There's a weird level of fear mongering occurring around AI that it will make you incapable of independent thought if you use it. People will throw around very light research that kind of points in that direction, but they'll never highlight any of the nuances in such an argument. You can use AI and still "think" if you choose to, just like how you can avoid AI and still opt out of "thinking" if you choose to. It has less to do with AI and more to do with the context of the individual using AI.
I think what Martin is hitting on in the article is though is that as more and more of the world’s communication is generated by LLMs we’re naturally going to adopt LLM-Voice into our own communication style. It is just as much about consumption, which is less in your control.
I mean, that's the main reason. How would you expect to see the kind of hype that Claude or GPT can generate when nobody can use it? The hype of those models are primarily generated from people using them to build impressive things and share those results. Can't do that with an unreleased model, right?
Beyond that, it seems a lot of people have learned that these agent leaderboards really don't tell the whole story. They're a good litmus test, per se, and there's plenty of hype being generated around these scores as-is. But anybody using AI in practice knows that benchmarks can only say so much, and one model scoring better than another doesn't necessarily imply that that model is actually better in the ways those people care about. You can see that in this thread as people are pointing out that even these benchmarks seem to suggest that Opus 5.5 is still leading in coding (which is what a lot of people, especially on HN, ultimately care about).
>just not enough publicity or hype as the other labs do?
Well, it's effectively both. Google is certainly not that pressed to generate as much hype as these other companies. They can afford to take their time. It's also worth noting that Google is definitely targeting different consumers, investors, etc., so their strategy will definitely be different.
For what it's worth, I've been using Argon since it's been available in Cursor and I really like it (at least at its current price). I think Google really did close the gap a meaningful amount (maybe not all the way, but enough to prove they're still in the game). Carbon being significantly better while being the same cost is pretty exciting to me. But until I can use it, I don't really care about it.
Maybe, but it's more like Astra has been trained on enough practical linear algebra problems to be good enough at "eyeballing" how adjusting some weights will result in different outcomes (among other things).
To put it another way: a human can learn to do what Astra is doing here with enough time and assuming the weights are small enough to be practically comprehensible. I know I've gone through the practice of whiteboarding the training of simple neural networks on a whiteboard to understand how they work, so it's not an absurd concept in itself.
The scale of this, though, is certainly extreme and definitely not something that I would have thought any publically available model could handle at this point. It's worth noting, though, that there's virtually no way these frontier labs haven't been exploring this exact concept for a while now.
>In other words, not black magic?
It's just math. Extremely complex and impressive math, but we're still just talking about matrix multiplication under the hood, and none of this suggests anything otherwise.
reply