Even so, if you can manage to get a agent into the loop of "change variable > inspect results > do change > repeat", you can replicate everything from server-side game servers to audio effects, as long as you manage to get the data into a format they can either "visually" inspect, or inspect by data. For audio, I've had success replicating synth sounds and effects by giving them MIDI access to analog hardware, spectrograms and a suite of other unix tools.
The tricky part is figuring out the workflow, once that's in place, you can encrypt the protocol itself as much as you want, the output (audio, video, JSON, whatever) has to go somewhere, and that can be captured and replicated.
A lot of the security features on microcontrollers are poorly implemented. Here's an example where someone recovered the encryption key on a consumer device with encrypted firmware: https://garyodernichts.blogspot.com/2024/10/looking-into-nin... . The microcontroller supports firmware encryption with a 128-bit AES key, which gets written to 4 32-bit write-only registers. Because it's split up like this, it's possible to find 1/4 of the key by writing every possible value to one of the registers and checking that the output matches, before repeating with the other 3 registers. This means that the number of possible combinations to check is reduced from (2^128) to (4*(2^32)), which can be brute forced in a few hours.
I wouldn't be surprised if the next version of Opus or whatever Anthropic has just eliminates these capabilities. I mean, we got DMCA shoved down our throats. What's stopping them from doing something similar again, I mean, we have a 70 or 70+ year copyright laws now, lol!
> Software engineers are getting laid off because of advancements made by AI.
This is said by a lot of CEOs trying to make layoffs look good, but I haven’t seen anything that suggests that it’s true.
> Yet I see far more of these blogs about the plight of mathematicians, not enough about the plight of software engineers.
Really?!? We’ve been living through 3 years of blog posts, LinkedIn hot takes, and podcasts talking about the plight of software engineers in the era of AI. There isn’t a week that goes by without an HN post about it.
I can feel empathy for the practitioners in a field that may have been profoundly changed, the same way I can feel empathy toward software engineers who are struggling with the changes in their own field.
In the end, we’re all trying to figure out where this leads. And how this will affect our lives and our work.
> Assume the model which initiates the worm is a high powered model.
The disconnect here is that highly powered models have billions or trillions of parameters and have compute requirements that scale with the parameter count.
If this became feasible with smaller models, we’d be in a very different world. One where you’d have your own local Fable/Opus/Astra on your desktop.
If we get there, this becomes somewhat believable. Right now it’s just too many orders of magnitude of memory and flops between the many H100s that run the current models and what smart devices have.
Your typical ESP32 chip, which is really beefy for embedded, tends to come with <1 MB of RAM and similar amounts of NVRAM. It runs two cores at a few 100s of MHz and does a bit of SIMD but it was designed for embedded IO (I2C buses, etc) not matrix multiplication. And its network speed is nothing like an Infiniband interconnect. You’re lucky getting 802.11b Wi-Fi.
> Instead of infrastructure being more easily shared with the collective, or valued enough to build a business, now companies can tailor even major infrastructure to their own needs, and avoid all that coordination.
Large companies have always done this. Tons of early 2010s open source projects came out of reimplementing white papers from closed-source Google projects.
It’s also the major downside of this kind of thing. Once someone gets acclimated to how things are done at a large company with custom tools, it’s not always easy to switch to another process, especially a simpler one. There are tons of stories about FAANG engineers joining a startup and being useless.
There’s real value in having consistent tools across an industry. It makes hiring and training easier. Even if you’re convinced that a LLM could crank out a payroll or ordering system that’s perfect for your org, you still have to train the accountants and admin assistants to use it. And you’ve now become the customer support.
The bar is lower with version control, but it’s still more friction than git + GitHub + a well-known workflow.
> I hypothesize that the kind of reader who wants a physical copy of a book is much happier to get a hardcover.
Nope. I vastly prefer paperback. I don’t like ebooks much, and hardcover books are too large, too heavy, and take up too much space on a bookshelf. For books I know I’ll re-read, I might go for one, and I’ve certainly made some impulse HC buys that I regret, but I vastly prefer paperbacks.
One of the best things about mass market paperbacks was you could fit one in a jacket pocket, and easily carry 1-2 around with you.
No, it's not how math works at all. Math is not experimental science. Results take years to formulate and are generally known to be true by colleagues via correspondence and seminars before a paper is even written. Retraction is very rare.
I don't know why people are talking about something like this that they are clearly not familiar with.
The greatest proof ever written by man, Andrew Wiles' FLT, was published with a flaw before being retracted and reworked. It's _completely_ normal in mathematics to find flaws in the argument. That's what peer review in mathematics is _for_.
Besides which it's not clear if these papers are considered published or preprint since they appear in no journal, so it's not really a retraction.
It's also very very normal to post preprints on Arxiv before peer review, so it's not the case that mathematics is kept private during review.
Not true. Wiles presented his proof in a series of lectures in 1993. The error was found by a reviewer, and he spent over a year trying to fix it. Once he did fix it, he asked colleagues to review it again, and only once it passed that review did he actually publish it.
You have no way to know whether the ideas presented are the authors own or whether they are wholly from the LLM. Nevertheless I think rejecting text purely on the basis that you think it is ai generated is a form of ad hominem.
This is pretty typical product positioning. You want to sell to both high-end and low-end users, so you offer products at a few price points. Then it turns out that that middle is a much better fit for most users. So you start making the middle a worse fit to push most of those users into the higher tiers.
Long term, this only works if you have a non-commodity, and if the higher tier is actually more profitable. We'll eventually learn whether both are true. For OpenAI right now, it's probably enough to just increase revenue, even if the higher tier is even less profitable.
The 200$ plan was appealing because you got 4x usage for 2x the price.
Now, as it's linear, it makes much more sense to downgrade to 100$ OAI and pick up a 100$ Claude sub. (without doing the numbers) the usage should remain the same, total paid the same, but having access to best of both worlds. It should be a win for the user, and a loss for OAI.
With this in mind, it sounds like a fumble by OAI.
The vast majority of their revenue comes from large businesses buying for their teams, which are almost certainly not going to juggle lower tiers of different subscriptions to save a few bucks.
If we're heading to a world where AI spending for companies will be close to salary spending - which is questionable, but is the only way future in which OAI/Anthropic survive - you will most certainly have people whose full-time job it is to juggle providers and figure out how to save a few percent this month.
Maybe. But developers are also private people who will also play with their private subs and projects. In my opinion, their private experience might influence some corporate level decisions.
Being grandfathered by OAI and happy is not the same as having both, and noticing "hmm maybe Claude is much better for my case, Ill suggest that to our manager"
Every dev I've worked with hates Jira and yet it's by far the dominant project management system for software companies. Software devs don't have as much sway as you think at most companies. (I realize that the cost of switching from Jira is higher than the cost of switching LLM providers, but I think the point still applies. I've seen this same behavior with dev tools as well.)
This is what I did. Hope it works out. The other benefit is you have a more natural method to avoid lock in. A lot of "improvements" to the agent harness I believe are attempts to build customer lock in.
It’s not ideal. It has serious drawbacks. It makes development and updates a PITA. But it does make it much harder to decompile your firmware.
reply