Hi Phillip! CMU alum here - super exciting to see this on the HN homepage :)
How does this compare to some of the efforts led by David Kosbie & others around open access online CS education? I recall this being a big push back when I was there (2014-2018).
Is this essentially similar spirit, but covering actual SCS curriculum, rather than "pre-SCS"/highschool content?
Absolutely! And I've worked in open learning my entire career too. CSAcademy, which Kosbie and Mark Stehlik created, has been a fantastic effort, focused on intro CS and high school learners. Our aim is to have courseware aimed at a global audience across backgrounds; we are targeting early to mid career upskillers, but all are welcome.
No, he does, and it's extremely good. It's completely replaced herdr for me and I run it on Linux (server), macOS, and my iPhone. I've yet to run into a bug. Access is being controlled via Discord and TestFlight - you can find the link to apply on their site.
Can you describe the iOS experience? Is it just a terminal that mirrors your sessions from the server? I have yet to encounter a good terminal experience on iOS that doesn’t make me want to rip my hair out so if they have some secret sauce to make it enjoyable I would be interested…
It has its own server-client format that replaces both the ssh / mosh layer and the tmux layer, allowing the iOS app to connect to existing sessions on the computer (also the Mac app can either connect to the local Rex server that is included in the app, or to remote servers over Tailscale).
It doesn’t replicate the “remote authentication” part of ssh, and leaves that to Tailscale instead.
Rex has "sessions" which are like groups of tabs and it is scriptable, I made a little plugin that when invoked pops open a floating terminal asks for a worktree name and sets up a new worktree.
i'm not sure what you're referring to - if herdr had worktree management features, I never used them. that's not something i'd want my terminal multiplexer to do anyway, our worktree automation is part of the repo.
Herdr made it quite nice and easy to have different worktrees/spaces. Eg. I wrote a simple plugin that rsyncs the dev repo (including uncommitted, untracked and gitignored files like `.env` and generated code, so it's ready to use right away) and create a git worktree and open it as its own workspace in the sidebar.
I see Rex has the concept of sessions and tabs within it, so am wondering how are people working and switching worktrees within it, automating flows etc?
> Some time ago, we were made aware of multiple attempts to exploit our permissive licensing to hijack our work, with a direct view to replacing the project and aggressively monetising Linux on Apple Silicon.
Is this referring to Omarchy? Anyone have context? TBH I'm surprised their work was ever MIT-licensed instead of GPL.
The whole reason why people are trying to replace them is because they have stubbornly refused to use AI, which has directly impeded progress in getting M3/M4/M5/M6 working. It's fine of course for them to continue pursuing their hobby of doing everything manually, and honestly it is insanely impressive.
But most people just want stuff to work, so it's odd to complain about it when others do manage to get it to work using AI to build on top of their prior permissively-licensed work.
Wanting stuff to work is a valid reason to not use AI for more sophisticated development projects. Having a computer that kernel panics once every three days is useless to me, even if it works for you.
A kernel panic is the least of your worries with this hardware. Asahi had to introduce a custom daemon to avoid frying speakers. Apple doesn't need to care about making the hardware interfaces safe if they control the entire stack, so they don't, and expose dangerous capabilities to the OS.
There was a fun period of time where Boot Camp still existed, but Windows could fry the speakers because it didn't have any of macOS's thermal protection. I think this was after the 2016 redesigns, as I had a mid-2015 with no such issue.
In the case of speakers, though, what would a hardware failsafe even look like? It would probably be some sort of secondary chip; Apple's design is to just move that to the main processor, the same way they don't have separate SSD controllers.
Monitoring the CPU temperature feels very different to me. I'm pretty sure the CPU can't cook itself. This is probably documented somewhere so we don't need to speculate, but I can't research right now...
Good thing they only ever write correct code which consistently obeys every invariant in that entire stack ever. Excellent design, meet perfect engineering.
Do the massive amount of vulnerabilities not make you think twice?
At this point, I trust human code less than llm code. Design and architecture of the code still greatly benefit from a good human touch, but I'm not sure how long that will remain true.
So validate with an llm and humans. People committing code they haven't read - which will inevitability happen - is dangerous and hard to remotely verify.
In the end there is always a separate reviewer. Even if the producer is adversial and not even looking at the code then this process exists to catch it. This is not at all different from people attempting to get backdoors into Linux.
People using LLMs put up much larger changes, and much faster. Depending on the LLM used the quality is often much lower.
A lot of projects just stopped accepting any outside contributions rather than deal with the tidal wave of slop they were getting. Choosing to not accept LLM code is IMHO a better compromise.
LLMs will introduce new vulnerabilities. What Linux should have done is stop using non-standard C, or better, stop using C completely in favor of Rust, support code analysers.
And humans can introduce new vulnerabilities, too. Linux maintainers accept patches from random people over the internet. They already have to just the patch on its technical merit, not on who submitted it. For them I don't think it makes a difference (technically) if the patch was written with the help of an AI.
I believe that there may be valid reasons to refuse anything that was touched by an AI, but that's not about security. It's more about the non-technical problems coming from AI (centralising power, increasing inequalities, carbon footprint, ...).
Some people will always disagree with something. That's what's great about free software. There will probably be some Luddite anti AI Linux fork at some point.
That's not what they meant. It's AI friendly, not slop-friendly.
There's a reason no Omarchy patch of any kind has made it to the kernel list yet. I did personally look into some of their code and it is certified, hacked-togeter slop.
Exactly. As an example, read that Gravity Linux guy's responses here. I will never ever trust him. I will stay away from his code as far as possible. Asahi team: I have my trust in you.
I'm pretty sure Asahi devs experiment with LLM tools, but not to Omarchy's extent. LLM ban is purely a filtration filter, so that they wouldn't be spending time overbearingly exhausting number of patches generated with AI, and not someone who cares about this than a shiny tool.
Everyone understands that LLM is a mean to an end, but it's clear that Omarchy got two confused and it's showing.
* Author posts patch with a manageable number of lines so maintainer doesn't gets burned out reviewing it.
* Tries to outline on the body what the patch is about and his rationale for the design decisions on the code.
* Promptly answers any maintaners questions about the line changes.
* Discloses which AI model he used to assist on the patch and how was it used.
AI slop patch:
* Submits patch with huge lines of code without explanation.
* The body is written in a self-patronizing tone with superfluous formatting that helps nothing but annoys who is looking for the actual substance of patch
* When questioned by maintainer about some line changes either goes radio silent or reply with the usual AI tone of marketin salesman.
It's just matter of ethics for me. You should be honest to the reviewer that you didn't write it entirely by yourself(100% your authorship) but a chunk of it was done by AI agents.
I also agree with that definition. But for example if I get an LLM to generate 95% of a program, but I understand and have read every line of code, is it still slop?
As an open source maintainer, when someone opens a PR, I don't know who they are. But I don't really care. I care about the PR: is it a "good" PR (there are multiple dimensions: is it understandable, does it add maintainable code, was it tested, etc) or not?
I think "AI-friendly" says "I don't care if it's written with an AI or not: I want good quality PRs".
I would think that "slop-friendly" is the kind of projects where the maintainers don't read the code. Maybe they try the feature (maybe not even?). I think the Omarchy guy was explaining somewhere that he does not read the code anymore, he just lets his AI write Rust. This is slop-friendly IMO.
If you are not AI-friendly, it means that you refuse to accept a good PR even if you cannot tell that an AI was involved. The mere fact that an AI was involved disqualifies the PR.
Are you one-shotting then reading/understanding? Then probably slop.
Are you iterating to help you understand, prompting to rename/restructure/document until you would have been able to easily understand without AI? Then probably not slop.
It's amusing that people still default to "human-written code is solid and AI code sucks" after the enormous quantity of bugs that AIs are finding in human-written code.
Honestly aesthetically, good human code is always >> Opus 5.5.
For example, recently Opus 5.5, in some multithreaded app I was working on at work, used complex timing analysis with hardcoded time values 1.2, 0.95 etc to ensure that some activities only happen in a sequence (A->B->C). I had to tell it to just serialize these activities by putting them in a buffer. Both versions passed all tests (I’m sure it wasted a lot of time tuning the first set of numbers to pass the tests, but no human would do that nonsense). Increasingly, you feel it’s thinking diverging from humans more and more as the labs lean into RL.
Nobody has ever said human code is perfect. But it is empirically the case that code written by LLMs is on average lower quality than code written by humans.
That's not really true, and it's only even potentially true if we talk about the absolute cream of the crop developers. Most human written code is quite bad actually
And by the looks of it, they're very intent on gatekeeping. Deep down I don't like AI either, it has taken away a lot of what I love about coding. But it's here to stay. What if we can use it to wrestle hardware control away from Apple and other authoritarian vendors? Hell yes, we should.
The likely problem with the Asahi team is that they (by not accepting AI) feel that control is being wrestled away from them, and instead of adapting they choose to swallow the poison pill of GPL.
We might not like it, coding as it was pre-LLM is done for. No one is going to come groveling at the feet of those who stubbornly insist on not using it.
not sure that concept makes any sense at all. the entirety of LLMs' capability to perform software design and development tasks, such as it is, comes from the massive resource of copyleft projects, including those using GPL, that the AI companies have ingested
Well, it's a more restrictive license than MIT. A lot of popular projects nowadays are under the MIT license. AI labs still don't give a damn about how the code their LLMs ingest is licensed anyway, and GPL'd code won't stop them one bit. If their code had been used in proprietary ways, then yes, GPL would've been the way to go. AFAIK Omarchy is MIT-licensed, which still requires preserving the copyright notice of other MIT-licensed code it's based on. So I'm not sure what damage has been done to Asahi itself.
Linux itself has always been GPL licensed. It's the license switch that I have a problem with, not GPL itself. Switching from a permissive license to one less so, just so others would stop using it in their MIT-licensed projects.
There is no "stubbornly", the project is theirs and they do the f they want, their reasoning is sound. Permissive license does not mean that you cannot speak out about what people are doing with your software.
Do you think there would have been something working on mac if omarchy didn't start from the solid foundation of Asahi? Please don't make me laugh too hard.
What is vibe-copying? The code was permissively licensed specifically so other people could do what they want with it. I'm so confused what people think permissive licensing is
> I'm surprised their work was ever MIT-licensed instead of GPL.
The intent was for their work to be usable on non-GPL distributions and OSs (including the BSDs).
The new licence is pretty restrictive — proprietary platforms will just regurgitate it with LLMs, but other open ecosystems will have to re-invent things from scratch.
I saw that's there is an Omarchy for Mac "team" consisting mostly of one guy who is active on Twitter (and quickly amassed many followers) and seems to have no prior experience of working on Linux kernel. They do not mention Asahi anywhere which isn't surprising to me given the reputation of DHH and Omarchy. This hijacking of open source by bad faith actors is very concerning.
> Omarchy on...
> Apple M1/M2 chips#
> Asahi Alarm is a version of Arch for Apple M1/M2 computers built on top of Asahi Linux. You can get Omarchy running on top of that with some effort. See the user-driven guide.
Omarchy has gotten somewhat of a reputation for getting a lot of sponsorship money from commercial actors. If by monetize you mean "Get people to pay us" omarchy are definitely monetizing. Not much different to Asahi themselves of course, but is it right/moral to fork another active projects code and use it in order to gain your own patrons? I don't hold the answers. But no they aren't selling anything.
edit: worth noting this example, where DigitalOcean started cutting OSS support only to pledge millions of dollars to Omarchy. No wonder people consider the project a leech to the OSS community, but also consider correlation != causation: https://itsfoss.com/news/digitalocean-open-source-credits-en...
It is worth noting the same article's discussion of nodejs reads as "DO shut down a program, possibly planning a v2 of said program, but failed to communicate it to its dependents and didn't realize until they started planning to leave". Or they were just fighting the fires caused by a github issue named "Transition off of Digital Ocean" caused \o/
Project I was involved in was one of dependants and there was nothing in email mentioning any "v2":
Hello,
We’re reaching out to share an important update regarding the DigitalOcean Open Source Credits Program.
DigitalOcean has decided to sunset the Open Source Credits Program. As part of this change, we are no longer accepting new credit applications or approving credit renewals, extensions, or additional credit requests.
If you currently have Open Source Credits that have already been granted, those credits will continue to be honored according to their existing terms and conditions. No action is required for your existing credit allocation.
Additionally, the dedicated inbox for the Open Source Credits Program will stop being monitored as part of this transition. For any further questions or assistance, please submit your inquiries through our standard support channel: https://www.digitalocean.com/company/contact/support
We appreciate your participation in the DigitalOcean Open Source Credits Program and the work you do to support the open source community. Thank you for being part of the program.
Best wishes,
Redacted
Omarchy is itself MIT licensed, with no restrictions that I'm aware of. If they later decide to make a paid version or something, the license wouldn't stop them, I think. But GPL would. All in all, I think it's a dumb move from Asahi because their GPL'd code will soon be eaten up LLMs that don't give a damn about licensing anyway.
There's people running close to frontier model grade local AI on consumer and prosumer hardware, let's not act as if it was impossible to skip having to rent from OpenAI or Anthropic.
I've seen businesses that made the infrastructure to share a 100k machine. They needed it "on premise", but the setup did not look too hard and the results were outstanding compared to Claude Opus.
I really don't get why people even start with permissive licenses if they have people they don't like to use their software or if they're just going to go "not like that" at the first real use.
It seems like many times it's open source but "just to look".
> In some extreme cases, in-development feature code was being lifted straight out of our personal forks and published without attribution or lazily rewritten with LLM assistance to obfuscate its origin.
Permissive licences require attribution. It's okay to complain when people publish your permissively-licenced work without attribution, and hence without honouring the licence.
changing the licence is much cheaper than pursuing legal action, and sends an immediate and very public signal of disapproval. given that legal remedies would be required either way, and are extremely time-consuming and expensive, it makes sense to send the signal first
Because permissive licensing, and free software in general, rightfully comes with a certain cachet, and it turns out some people want the benefits of that reputation, but don’t actually believe in the freedom “to use, copy, modify, and distribute this software for any purpose.” Hence the treadmill from permissive licensing to copyleft, and then later from copyleft to “source available”…
“But software which OpenBSD uses and redistributes must be free to all (be they people or companies), for any purpose they wish to use it, including modification, use, peeing on, or even integration into baby mulching machines or atomic bombs to be dropped on Australia.”
No one is being forced to use or contribute back to the project though. In fact, you can even use GPL licensed software, and so long as everyone with the binaries doesn't ever request the source, you do not have to publish it at all.
> Some time ago, we were made aware of multiple attempts to exploit our permissive licensing to hijack our work, with a direct view to replacing the project and aggressively monetising Linux on Apple Silicon. In some extreme cases, in-development feature code was being lifted straight out of our personal forks and published without attribution or lazily rewritten with LLM assistance to obfuscate its origin. The MIT licence explicitly allows this, and has no carveouts to protect against bad faith actors.
It's one reason I've favored GPL of late, after many years using MIT. Not that many people these days would explicitly follow the terms of any license.
There are licenses that require attribution without being copyleft though. I'm not saying the Asahi developers are wrong to use copyleft, but it's weird to me to mention "without attribution" as a sticking point when there are fairly standard permissive licenses that require attribution like Apache and BSD.
MIT isn't copyleft and it does require attribution, it's basically the only requirement of it. I guess they just don't like that attribution means "put the license snippet deep in a secondary readme file hidden somewhere, along with 50 other licensees used in the project".
I think it's clear that they don't want people to use their code and close the door behind them. They also clearly want clearer attribution. I'm sorry but if those are things you desire, then deliberately choosing MIT because of a GPL hate-boner was plain stupid.
They are mad that vibecoders are building on top of their work, making real progress, then re-releasing it as "look it works" while containing a bunch of their work. Which is sort of fair but I don't get how that works in practice because in reality anyone can read the code which is open source and do whatever they want. Licensing won't stop them.
Your credit is your first commit. After that people copy/paste patches back and forth as they will. It happened before AI. I've run patches in prod that were basically someone's random RAID controller patch posted to the Gentoo forum. If it works, it works.
Right, this seems deeply confused. If you care about how your code is used, don’t use MIT. Especially, if you care about monetization, probably use GPL or AGPL.
Of course you can, its like putting candy for Halloween on the expectation that one kid is not going to get the whole bowl at once.
The way you can exploit an open license is to grab the code, invest massive amounts of resources in it and close the platform. Embrace, extend, and extinguish.
This is an extremely “all or nothing” take. I love computers and started programming in elementary school. I am having more fun now building software than I can remember.
I happen to love the process of building software by coordinating agents. It’s an addictive form of flow state that I enjoy a great deal - similar to the flow state I get playing real time strategy games like Age of Empires. It’s a different kind of flow state than writing code, for sure.
But to say that the only people who enjoy building software in this new way are those who hated coding, technology, or problem solving is simply untrue.
I’m glad you mentioned flow state because that’s exactly the high I’m trying to chase with AI and why I like it despite programming since elementary school as well.
I realized I most enjoy higher level problem solving but don’t really enjoy implementation. Like I enjoy algorithms, data structures, memory allocation strategies, SIMD optimization, etc. But writing the code, and debugging it is tedious and takes me out of flow.
And if I’m really interested in the project itself then the algorithms take a back seat and I enjoy the even higher level problem solving of building a cool project.
I don’t think it makes you lesser to enjoy these things more than literally banging out code, and hunting down bugs. Perhaps I’ve written enough code that I don’t find the same joy in that. Who knows.
Oh my god, and the author even supplied a "proof"[0] visual diff harness... that it replicates the original game pixel for pixel.
Just the cherry on top of great demonstration of our collective new superpower: asking computers to do something we can describe how to do, but would (probably) never take the time to do ourselves.
tbh I thought this was a commonly used technique even prior to LLMs? I know I've been using it extensively myself, but I was inspired by Dolphin's extensive visual CI system.
If it is common in the world of video game porting, that just shows my ignorance. I'm familiar with visual diffs in CI for e.g. web development (comparing a static component), but to do that to compare frames over time in a video game/3D environment is new to me.
There are so many more degrees of freedom, which I can see Claude handled... mipmaps, subtle differences in lighting/positioning/compositing etc.
Even then, visual diffs were pretty flakey for web development, because one's OS and browser choice would slightly alter the exact pixels blitted to the screen. At least this was the case for the tests that would simply match pixels instead of computing a sort of visual hash.
It's also partly why some people preferred snapshot tests that compared the DOM tree instead, though that was brittle in other ways (e.g. tests would break if an application's frontend used a major UI library and an update to the library permuted the order of classes in some part of the HTML).
For web UI tests this is mainly solved, at least when using Playwright. It allows setting thresholds, percentages and some other config items to allow some small differences in pixels. https://playwright.dev/docs/test-snapshots#options
That's an attempt at mitigation, but most definitely not solved.
It still causes both false positives and false negatives through that. The only true "solution" is to make sure generation always happens on the same platform as your ci... And various mitigation strategies around that (eg fall back to structure tests on other platforms vs actual visual diffs in ci
Also not unique to playwright. Been available basically everywhere since the start
I'm getting a 1997 PC game to run on modern hardware and fixing bugs and upgrading graphics as I go, and the amount of quality support tooling Claude is producing along the way is impressive. Fully headless in-memory execution (which, among other things, is used by it for per-pixel diffs too), logic VM devompiler and visualizer, asset explorer, CRT simulator... I just say what I'd like to see, and Claude does 120% job on it each time.
I do that on my renderer. each commit when it's ready to be merged gets a class of visual diffs. Dolphin's way was a major inspiration how to structure it, but it was _the way_ in rendering way before it.
The visual diff harness was probably how they got rid of a lot of visual bugs, just tell the LLM to keep going until the pixels match exactly as the verification criteria
There are several good algos used for diffing graphics for not so pixel-for-pixel rendering allowing thresholds (you are going to end up with differences between software and GPU rendering at least - it's inevitable):
I've recently gotten an Oracle in my translation system too, I think it's a common need to compare bytes betwitx to different systems. I didn't think it was specific to one company.
How long before the same thing is done to like, banking back ends? Wallstreet proprietary software? Amazons logistics and distribution systems?
It seems like we might be weeks/days/hours before a situation where someone back engineers and spoofs a system so pivotal to modern human society that the plug needs to be pulled.
Like I've said many times: LLMs are useful for this (i.e. porting software from one programming language to another).
Porting software is painstaking grunt work which still takes a moderate amount of intelligence. It's therefore extremely expensive to port say, COBOL banking software running on mainframes, to another language like Java. That's why a lot of COBOL software is still in use. I expect this to die out in the coming years as many of these systems will finally be ported to another language (could be Rust or any other language).
In many ways the financial system is a distributed monolith. COBOL is still around because of bugs and issues (sometimes in code, sometimes in the OS it ran on, sometimes in the computer hardware) that have been around for decades are often dealt with downstream with their own exceptions. This is why IBM still sells mainframes. They literally emulate software and hardware bugs from equipment as far back as the 1960s.
I spoke with a semi-retired COBOL programer about a decade ago. They literally often can't fix specific bugs because multiple other consumers (from banks, hedge funds, insurance companies, the central bank) outside of their organization (and subsequent downstream consumers as well) would all need to adjust their code. Rewriting it is grand, but won't fix the spaghetti code mess. This is made even worse in the US, which has a much more fragmented banking system. "Modern" COBOL isn't even that bad, but they can't even use it most of the time. There is SO MUCH that isn't even documented - and if it is, is in a binder that's been shelved decades ago and half of this developers time was going into the corporate archives to dig it up.
This isn't to say that a lot of legacy code can't be modernized, but it's not always that easy.
If someone recreates Amazon's logistics and distribution systems they could try to compete with Amazon? But they'd also need the connections, distributors, transportation, etc. same with banking software, you need capital to be a bank not just software, and if they have the capital then the technology is working we intended making it easier to make new things and innovate, or at least just compete?
No, I am not talking about "taking over" companies and trying to emulate them and do business yourself. You just need to be able to break trust in the api calls and no one knows if a purchase order or transaction is legitimate.
Obviously you need to have access to the keys, BUT I don't see this as a dealbreaker anymore because you just get your agents to go and find them.
I think you’re saying ‘being able to do this means the opportunity for more fraud, by producing fake XYZ as proof’.
Photoshop has been around for around 35 years, fraud has always been an issue. There are plenty of reports of people selling things via Facebook marketplace and the ‘buyer’ showing them sending a payment on a fake baking app. Fraud will always exist and I don’t think tech will make it worse, everyone needs to be more cautious and tells friends and family to be the same.
I'm talking about cloning hmi platforms to send fake instructions to offshore oil platform valve bodies or insert false market trades to collapse companies.
Ah. So in that instance are those platforms not validating that the things submitting information are correct and true.
A bit like a utility company needing to do manual reads every now and then to ensure they are getting a correct signal.
Or they can just take your money and not send out anything.
Alternatively, you just act as a middleman drop shipper and slightly raise the price more than Amazon’s and skim the difference. It might be a while before they find out.
That’s an insecure design. The way we do it here is that you install an app and register your register number and payment card in it. Then when you drive in and out from the parking lot your license plate is scanned and you’re automatically charged. There’s only two providers so it’s not a huge hassle, if there was a single app per garage it would not really work from UX perspective.
> spoofs a system so pivotal to modern human society that the plug needs to be pulled.
why would a recreated system be detrimental?
If currently there's a monopoly on a software, this AI recreation is a good outcome to poke holes in that monopoly. It's only bad if you are financially invested in said monopoly, and this would be a minority compared to the amount of benefits that society at large could obtain.
This is basically a digital era anarchist view - the problem you're overlooking is that a lot of critical infrastructure we rely on runs on systems that are considered security through obscurity. Software most people probably wouldn't even know or care that it exists. If you can break the trust of vendors by being able to spoof their proprietary platforms, a lot of the highly efficient networked systems becomes vulnerable to injection and abuse if you can't trust whos making calls to it.
In the past you'd need nation state actors with considerable budgets to do this kind of thing, and we're on a trajectory that could see any kid in his bedroom could do it.
revealing that security thru obscurity is broken can only lead to a better future, even if in the intermediate one there are lots of breakages. It's suffering that needs to happen, and better sooner rather than later imho.
And i assume you don't truly mean spoof as in man-in-the-middling someone - i assume you mean the end user knows they are using an alternate system and are not being defrauded. Like using a photoshop replacement.
> And i assume you don't truly mean spoof as in man-in-the-middling someone - i assume you mean the end user knows they are using an alternate system and are not being defrauded.
I am, actually, talking about MITM attacks that are much further in scope than just defrauding some people using their banking app. I work in resources and operate HMI systems that are networked, but not exactly the bleeding edge of modern software development. If you had an ability to decompile it and recompile your own version you could start sending instructions to infrastructure all over the country - the only thing stopping you is the keys, which if you're intent on hacking someone you'd have the means to obtain anyway.
I can see a lot broader attack vectors than just stealing peoples money. It's the erosion of trust in the api calls themselves.
I’m already seeing videos of people who have used LMs to reverse engineer and clean room reimplement entire video games. I estimate this shit is minutes away from being shut down, because as we’ve all seen companies stealing is OK, but individuals stealing is heinous and a crime.
I mean some people (me included) have been begging society to pull that plug since over 10 years now.
The plug being "the cloud" and "hooking everything up to the same internet".
These confusion attacks can only confuse people, because critical systems can exist in the same space where entertainment systems and all other categories of systems live.
This was wrong even before LLMs.
Adding to what you both said. I run pixel checks against live websites in a real browser, and I stopped doing full-page screenshots early on: scrollbars, lazy-loaded images and font timing shift pixels between runs like clockwork.
What works for me is diffing small stable regions, one component or one flow at a time, with a small per-pixel tolerance. I also keep a DOM-level assertion in front of the visual check, so when something fails I already know what changed structurally before I compare two images.
For the game port it's genuinely harder, the renderer decides the pixels, not the test. But the region idea transfers: lock the viewport and only diff what's stable.
Actually Claude just did something similar for me, as I'm working on something else with Quake.
On its own it decided to do demo playbacks and take periodic snapshots, and compare them pixel by pixel if the PNGs differ.
I was also working on a web-based port but it saw the original Quake code was not modified so it compiled a native version on its own to use for this.
It had to apply a small patch to make the game completely deterministic, but it figured that out on its own by reading the code.
It then asked me to record a demo with various elements, say an explosion or being under water, and visually verified that the screenshots had those elements present.
So now I have a solid set of tests to verify against.
Opus 5.5 Medium. Used at most 1% of the weekly limit of my $20 plan.
How does this compare to some of the efforts led by David Kosbie & others around open access online CS education? I recall this being a big push back when I was there (2014-2018).
Is this essentially similar spirit, but covering actual SCS curriculum, rather than "pre-SCS"/highschool content?
reply