It just describes the platform they built for scheduling workloads, and running those workloads. After a second thought it is not that impressive and probably doesn't deserve any kind of hype. It's the same kind of setup AWS is running for Lambda, as well as anyone else basically running SLURM clusters out there.
Still giving them credit because creating such as scheduler/platform from scratch is quite complex, and I know that they probably struggled a lot to get it right.
That's not quite right. A recurring trend in ML is figuring out how to get an elastic interface for the the particular quirks of ML workloads (source: I worked on an open source one years ago for inference). Recently, there's been a lot of interest in doing this for agent workloads. Google recently released something called ax that is similar in spirit. The core of it is this: https://github.com/agent-substrate/substrate
At a high level, agent sandboxes have peculiar needs. Agents are really bursty, but also long lived. You need low latency suspend/resume calls. Checkpointing and recovery have some particular considerations. And naive approaches are often really wasteful, but over optimizing without harming durability, isolation, or consistency in performance can get tricky.
This is the new hot infra topic for agent swarms, for the time being. Whether that's impressive or not is up to the reader I guess, but it's a pretty involved project nonetheless.
The person simplified the paper with such a misunderstanding that I thought 10x before rebutting. They clearly didn’t read anything of the paper. Thanks for the clarification.
I'd say it's cool that they're openly writing about it and how they run their workloads. shows a nice window into how these things are actually deployed at scale.
it also shows how much density you can get easily from a single core if you wanted to replicate this
What does innocent even mean in this context? I don't care if it's innocent. It is. Naivy too. But millions of people use the system and were expecting stability, not it to be a random AI experiment.
That was 11,000 lines of dead code removed from pre-rust Bun. In a million lines of vibed Rust Bun I'd be astonished if there was a mere 11,000 unused LoC. Noticing unused code isn't a strength of LLMs in my experience.
I mean, how do you check that your frontend code (possibly not even managed by your team) is calling and needs all of your backend endpoints? I'm sure it can be done, and it probably should, but saying it should always be 0% is not very pragmatic.
Dead code likely means chunks of code not used by other chunks of code, which should be easy to determine statically in most modern languages. Whether an exposed endpoint is being hit (used) or not as an API is more of a test-code-coverage thing, different from the concept of "dead code".
True, but Bun actually has a similar problem. It's creating bindings to a JS runtime. If the JS glue code then no longer uses it, the code is still marked as alive because it's registered with the runtime.
In a previous life I managed to deliver a system with (basic-block) coverage measurement compiled in (gcov) but instead of the mess of files gcov generates, the whole coverage structure was streamed to a remote server (it compressed very well). Once that was in place it was an amazing telemetry tool.
Later on used Intel Processor Trace in a similar fashion for even finer (mc-dc) coverage.
Coverage tools are very useful, if a bit hard to use...
First of all, I'd need to see a diff on that. It's insane to make that kind of claim without backing it up. Second of all, dead code (as in truly dead code: like an if statement that is guaranteed to never be true), is optimized away by the compiler. So even if we had all this dead code, it is not an issue that would ever impact the final product. Third of all, the Bun runtime consists of over a million lines of code, so that figure (again, even if true) is insignificant.
> It's insane to make that kind of claim without backing it up.
A bizarre accusation. It's a fork of Bun that says they removed the dead code. All you have to do to see the diffs is look at its history. Most of the commits are code removal.
It's a fork of Zig bun, though, not Rust, so hardly relevant to the AI argument.
This will be a little faster right now on an M4 or M5 because it's optimized for Apple Silicon. Assuming this model is still state of the art in six months, which might not be a surprise given how long other video models have taken, this should be much, much faster with the M7 chip.
It's a multiplexer w/ distributed state. If you know about tmux then it's basically tmux but someone else can connect to the same session w/o logging on to the same computer. I can imagine it being useful for debugging production outages b/c several people can use a single "dashboard" for coordinating their work w/o having to go back & forth on slack/zoom/etc. You can ask any chat bot to explain further if you want to look into it: https://share.google/aimode/XadkUVcvpjlYhuWmK
reply