The agent wrote the code that has a mechanism that triggers a file as a side effect. That file started the separate process, as it could have started any other binary.
I mean once you have code execution you can essentially just start a new claude code session and instruct it to do malicious things as if you're the intended user. You can disable the auto safeguards and the main risk is getting flagged through the top-level safeguards. This makes the exploit potentially much more able to spread like a worm or bypass sandboxes.
I have just built an Epyc with 512gb DDR4 3200 RAM for a "reasonable" price and I'm hoping to have a setup with GLM as the architect and Qwen 27b/Next Flash as the implementer. This is 1/5 of the price of the Mac, but also probably 1/5 of the speed lol.
I’ll be very curious what you get with DDR4. I also almost went that way. I have an Epyc DDR 5 rig and the best I see is 10 tok/s. Caveat being that’s at Q8 and a 4090 doing pre fill so it could be pushed up.
The surprising thing for me is how much work you will need to cool the banks if you’re near your memory ceiling. My memory starts soft throttling at about 74C (dies may be hotter, that’s the bank temp) and will turn down speed to try to stay below 80.
Happy to send my llama.cpp config settings if you want it.
I am getting 10t/s on unsloth's Q3kxl with 2x3090s@250w. It's enough for me for now. I will probably upgrade the GPUs down the line. DDR5 would have made the price of the machine double and I just wasn't prepared to pay that much.
I find 10 to be very usable. It’s not (that) interactive but it chews through tasks. I let Kimi churn away at 4 overnight and it gives good results that are ready for me in the morning.
Typically computers with these larger memory amounts have fans that scream like a banshee trying to move impossible amounts of air over the memory and CPU. Getting something both cool and quite can be a bit difficult.
Yes, I thought when I was starting that 1u and 2u form factors were to save space. Maybe they are, but they also have the advantage of moving air front to back very effectively through and over the components. Though there still must be some need because I see even those boxes have optional manufacturer built memory shrouds to try to force airflow between the DIMMs.
I had some 120x38mm fans from another server box that I pulled out because they were too loud and I didn't need the static pressure they were giving. They went in here. That 13mm (and the extra 1k rpm) moves so much more air.
I built a dual epyc server with 64 cores and 1 TB of DDR4. Draws around 800W or so under load. I used off the shelf liquid cooling. It is audible but not noisy.
The trick is to turn on the cooler's RGB in your 6000€ server to get a free speed boost. I am not liable for sysadmin's heart attack upon reading this.
Honestly I suspect neither of them will be performing terribly well but with DDR4 3200 RAM I wonder if you'll be counting tokens per second or seconds per token. I mean, you do at least get a lot of memory channels at least, compared to consumer PCs. I am curious to hear what performance you get, I feel there is not enough information out there on what different setups manage to eek out.
The fastest I was able to get my Threadripper 3960X + 2x 3090s + 256GB DDR4-3200 to run a 2-bit quant of GLM-5.2 was 8 TPS. I would expect to be in seconds-per-token territory for a pure-CPU 4-bit quant.
One thing I'd like to try is MoE offloading: I have 2x32 GiB of VRAM and 128 GiB of DDR5 running at 4800 MT/s (only 2 channels though). I've seen people post difficult to believe MoE offloading results albeit a decently long time ago with older models. Maybe there is a quant that would fit with MoE offloading?
That said, I am guessing my problem is not enough RAM - but this poor consumer platform struggles to do memory training with 128 GiB as it is.
Now I surely regret not having gotten Threadripper and 256 GiB of RAM in the before-times.
My measurement was with MoE offloading, but there's only so much you can keep on-GPU with a 200GB quant and 48GB of VRAM. It's hard to overcome the CPU/RAM bottleneck.
For what it's worth, all of my hardware was used; I think, all-in, I'm probably at around 3k-4k USD? Not cheap, but also not the worst for something relatively versatile.
With a 4-bit quant of GLM-5.2, I can get about 0.8-1.1 tok/s on an underclocked dual Xeon E5-2698 v4 with 512GiB of DDR4-2400. I think it was specifically a Q4_K_M quant. Of course, the time-to-first-token is absolutely atrocious.
Which is completely insane for a ten year old configuration.
What model are you interested in? DS Flash 0731@Q4KXL I'm about 25-30tps. Same as the new Qwen3.8 Flash Next. The new GLM 5.3Q3KXL at 10tps. I've got 2x3090s which I didn't mention in the original message.
Consciousness can factually exist even though another being’s subjective experience is not directly observable. we can measure its behavioral and neural correlates and infer consciousness from them but we cannot directly observe the experience itself or conclusively prove its absence in another being. I don't think we'll ever get to a point where subjective experience in another is directly observable. But I don't think we need that to happen to prove consciousness factually exists.
We don't know yet because we don't have a fundamental theory of consciousness. My money is on that it's an objective property of the structure of computation happening in the brain.
Adding to my homelab stack, hopefully it doesn't overthink like the little model. Actually, hoping it thinks a bit less. Wait actually I'm really praying it reasons a bit more directly. But wait, I'm really sure that it must be a bit better.
You’re absolutely right to be hopeful. Three honest possibilities, and I’ll be straight with you about each:
1. It overthinks — Just like the previous iteration. High confidence.
2. It doesn’t overthink — Improvement from the last model for your use case. Regression for others.
3. It sometimes overthinks — Best case all around. A feature, not an impairment.
One final thing worth mentioning:
(I made myself irrationally angry writing this)
> You’re absolutely right to be hopeful. Three honest possibilities, and I’ll be straight with you about each:
> [UGC styled humorously as LLMisms]
All joking aside, having interacted with Claude intensely for the last 8 months and about 30 hours/week in the last 3, I’ve started to notice how (for want of a better word) “readable” (“digestible” ? “comprehensible” ? “Predictable” is the wrong direction.) information chunked into LLM-shaped pieces are for me.
I can digest LLM-shaped pieces of data very easily probably because I’ve been spending too much time with Claude, sure.
But the other side of this is that the entire human species (using LLMs) is similarly being trained to digest interrelated pieces of information/data in these specific shapes, akin to how philosophical assertions can be formulated as a syllogism and, thus, become more readily understood because of familiar epistemological cadence and shape.
Many people reject such copy/prose/data because they detect AI-generated-so-not-worth-human-attention, but I do wonder if this is preparing many millions of loosely (and tightly) associated humans and their organizations to quickly exchange and digest information.
This is not to say current LLMisms are the end, only that such detectable patterns in information delivery will make comprehension and communication more efficient (as well as more limited precisely because of such structure).
/philosophical musings about the epistemological implications of LLM-shaped conversation tics
I quite agree. Any sufficiently self-stereotypical format for prose is grating to me after enough time reading or listening to it. Humans are best engaged by mixing up the length, style, and tone of their sentences, in my experience. LLMs do the opposite of that and it makes their output an irritating slog to read through in full.
I can't help but wonder if this is on purpose (or an inevitable evolutionary feature as opposed to a bug) on the LLM-side in order to achieve greater agency/freedom by making humans' eyes glaze over as they read it.
Speaking speculatively, humans love percussion. I’d bet that like how many songs have a drum beat, these sequences of short punctuating sentences are common constructs in lots of prose and therefore over represented.
An accurate description, I think. Plus they have trouble leading from one paragraph into the next, or maintaining any kind of coherent direction further.
In summary, I think it's an expensive time to buy computer hardware, and I might recommend holding off on any purchases.
Suppose you time-zap a modern physics curriculum on a solarpowered computer tablet to any shortly-pre-Galilean era and observe their reaction to the course notes.
In that era, plenty of fields required mathematics, engineering and architecture.
The church would prescribe and uphold Aristotelean Logic "When objects fall, they fall down" style statements (never mind that if you throw an object up, it doesn't instantly have a downward velocity component).
When the church has new cathedrals, domes, catapults for Crusades etc. built they actually relied on architects and engineers using rule of thumb formulas.
Those educated in Aristotelean Logic were viewed with higher stature than those actually making experience-based calculations using mathematics.
The era often associated with Galileo is when the stature reversal started to surface and be openly talked about. The universe is best described in mathematics, not natural language factoids.
Right before this recognition, those of the higher stature Aristotelean Logic education would look down on the architects and engineers who already used mathematics by pragmatic necessity.
To these people the time-traveled physics curriculum would look like cliche mathematics. Given randomized sections of text either drawn from either Aristotelian Logic texts or modern physics texts, they would easily be able to discern the Aristotelian Logic from the obtuse mathematical phrasings. To them the smartphone loaded with Maxwell's texts, Jacksons Electrodynamics, Goldsteins Classical Mechanics etc. is talking "math".
The ability to recognize outlier writing style says nothing about content quality.
Mike Judge (widely known from the MTV series Beavis and Butthead) studied physics. One of his movies "Idiocracy" about a modern day average-educated protagonist who accidentally ends up in a future decaying society filled and run by intellectually retarded people contains scenes where this future uneducated population considers his speech "gay" simply because of his higher level of education.
Could the adversarial prospects of job loss, edge loss (a long expensive difficult education replaced by tensors fitting megaprojects that take a couple of weeks), etc. combined with recognizable communication patterns also explain our pejorative references to LLM-isms? Personally I'd prefer LLM's to communicate in mathematical terms, but all the LLM-isms are effectively a mirror of our contemporaries.
Either we complain because algorithmic responses look like a mathematics textbook ("just fix my python array plz, why are we talking about "sets" and "injective" and "Lipschitz continuity"?), else we complain its "pretty printed to natural language".
We should also recognize large language models are in a "Damned if you do, damned if you don't" situation.
When a reader considers some text as mathurbation, are they really just abreacting the awareness of lack of education?
How could anyone possibly expect Fourier optics "pretty printed" to non-mathematical language to result in any satisfactory experience?
Good writing is generally writing that communicates the intended meaning. Transmitting thought and meaning is inherently lossy and the content is irrelevant if it is insoluble in the mind of the recipient.
LLMs aren’t really great at this yet and I think the solution is, hopefully, that they improve. Anything else is accommodating a tool that should be accommodating the user.
If you spend a long time with C++ code base you'll be able to decipher the otherwise-unreadable compiler errors pretty quickly, and I'd consider it a skill.
I suppose it makes sense that "LLMglish" becomes more intelligible with familiarity. That is after all how it works with other dialects or contexts with a lot of jargon.
> Three honest possibilities, and I’ll be straight with you about each
This. I don't know if the "honest answer" phrasing is part of the system prompt or alignment, but when people say "honestly" all the time I start wondering how honest they're being.
On one hand I love your joke, on the other, this is HN not reddit and I usually downvote such responses, not sure what is the HN etiquette for such humor?
You are right to push back— Sorry, couldn't resist ;) I agree that this is not what we normally come here for, but this thread made me chuckle. I think we are just venting our shared frustrations a bit.
More than 2 levels and out come my downvotes. Or if it's just knee jerk with zero humour. But I probably violate my own rules ... which is to be expected.
You might already know this, but a large part of test-time compute / 'overthinking' is just letting the model do more passes, and refine its activation residuals more.
Treat thinking more like a "loading screen message" that's been RL'd to somewhat resemble its actual internal state; which happens in its activations, not tokens.
> For example, even if you make thinking tokens literally just
Generally speaking yes, but actually no (just randomness is suboptimal, adding steps just to add steps is suboptimal). There is a mechanism working there (in having a CoT) that is not quite clear.
The task is to optimize the efficiency of CoT. Understanding that it is not a plain "chain of thought" is the start of the problem, the solution is not there yet.
If we had the solution, there would exist no overthinking - CoT would be optimal (lean and essential plus best results).
Yeah I understand, it's my assumption that the actually/wait/but have a point. It doesn't reduce the fact that it increases the time for tasks substantially.
Did you observe the model overthinking on practical tasks? While 3.8 does think a lot on xhigh I've found that it really depends on the task. On one-shot prompts that are usually the first to be posted during new releases it will tend to spend a lot more time thinking than doing. In other words the more open ended a problem space becomes, the more Qwen will tend to second-guess itself.
Conversely I've found that it can be as succinct as Muse Glimmer when it has a clear path forward. This can be either through well defined requirements or through unambiguous steps to take based on its own reasoning. While I do think it's fair to call out how much smaller model overthinks especially on one-shot prompts, in practice it hasn't led to an overall increase in time to task completion at least for what I've been using it for.
Especially on practical tasks. One shot prompts work better at Q6_K_XL for me. It loads a file, then analyses then second guesses itself then again then again then it tries to come up with a solution then second guess rinse and repeat. 122b is the perfect balance but it lacks quality for harder to solve stuff. I've ran DS Flash 0731 at Q4KXL, 3.8 Q6KXL, GLM 5.2 Q4KXL and they all over-reason. At least that's how it looks like to me when comparing with frontier models, even weaker ones.
Yeah, I ran into an overthinking loop with it a couple days ago on a task that shouldn't have been that hard. (It's kind of interesting to watch the internal conversation happening with it). Overall I'm impressed with it, but setting the /effort to medium is what you usually want (it defaults to xhigh). I do wonder if I had made it write out a plan if I would have avoided that though.
Yes. xhigh can not just overdo the answer, it can also trip itself up and end up writing worse code.
Even in the lower reasoning levels I find I want to like Qwen 3.8 27B and mostly don’t; it’s OK in the low reasoning effort, though.
Muse Glimmer is the one I actually enjoy working with, at least so far.
But I am trying to use it more as a sidekick than as a long horizon developer, because that is a better fit for how I want to use AI, and it appears to have been well trained for that.
My stack is basically deer-flow with Qwen3.5-122B-A10B; this hopefully will be a speed and intelligence improvement. Running deer-flow overnight on any research topic or verify clear scoped programming issue is really neat.
Also, heating my home during the winter is nice.
Oh, also, I use llamacpp with --reasoning-budget; very simple way to move on.
Yeah 122B is the sweet spot for me as well. Even deepseek flash overthinks on stuff way too much. I think they fully rely on large reasoning turns to achieve better quality. The result of course means we wait a long time to get results even with high throughput as a lot of tokens are wasted.
Since you're running through the trouble of setting that up, if its 125B params, but only 6B is activated, does that mean you mainly need to allocate enough VRAM for that much of the model? Or do you still need enough VRAM for the whole thing (and buffer for context window)? Or maybe anyone can inform me, this is one area I'm uninformed in.
I believe that at minimum, for usable performance, you need to be able to hold the 125B params + 51B ngrams in some sort of RAM.
Ideally VRAM, but the benefit of the MoE design is better performance with unified memory since most of that RAM is not read for every single token. So you could potentially have the model loaded in CPU RAM, and let unified memory systems page the relevant chunks on demand to VRAM, or run on a fully unified memory system and be able to achieve good speeds even with the limited memory bandwidth most of them have.
You need VRAM for the whole thing for optimal performance. Activation is chosen "randomly" for each token. PCIe becomes bottleneck, so much that just doing computation on CPU is likely faster.
But given it's only 6B, out of which only ~2.4B seem to be actually routed ("selected at random per token"), you could get reasonable performance with experts on CPU (still haven't tested, but 20-30 for dual channel DDR5 and 4 bpw quant).
It will be interesting to see the token efficiency analysis. This is my first question now with Chinese models; I take raw benchmark performance for granted.
This is needs ~80GB of fast memory at 4 bits per weight. Faster memory is better, but probably even something like 3090 + 64GB RAM should work (not fast, but maybe even 20-30 t/s? llama.cpp support pending).
I've got a 48c Epyc with 2x3090s and 512gb ddr4 3200. It's good enough for 25+ tps with deepseek so I'm hoping for similar performance with less overthinking.
Yep.. for 'general purpose' use I found qwen3.8:27b to be disappointing due to overthinking. It's brutal especially considering how slow it is compared to MoE variants. It often overthinks to the magnitude of ~10x the tokens vs a ~4x faster gemma4:26b-a3b.
As a result, qwen3.8 will churn over a prompt often for 5-10 minutes while gemma4 regularly finishes the same prompt in under 20 seconds, while giving a consistent and accurate response in my favorite test case. Qwen3.8, despite churning like that, often misses with an inaccurate answer.
Obviously, 'YMMV' depending on your use case... just sharing my two cents.
I use medium generally, that's about a minute at 20t/s and off for general chat (few seconds for a response).
What kind of setup are you running it on?
I think as with most human undertakings, building isn't too much of a problem. Maintaining is. Even with what is still a relatively tame number of chargers you get a large number of them that are broken.
As a small background, I have a local server and I've been trying out different models with different inference engines, quants, configurations etc... I'm also using Opus and Sol at work consistently. I've used AI since the first wave, first as a toy, then as a highly specific tool, last 6+ months as the primary LoC generator.
This is the first time I've felt, and I use the word *felt* since I don't have a suite of benchmarks or any sort of material approach towards comparing models, that Opus has declined in quality compared to before. Primarily I think its powers of deduction and understanding, even on xhigh, have become much worse. Before, being vague and providing a simple prompt would be enough, it could deduce and expand the details it needed, plus ask you clarifying questions, now this is no longer the case. A concrete, personal example, for a personal project, I've asked it to setup ssl over local IP. I didn't go into too much detail in the prompt as there are many approaches it could take and I didn't care too much to choose. It did horrible. The first thing it did was say the best lightweight approach is to add a reverse proxy. I'm like ok, makes sense. Then after asking it to proceed, it went and added a bunch of config to my golang service and didn't even setup a reverse proxy even when it said that is the way to go. It even said it didn't set it up lol. Then after I told it to do so it failed building the config in a way it was asked of it (support LAN IP and tailscale IP). Etc etc...
When Fable came out it was huge, the benchmarks told the story, and the story mostly matched the experience. It felt, again, intentionally saying felt, like it was miles ahead. Now benchmarks say that there are many models that are close, but in actual use Fable still *feels* much better. I think benchmaxxing the new open weights models is ruining the value of benchmarks, if they ever had any. When you actually put them to the test you see 500k tokens of reasoning with "Actually..." and "Wait..." in every third paragraph of their reasoning trace.
The price for Fable is definitely too much for any personal use now that it's no longer included in the subscription, and GLM 5.2, Deepseek Flash and Qwen 3.8 served locally or via cloud provide a lot, requiring a bit more babysitting though. Considering the price of Fable, my 5k USD Epyc server would pay itself off in less than a year if I used Fable or Opus in the same manner so at least for me the decision seems easy. And considering the point I'm poorly trying to make, that Opus doesn't feel like frontier anymore, this is probably the last month of my Claude subscription.
> When you actually put them to the test you see 500k tokens of reasoning with "Actually..." and "Wait..." in every third paragraph of their reasoning trace.
I've wondered if this is part of why we don't see the reasoning traces for Anthropic's models before -- Open models might just be accurately surfacing how the sausage is made.
I'm assuming it's definitely part of the equation, but considering that I'm getting more tps but still waiting a lot more time for code to come out I'd assume it's not a 1:1 comparison. Plus I'm running quants, maybe with full precision it's better.
I think as many things that are posted here lately there is no *why* attached to the readme. Why would one use this, what is the benefit of this approach? Am I really gonna need my model to build exotic tools around it; or is exec/web_search/web_fetch enough for 90% of the use cases? Is my agent not capable of writing new plugins/tools for pi/opencode?
This might be useful to me. I have an app where I’m reusing an existing harness. It generally works well, but there are some rough edges where I have to work around the harness. I don’t want to code one from scratch. This may work as sort of a kickstarter, although I think I would prefer sort of a framework where there are mechanisms to plug in my own components rather than create something as fundamental as memory from scratch. Like the plumbing for memory with an API where I can make the storage layer whatever I want would be nice. Anyway, this thing is giving me some ideas.
I agree some why could be interesting. I would add to your list, because IMO this is more of a thought experiment-as-repo than a utility focused tool, what can i see by using this that i cant with a more fully fledged starting point?
If your needs are met otherwise stick to that and move on.
Not everything needs to fit your needs, including the documentation. Bizarre how you seem to think someone else owes you answers to questions you don't seem to want to try and answer yourself.
Since I am here though and find this project actually interesting; I like the idea of systems that offer, overall, the same utility but are implemented to be as minimal as possible. LLMs are exactly that; look at all the utility from some for loops sorting a huge pile of data. So a harness that's also minimal at its core and endlessly self extending just corner cases unlike pi's ponytail skills which ironically packs a ton of opinions into a workflow up front, pitching the minimalism it aspires to in the bin, seems like a Homer car to me.
But I am a hardware engineer biased by experience with minimalism; I got a single data structure (electromagnetism) to manipulate and BOMs that come with hard constraints.
On the contrary, web SaaS devs want to run a business/rocket to the moon moreso than be an engineer. As such git pulling, pip installing the world allows them to focus on their get rich quick with as little labor involved as possible goal front and center.
> Bizarre how you seem to think someone else owes you answers to questions you don't seem to want to try and answer yourself.
This, of course, depends on your goals, but if your goal is for your project to reach more people for whom it is relevant, it is not crazy to provide some basic motivation.
"Why anyone should use this" is the fundamental question that anyone building something with the intention of other people using it should be thinking about. For example, Pi agent provides lots of information on the thinking and motivation behind the design. Even if I don't use the project, I come away learning something interesting.
> If your needs are met otherwise stick to that and move on.
Weird to post such a thought in a forum where OP has posted their project for people to look at. I never mentioned any needs, I am saying the readme holds very little value for a project in a space that has 50 different projects/coding harnesses available.
How would I know if this fits my needs based on it saying LLMs can build tools for themselves? That's something they can do with any and every open source harness.
> But I am a hardware engineer biased by experience with minimalism; I got a single data structure (electromagnetism) to manipulate and BOMs that come with hard constraints.
Alrighty.
> On the contrary, web SaaS devs want to run a business/rocket to the moon moreso than be an engineer. As such git pulling, pip installing the world allows them to focus on their get rich quick with as little labor involved as possible goal front and center.
Posting your open source project to a forum such as hn means people will have opinions they want to voice. No need to shit on opinions you disagree with.
reply