Hah not only same message from me but exact same year too, end of the GMT800 and the TJ. Probably other stuff I’m forgetting.

Cool. I found halogen but was turned off by the closed source nature (github repo that just turns out to run a download shell script, lame).

Do you mind sharing more details about your setup? I have been using opencode with lemonade for inference on my strix halo but haven't quite hit the sweet spot.

I'm using it in opencode through llama-swap.

Here's the gufo command:

gufo serve 1lm \ -m /mnt/LLMs/models/unsloth/Qwen3.8-Flash-Next-GGUF/UD-104_XS/Qwen3.8-Flash-Next-UD-1Q4_XS-80081-0f-00083.gguf \ --served-model-name Quen3.8-Flash-Next-UD-104_XS \ --host 127.8.8.1 --port 10083 \ -c 262144 -n 16384 -j 1 \ --prefill-chunk 8192 \ --speculative mtp \ --mtp-model /mnt/LLMs/models/unsloth/Qwen3.8-Flash-Next-GGUF/MTP/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf \ --cache-disk ~/.cache/gufo-fork \ --cache-disk-bytes 68719476736 \ --cache-disk-staging-bytes 8589934592 \ --cache-ran-bytes 4294967296 \ --log-progress

Cui bono?

Squishing the budget deficit for 1 cycle in 2 years.

How is money easily made off of adherents to these TA methods?

High frequency trading shops like Jane Street.

Their brokerage dark pools their trades and front runs them.

Frequently they get pump-and-dumped based on some bullshit they read on Reddit.

These people lose money relative to index funds, so even putting money in index funds will let you take advantage of them.

By becoming their counterparty

Are you referring to a case like this one? Don’t know if 7.5 years counts as quickly. Maybe there are faster ones I couldn’t find.

https://www.researchgate.net/publication/342618008_Variant_C...

It's kinda like eternal September but for software engineering. There was a time that just being on the internet, having an email address, etc. was a signal that indicated a level of technical proficiency. Now everyone has that connectivity, it doesn't mean anything anymore.

Same concept, but github repos now.

FastFlowLM is what you're looking for - Lemonade does package it, as sibling comment says.

https://fastflowlm.com/

I wish they supported older gen XDNA though :(

Did it used to be a lot bigger? Looks like a couple wooded lots side by side now.

Much much bigger. Visiting it now is quite sad.

If you go to England, you're better off going to the New Forest in Hampshire. Despite the name, it was established during the Norman period and at least feels like a substantial tract of older woodland. (Tiny by North American standards of course.)

Forest was more of a legal term, and tended to be focused on deer than trees.

Things with 'new' in the name tend to age quite rapidly. See also Oxford's New College.

Or the Pont-neuf ("new bridge") in Paris being the oldest one in the city.

I am really surprised you’re seeing half the generation performance of 3.6 with 3.8 at the same parameter size and quantization (and same prefill performance to boot) - is there just an optimization in the stack somewhere for 3.6 that hasn’t landed yet for 3.8?

Curious if other folks are also surprised by this apparent discrepancy or have a ready explanation.

I suspect this might be due to MTP mispedictions. Either because the 3.8 quantized model weights do not include MTP heads, or as what happened with Ornith-1.5 recently, corrupted MTP heads, or due to a software issue in Ollama.

I remember an article a couple weeks back where someone used an HPE server with two older Xeon E-series CPUs and got reasonable performance by compiling llama with optimal switches for the architecture (memory alignment, page sizes, etc). I was impressed because those Xeons only had AVX2. If you are willing to spend a little more, you can get a slightly newer Xeon with AVX-512 and 4 sockets.

I can't find the post though.

Edit: a quick search from a server refurbisher nearby gives me a dual Xeon Gold 6330 28-core machine with 512GB and a 16GB V100 GPU. The memory is spread across 32 slots, for about £6,470.

If I understand correctly, it’s not that the LLM can’t write a good comment, it’s that you want to be able to interpret and understand the generated code without comments - and in that process end up writing comments yourself.

I don’t get that argument. Most of the time by the end of the session the comments from the agent encode tricky details that I told the agent to write down so it stops making “simplifying” assumptions. Thus comments at the end of a couple days of agent-only coding, when I start to actually read and edit the prose, contain the details which aren’t possible to know from reading the local code. It may help that my last couple sessions before I start reading the code myself are variations on telling the agent to self-review and improve the comments in specific ways, so the comments left are only those the agent thought remain meaningful at clarifying unexpected interactions between the local code and other code that needs to be referenced to understand it.