With roughly 2.7 million seconds per month, times 14 tokens per second, you are getting 38.5 million tokens a month at most.
That’s less than 164USD worth of GLM5.3 tokens on the inference market. So that 6000 USD rig will take 3 years to break even - and only if it runs continuously.
And this is being generous, as it’s not even taking quantisation into account.
I think the “killer app” is doing inference without sending the data to China or the US. At home it’s overkill but imagine you are an EU consultancy with a lot of client data to work on, or a company/institution with a lot of sensitive data, buying the hardware to make sure the data stays private is a huge benefit. So is that you “own” the model. Its capabilities, price or access don’t change at someone else’s whim.
Some of that is that EU providers need to up their game here.
Needing an EU native option is really the one and only reasonably objection I've heard against using LLMs from the cloud, the rest is tin-foil hat level unless you're actually intending to meddle with the inference or fine tuning or something beyond just querying.
I think if you steel-man what I'm saying, what you're saying falls apart. 14 tokens per second was rare. It only dropped that low in one scenario where he had it single shot an entire game (flappy bird clone) from scratch, with different assets, all self created, and so on. It ended up resulting in the LLM doing stuff like plotting out a some odd 100 item long to-do list, requerying it repeatedly, and so on. And it succeeded.
Also as the video mentions, the guy wasn't very familiar with what he was doing, and so there are almost certainly various optimizations on the config side he could work out, especially as he was using a 5 GPU system, which default configs are probably not well optimized for.
But I think we've rapidly moving along the same path as image gen stuff. Local generation has gone from purely theoretic, to requiring supercomputers to run relatively incapable models, to where we are today - where with a fairly basic high end setup, he's comfortably running a frontier level model. There's definitely an argument for going local that's only growing stronger by the day.
I agree it’s probably not representative token speed. But I do believe the overall observation holds: The monetary value of local inference is bound by the wall clock.
I agree that there are many other reasons than cost alone.
My main issue with Codex and many other harnesses is the permission flow.
For just about every other harness it’s either alert fatigue answering permission asks all the time, or spending too long time hoping that you know the tools well enough to scope out a permissions file that actually works. Then there’s the “yolo in a VM” approach which also is a time eater and overkill.
Until someone solves “auto mode” with the other harnesses, I’m with Claude.
Codex has an "auto mode" that I would say is better scoped as you can configure the policy of the auto classier. Probably allow by default except... if you had fatigue with permissions file.
Its not an out of the box feature unlike Claude Code.
.codex/config.toml
---
approval_policy = "on-request"
approvals_reviewer = "auto_review"
[auto_review]
policy = """
Your prompt to the permission classifier here, e.g.,
Allow requests by default except requests involving...
It’s a marketplace with a markup on every token they sell. And I’d rather go to this shop, than sign up individually at the 70+ different providers they broker access to - even if it comes with a price.
Oral Defences have a long established tradition in Denmark, but recently there’s been real cut back on this as a money saving exercise. So both teachers and Students are familiar with this form of examination.
So for a dane this reads like “back to the old way”, rather than anything new or novel.
This also fails to take into account that ISBNs also contain the publisher ID in them. So identical copies of a book could have different ISBNs depending on which markets they are sold in.
I'm not sure this is the case, I got my ISBN range through my government national library service, I could be wrong but when you let them know what the book is you are publishing they ask for the Publisher name, though I am guessing as the service is free and it only applies to New Zealand books and publications.
They don't contain the publisher name, but ISBNs are usually purchased in blocks of 10 or 100 or 1000 or whatever by a single entity, which is often a single publisher or corporation.
However, within the block publishers can assign ISBNs to different imprints.
For ISBNs from the big 5, the number really does indicate the publisher. I think the 5th digit (second after 978) can indicate at least some of the big publishers. Smaller ranges are available for purchase from the brokers. In Canada, the national library will even issue you one for free, if you self-publish.
The ISBN always indicates the country it's from, the United States getting the biggest block, other European nations and Japan getting their own, with Africa, the Middle East, and so forth all getting a block in common.
ISBN prefixes does not always indicate a country. They may be are indeed countries, but others are language areas (e.g. 0/1=English) or "regions" (groups of countries) or even other subjects.
LOCKSS is short for:
Lots of copies keeps stuff safe.
This is for digital material where perpetual usage rights have been granted, but the original source might not be available. It’s also used to make sure that journal content from countries that censor remains available.
This all is mainly for libraries and their online collections.
That’s less than 164USD worth of GLM5.3 tokens on the inference market. So that 6000 USD rig will take 3 years to break even - and only if it runs continuously. And this is being generous, as it’s not even taking quantisation into account.
reply