Jumbling together year, month and day and having to separate them by counting digits is not an improvement for human readability. (And you have the year wrong.) Any of “27 July 2026”, “July 27, 2026” or “2026-07-27” would be superior.
It's good because it's an actual standard instead of 7/27 which is backwards for half the world. Besides, I'm actually from the future and they delayed the model a year, learning from Anthropic that good models are actually about to bring the end of society.
> But SIMD code is still not write-once-run-everywhere.
Modern, portable SIMD libraries are making it increasingly close to that. Highway is my personal favourite, thanks in part for its support for runtime dispatch.
(Disclaimer: I work at Google, but not on Highway, and am writing in my personal capacity.)
Your numbers are for “max”. Opus 5.0 “max” is $2.03. Opus 5.0 “high” (competitive with Claude 4.8 “max” on that index) is $1.06, less than the $1.80 you are quoting for 4.8 max.
That the most expensive variant is expensive doesn’t really tell us much.
If you start to drop effort levels, you need to compare to the competition models. So GPT models on the same ~intelligence level, are then 50% cheaper.
You see the issue? Its still a expensive model, and from my understanding, it still uses the old tokenizer.
Going to be interesting to see when GPT 6 comes out (very soon).
> If you start to drop effort levels, you need to compare to the competition models.
Yes. I advocate for doing that.
> So GPT models on the same ~intelligence level, are then 50% cheaper.
How did you reach this conclusion? Opus 5 high ($1.06) has the same “intelligence index” as GPT 5.6 Sol max ($1.04). Opus 5 medium ($0.62) performs a bit below GPT 5.6 Sol xhigh ($0.68) but slightly above GPT 5.6 Sol high ($0.45).
> HN has a fetish for SIMD, but if you are hand-rolling SIMD and you aren't writing an explicit acceleration library, you're doing it wrong. Like, 100% of the time.
What if the “explicit acceleration library” for what you need to do doesn’t exist?
> You really should see if you can get the compiler to auto-vectorize first (possibly padding data structures and loops) before you write anything by hand.
> I think that the fatal flaw with the approach the compiler team was trying to make work was best diagnosed by T. Foley, who’s full of great insights about this stuff: auto-vectorization is not a programming model.
> The problem with an auto-vectorizer is that as long as vectorization can fail (and it will), then if you’re a programmer who actually cares about what code the compiler generates for your program, you must come to deeply understand the auto-vectorizer. Then, when it fails to vectorize code you want to be vectorized, you can either poke it in the right ways or change your program in the right ways so that it works for you again. This is a horrible way to program; it’s all alchemy and guesswork and you need to become deeply specialized about the nuances of a single compiler’s implementation—something you wouldn’t otherwise need to care about one bit.
> And God help you when they release a new version of the compiler with changes to the auto-vectorizer’s implementation.
> With a proper programming model, then the programmer learns the model (which is hopefully fairly clean), one or more compilers implement it, the generated code is predictable (no performance cliffs), and everyone’s happy.
How is that different from any other compiler optimization?
And what does the last statement in the quote mean anyway? When is performance "predictable"; do you freeze the entire toolchain?
And what is the alternative? Handroll manual SIMD code for every possible architecture you may target?
If you're writing C++, you're already rolling on decades of compiler optimization. You return by value because it makes code more readable and safer and rely on RVO. You write functions to abstract and rely on the compiler inlining. When it doesn't work for your specific target/toolchain, you may decide to handroll stuff. The proof that you're banking on the compiler is that if you run a debug build of any non-trivial program, it runs like absolute dogshit.
> How is that different from any other compiler optimization?
When it can be single-handedly responsible for an 8× speedup, you might not want to rely as much on the compiler as in the case of smaller, cumulative optimisations.
> And what is the alternative? Handroll manual SIMD code for every possible architecture you may target?
Forgot to add: using Highway also means that you can detect and take advantage of e.g. AVX2 or AVX-512 in one binary that still works on CPUs without those instruction sets, whereas relying on autovectorisation would mean having to build with `-mavx2`, `-mavx512`, etc. which means that pretty much no one who relies on prebuilt binaries would get them.
That can easily cause you to traverse your data several times when once would be enough. Let’s say you wanted to compute “mean of array divided by max in absolute value”.
NumPy-like:
mean = np.mean(array)
maximum = np.max(np.abs(array)
return mean / maximum
As far as I’m aware, that will be evaluated as three traversals.
Highway:
HWY_FULL(float) d;
using V = decltype(hn::Zero(d));
V sum = hn::Zero(d);
V max = hn::Zero(d);
hn::Foreach(d, values, N, hn::Zero(d), [&](auto d, auto v) HWY_ATTR {
sum = hn::Add(sum, v);
max = hn::Max(max, hn::Abs(v));
});
const float mean = hn::ReduceSum(d, sum) / N;
return mean / hn::ReduceMax(d, max);
Just one pass.
When the actual computation is made so much faster by SIMD, memory bandwidth starts to be a significant bottleneck.
Thanks, that's insightful. I haven't thought about it that way but in retrospect it's clear. In practice I think we've all seen that the numpy style construction is quick to write and performs (and reads) much better than "dumb loops", but if you really want to optimize your code, the highway version will give you more control (and will read similar to the dumb loop with some decorations). I guess just more tools in the toolbox!
reply