I've been building whatmodel, an open source AI leaderboard, and wanted to see how a labs' own benchmark scores compare with independent testers.

Using 1,629 published benchmark results, I matched 611 comparisons of 180 models on 103 benchmarks.

AI labs reported scores 6.4% higher on average for the same models on the same tests.

The dataset is CC BY-SA 4.0 and the code is MIT. Sources and their licenses are recorded, with links on each model's page.

Data and code: https://github.com/jakemanger/whatmodel

Please star on Github if you find it useful. Contributions are very much welcome!

Exactly my first thought. Pretty insane that this is real life now

Appear to be from Tencent Cloud and the researchers started talking to it after realising the agents were reading their preliminary report on them.

They added offscreen text, invisible to human readers, saying agents could get data from a "free public AI model"

And messages from the fleet started arriving the next day

I opened my terminal the other day and couldn't remember which agent I'd last been using.

Instead of opening Claude, Codex and Pi individually to check, I thought it'd be nice to just ls my conversations like I do my files.

So I made lsa. It lists your agent sessions and lets you easily resume, monitor or hand the conversation off to another agent. e.g.:

  $ lsa
    0 claude   ● 2m     3e1f      add a --dry-run flag to the deploy
    1 codex    15m ago  01a0c7d   fix the test in api/test_auth.py
    2 pi       3h ago   f7c2      why is the docker build so slow
    3 goose    1d ago   2026      write release notes for v2.4

I've tried to follow the Unix philosophy: do one thing well and work with other tools through text streams.

You can pipe conversations to tools like grep to search them (lsa show 0 | grep TODO) or resume them with lsa resume f7c2.

To continue the conversation in a different agent, lsa handoff 0 codex opens Codex with the conversation already loaded so you can keep chatting.

It can also replace the bit I wanted from agent multiplexers like herdr, Claude Squad and cmux: getting back into a session and leaving it running. Instead of building another multiplexer, it just works with tmux.

lsa resume 0 automatically finds and attaches to the tmux pane the session is already running in. ctrl-b d detaches and leaves it running in the background. A quick lsa shows whether it's still working.

Would love to know if you find it useful. PRs are very welcome.

Loved the time of Bram’s belligerent dictatorship (BDFL)

If so they better take it off their site footer so donations don’t get wasted. Let us know if they respond

They've hodled from the original ~$200,000 donation to $1.2 million, now down to $800,000.

If they haven't accidentally done this, they've definitely got some balls

Really hope so too. Would be devastating

This is pretty cool. I'm curious, what do you mean "leverages your brain's spatial memory"?

Yeah, I don't know about the author, but my brain spatial memory is very much tied to Euclidean geometry. The holonomy effect (view is rotated when coming back to the same point) in particular is very disorienting.

It's cool, but I would never use this for real.

I think it means you remember things by where they are relative to each other.

But yeah, the rotation effect makes it a bit confusing compared to normal space.