A simple line chart would be more helpful, informative and humbling than a heat map.

I agree, I personally dont like the heatmap and find it very hard to read.

I agree, except for how it lets us see that AI agents increasingly don't mind working on the weekends

adding multiple charts now!

added!

excellent, thanks. Yes, much more interesting.

A weak minded action doesnt produce a win for all parties involved. In fact I applaud their bravery.

The researcher in question has publicly stated[0] that he wanted to try Google's paranoid_crypto tool[1], so he ran it on some roots as a quick proof-of-concept, and the first thing that happened was that it detected these intermediates. He also states he's been running a lot of certificate linters so this was just the next logical step I guess.

[0]: https://groups.google.com/a/mozilla.org/g/dev-security-polic...

[1]: https://github.com/google/paranoid_crypto

That was an enlightening read. I find it incredible that apparently Google themselves don't use the existing corpus of certificates as a test for paranoid_crypto.

I appreciate this.

athropic had an experiment that did exactly this. we need more experimentation in this space.

Wow! This this reminds me of when we tried doing this with gpt 3 and.. it worked!

The tmux thing had me confused for a moment until i realized it was emulating tmux. Super fun. Is the model window width aware?

The first part of the context is the window width:

``` BOS SCREEN_START 32 96 ```

I trained it on three different resolutions, but it should kind of work with in between ones too.

Very thoughtful, nice work again!

good old days https://news.ycombinator.com/item?id=33847479

Thank you for linking that. I distinctly remember this moment as an important point in time, played around with emulating a terminal for hours, couldn’t believe how realistic it was, it even remembered when I changed a file earlier etc. Another point was when people on twitter with early access were like “oh my god it can write code too! I asked it to design a website and it just works!”, programming wasn’t even what LLMs were about, and now it’s one of the main use cases. Another revelation was when it could kinda play chess. A single model that can one shot a huge array of different problems, still mind blown how well all of this works to this day.

There's nothing that makes this paper the be and and end all opinion pieces of the paper. The community will review his review, and people who wish to understand more will read more and more of the paper. I see nothing wrong with this process of progressive understanding.

This only applies to its value in regards to training data for LLMs.

No, it will also be reflected in the value of IP.

Ha, reminds me of how anthropic publishes their math papers with meat proxy math professors who were compensated for their "authorship" of the paper.

Essentially gifts for the annointed.

can you turn this into a decision model

You can train a traditional classifier on it quite well.

But the one-shot and few-shot classification performance of decision models really needs in-context learning. The most common way of getting that is by grafting a small LLM onto it, which is certainly possible.

At that point though, why not just use a small LLM with audio capabilities. Gemma E2B is a good model that is already setup for this, and training a decision model LoRA adapter is very easy.

Edit: Apparently people have got decision models working on EmbeddingGemma, eg: https://huggingface.co/mrm8488/embeddinggemma-2-ft-typed-dec.... I'm not seeing it ranked on https://huggingface.co/spaces/multimodalart/jev-decision-ind... so I have questions about how well it does in-context learning.

What would a decision model add to and audio transcriber?

Reducing cost for a large scale corporation call center. Press or say one.

Basic ML is probably even cheaper. But LLMs might have higher accuracy. “Uhh one I think” = 1

I haven't tested audio, but in general, for specific tasks, classifiers tend to do better than decision models ("general classifiers" based on LLMs): https://jeffyclassify.com/blog/when-llms-fail-at-classificat...

Theoretically - if the change was deeper than just restricting output - you could feed it thousands worth of audio tokens and get a structured response in milliseconds rather than minutes?

Say classify recorded interviews into viable/not viable candidate?

It can already classify decently well, and I tried training it as a decision model, It worked well on domains I trained it for. It was beating Liquid's small decision model, but somehow, it never generalized. Out of the trained domains, it was just above chance.

Isn't EmbeddingGemma2 already a decision model? Am I misunderstanding your question?

EmbeddingGemma 2 is not natively a decision model; it is a multimodal embedding model

it was a joke

Hard to call it a "unified platform" when a good portion of the developer base is using pages router despite app router being out for a few years now