> Create a realistic eval suite for your use case using your actual data that you manually label, with easier, more complex, rarer samples, and run each model you are considering against it and decide based on the usual confusion matrix, cost, latency, or whatever metrics matter to you.

I’d imagine there’s only a tiny percentage of users that can or want to do this.

if you consider every single person that uses some LLM then sure, it is tiny percentage, but I would wage that large percentage don't write technical documentation as GP. large userbase probably don't need THE best solution for their use case and will be comfortable with A solution that is within their budget. so for them simple luna < terra < sol < astra would suffice.

but if you want to use some LLM agent for anything commercial and you want to get the best and at the same time you don't want to overpay the easiest and the surest would be to create your own benchmark for your specific use case.

If they’re targeting a $2T valuation, then spinning up a $20B company would be a 1% increase in value.

They’re chasing massive markets strategically, not trying to brute force small ones.

1% * 100 is... 100%

(and diversified risk)

a better reason might be that people would not want to use OpenAI because they might be sherlocked.

But what’s a reasonable win rate? 50%? Then you need 200 bets that could realistically capture $20B each (probably more to cover your losses).

$20B is a massively successful company.

And who’s managing the strategy behind a company that’s competing in hundreds of diversified verticals?

At that point, just take a % of an industry leader in exchange for giving them your frontier AI capabilities, some compute and maybe a little AI-native training/expertise.

Or just pick 10-20 $200B opportunities to go after.

> What if I specifically tell it to not do anything nefarious or illegal?

Sure, but a reasonable user would expect that condition to be implicit in using a commercial model.

It’s what the safety/alignment discussion is about: how we can get models to actually comply with laws & ethics.

But the same problem exists in the real world. If you tell someone to “Try everything possible to get it done, I don’t care what it takes”, I’d imagine that people would be far more likely to bend/break rules than if you said “could you please help me with… ?”

Do agents know exactly what jurisdiction I am in and which national, state, and local laws apply to me?

Good point. Not sure the average user would consider this though.

It’s not their responsibility to know, it’s yours.

That might be the ultimate legal guidance, but is it settled though?

> Sure, but a reasonable user would expect that condition to be implicit in using a commercial model.

In fact, adding it might make things worse as it shows at least some awareness of the risks.

Cool experiment Zach!

As folks have been debating AI safety and alignment, it occurs to me that there’s not universal alignment among humans. Per the stat you surfaced: 81% of humans pulled the lever, 19% didn’t.

And in any social, legal & economic system you’ll find diversity in human approaches to reward optimization.

Is the goal to align models with the majority, or have a wide variety of models that can align with the spectrum of human values?

The first goal is to align the model to the least common denominator of ethics. Even if there is diversity in values, everyone agrees that murder is bad, or that you shouldn't torture your neighbor for fun.

The whole point of the thought experiment is that not everyone agrees that pulling the lever to kill the one man is murder and thus immoral.

No audio on Safari mobile either

For me it works fine on mobile safari, possibly earlier bug/hug?

> I don't think any new languages will easily beat the existing ones, simply because of the mass of training data that is available.

Seems like a solvable problem though by generating synthetic data that’s guaranteed to be accurate through linters, compilers and tests.

If it’s ~9 figures to train a frontier model, it seems like training on a new language could be a rounding error if it was a priority.

I could see the appeal of a new agentic-friendly language that’s focused on minimizing tokens. The standard library could be massive with no concern of making the language easily readable or learnable.

> Seems like a solvable problem though by generating synthetic data that’s guaranteed to be accurate through linters, compilers and tests.

This makes sense and is very insightful, thank you. But with that solution it seems that only LLM companies will be in the position to create new languages.

If it was a poison pill then why make the severance $8M instead of the majority (or all) of the value of the company?

The demand curve for speed and intelligence seems pretty steep to me.

If you look at the hiring marketplace, being just marginally better than your peers can be very lucrative.

If you’re competing on speed or capability as a company (or as an employee), you’re probably going to be willing to pay for the frontier.

> If you look at the hiring marketplace, being just marginally better than your peers can be very lucrative.

I would say that in software this is completely false.

Someone straight out of college, not very useful, makes 75-100K.

Top level senior outside of FAANG is making twice that at best (and at least 10x more capable).

You told on yourself about being either European or from a flyover state.

I happen to be from NJ, but you're also not going to be making a Meta/Google salary in NY outside of NYC.

Most companies have a limited budget.

Good enough with a lower price will win the masses

Employees can quit too.

In both scenarios, it really depends on how many alternative options for income you have available.

Sure, but as a business you're ideally already somewhat diversified in clients.

As employee, it's all or nothing.

When you’re paying someone’s bill/salary, it changes the dynamic.

Just like someone in customer service might act differently with coworkers vs customers.