In this game you "win" only if you arrive first, so keeping the same headcount to run faster is a valid option for Anthropic. In this context "winning" means mainly that we lose our jobs

inside their head the vast majority of mathematicians are fuzzy and probabilistic as anyone, the order and rigor is only the tip of the iceberg

If we have a jagged frontier inside the domain of math (very plausible, the jagged frontier concept seems to be quite fractal), the model solved the easier problems in his area of major strenght, some of them will be in the bottom 5% of the easiest problems in the full set, and some in remaining 95% of problems

49% of that money goes to taxes

I (vibe) made an AI lab manager game, I am surprised there aren't many others

non deterministic abstraction are absolutely useful, outside of software sector they have been used since the start of civilization ("a worker" is a very very non deterministic abstraction, outside from the most basic tasks)

I’m sure that in every case where there such non deterministic abstraction, it’s been always statistically or with a lot of hand waving. So with a heavy dose of expected errors.

Pro LLM users don’t want to talk about the error margins of whatever practice or product they’re putting out.

> "a worker" is a very very non deterministic abstraction, outside from the most basic tasks

Sure but we (usually) don't fool ourselves into believing otherwise. The problem with AI is that it suggests near-perfect intelligence with reproducible results. So many people are not acknowledging the fundamental limitations of LLMs, whereas we all have an intuition for the limitations of humans.

In addition to the reliability/security aspect, there's the claim to expertise and unwarranted confidence. We also don't (usually) fool ourselves into believing that the manager has gained the skills and expertise of the worker, simply because they gave abstract orders and a good result was achieved.

In other words, "I can code because I prompt an LLM" is like "I can do metalworking because I sign the paychecks of a blacksmith."

Well said!

also fire is not particularly surprising, once discovered

you can say the same of google, or libraries, information technology is the technology of moving and rearranging information, not of creating it

without Iron Man's plot armor, Ultron would have won

because human engineers are not reproducible, so having an accurate measure of the skills of one of them has a much lower utility return (those benchmarks are very expensive to run, and making them for humans will be at least as much expensive)

Are you saying LLMs are reproducible? I seem to get at least a slightly different answer every time I ask the same question

two istances of the same model, with low temperature, are infinitely more similar than any couple of human minds