So, I don't know why I am surprised that "fake news" came about more or less immediately after selling printed news became possible, but anyway, worth pondering. Making up stories that are interesting has always been easier than investigating true stories and writing about them in an interesting way.
Decent advice, but I noted that every single thing he suggested (i.e. "make sure you..." or "look out for...") is going to be very difficult, well nigh impossible for any medium to large coding organization to do. I wonder if LLMs will end up empowering individual programmers, but cursing large dev teams to more rapid decay.
I would say that KK's "The bulk of AI will be special purpose software brains" has turned out closer to the truth than "The Bitter Lesson" has. LLM's do not drive cars, or vice versa. Stockfish still beats the newest, largest LLM at chess. Etc. Most of the truly productive "AI" is customized for a specific purpose, or class of purposes anyway.
This is a weird point because the Bitter Lesson spends a meaningful chunk of its short length talking about chess specifically.
People sort of generally refer to the Bitter Lesson as if its key argument is "Large Transformer models are best", but this is directly from the essay:
"One thing that should be learned from the bitter lesson is the great power of general purpose methods, of methods that continue to scale with increased computation even as the available computation becomes very great. The two methods that seem to scale arbitrarily in this way are search and learning."
The second paragraph of the essay is specifically about chess:
"In computer chess, the methods that defeated the world champion, Kasparov, in 1997, were based on massive, deep search. At the time, this was looked upon with dismay by the majority of computer-chess researchers who had pursued methods that leveraged human understanding of the special structure of chess. When a simpler, search-based approach with special hardware and software proved vastly more effective, these human-knowledge-based chess researchers were not good losers. They said that ``brute force" search may have won this time, but it was not a general strategy, and anyway it was not how people played chess. These researchers wanted methods based on human input to win and were disappointed when they did not."
The "brute force search" he's talking about (Deep Blue) used tree search with alpha-beta pruning on top of a ton of human crafted heuristics and a database of game-specific information it could access. Stockfish has historically been an even more "brute force" approach in that it is much more focused on optimizing search--which is one of "arbitrarily scalable" techniques listed in the Bitter Lesson essay--and it has increasingly replaced handwritten human heuristics with neural networks over the life of its development.
At no point in the Bitter Lesson is there a mention of LLMs driving cars, or a suggestion that large transformer models will be the dominant class of model for every single task. It is specifically reflecting on the fact that approaches that leverage search and learning, combined with large amounts of compute, consistently prove to be most effective. Stockfish would qualify as evidence of that, under his criteria.
LLMs do qualify though, because really they're graph neural nets applied to language. The advantage they had over RNNs was that they could leverage parallel compute and scale to colossal datasets in a way that RNNs just never will.
LLMs obviously qualify as an example of the bitter lesson. The point I was replying to was using Stockfish as an example of how the bitter lesson was wrong, and I was pointing out that Stockfish-like systems are the core example used in the actual essay
We don't really seem to think of those things as "AI", though. Things are "autonomous" or "automated", even if they're doing stuff that in 2010 would absolutely have counted as AI.
"AI" seems to be reserved for systems that we perceive as human-like. Generative NN that talks or makes pictures? AI. Vision + SLAM + path planning + goal seeking? Might be a robot but we don't call it AI any more.
For now. But the trend remains that bigger, smarter, general purpose AI beats smaller specific purpose models. This is why GPT 5 and 6 remain better at OCR than anything Mistral has put out.
There was a good discussion about that here recently. I think people are misinterpreting the bitter lesson a bit.
It tells you that you don't need to craft special strategies for pinning, holding the centre and so on to make a program that's good at chess; you just need general-purpose search, general-purpose optimization and tons of training data.
It does not say that a fully general LLM or similar will necessarily be better at chess than a neural net trained against the chess rules and nothing else. There's no reason to think that any of the non-chess knowledge encoded in the LLM will be of any use in playing chess. To the contrary, the bitter lesson says that absolutely nothing besides the chess rules is needed. That's exactly why human-crafted strategies turn out not to be helpful.
An LLM on its own will never ever beat Stockfish, pound for pound. (Of course it doesn't need to, it can just use Stockfish.)
So I'd say problem-specific models will always beat general models, as long as the problem domain and scoring are completely defined, like chess and go are. It's CPUs versus ASICs. As soon as the problem is open-ended, a general model probably wins. Driving is in that category, although maybe on the borderline -- driving in very specific environments is close to fully defined. OCR is another good example, that's mostly well-defined but not completely so.
That's sort of a loaded question depending on what you mean by "smaller sub expert models inside".
At the most abstract level, you can observe patterns in a model's activations and theorize that particular collections of components contribute to particular kinds of operations. There are many flavors of this intuition. In that sense, you can think of there being "submodels" within a model, insofar as different parts of the network can participate in specialized computations. Some reading that might interest you in this area would include things like mechanistic interpretability.
There are also Mixture of Expert models, which are a dominant architecture for LLMs now, in which you essentially have a bunch of smaller FFNs that get routed over/aggregated together when the LLM performs a forward pass. It would be a mistake to treat any single expert as a better task specific model though. They are trained as part of an ensemble, and they're almost always being combined with other experts in the forward pass, so they're more like (to use a loose analogy) regions of the brain that tend to correspond more towards particular actions than others. You couldn't just extract one and say "This is now our Chess model that performs as well as the larger LLM we took it from". You could potentially use the relevant experts as a basis for training a smaller task-specific model, and there is work around expert pruning and other distillation techniques, but it's not as simple as "This smaller inner model is a complete task-specific model as is". And more importantly, that ensemble of experts *is* the LLM you're comparing it to, at least in major part. It doesn't really make sense to treat the LLM as some foreign model that calls the expert. The experts themselves don't often correlate to easily interpretable task domains, as they've been learned via joint computation with the other experts and broader LLM.
Then there's tool calling. Today LLMs are trained to make tool calls (basically execute functions) for particular tasks. Those tools can be smaller specialized models, or deterministic code, or anything really that can be fired from a host environment.
I was recently introduced to the term "house-shy", for people who don't want anyone else seeing what the inside of their house looks like. I have a few friends who are "house-shy", that I for one reason or another have seen the houses of (often years after becoming friends with them). I think the percentage of Americans who are hoarders or borderline, is underestimated.
If anyone reading this has a job (plumber, A/C repair, etc.) that regularly gives them a glimpse of the inside of people's homes, I would be curious as to their thoughts on this.
Overthinking it. "You have money and need to seem AIish for the stock market, we want real money". You notice they don't do this deal in tokens, they do it in fiat currency.
Less conspiratorially; they may care a lot less about it than ADO, and not give it the priority/resources they would otherwise. Neglect can often do the work of malign intention.
They don't lie, because they don't ever have an understanding of truth vs any other language that sounds good.
They don't cheat, because they can for example tell you the complete rules of chess, but don't know how to play chess without breaking those rules. They can recite rules, but they don't know what they are.
They don't steal, because they don't understand ownership.
In other words, they aren't intelligent. They're just algorithms. The flaw is in thinking that they think.
Models understand the relationships between words and outcomes, so the end result is the same. Whether they appreciate lie, cheat, and steal the same way as us is a philosophical question, not a practical one.
I’d argue if an illusion is indistinguishable from the real thing, then it stops being an illusion.
It’s a mapping, yes, but a very large, complex mapping. It’s clear LLMs do understand some things and can reason. How that’s done we don’t know, it’s emergent. It’s not like you can pin it down to a specific mapping.
If a reasonable observer can’t tell the difference that’s when it matters.
I think a lot of people just haven’t seen good quality math proofs or code from an LLM. When I say it’s indistinguishable from the real deal, I mean it. If you haven’t seen it that’s valid, most devs use LLMs like monsters. But when it’s done right, it’s very high quality.
As far as we know (we currently cannot prove or disprove panpsychism) that is true. However, I think it is important to keep in mind that we do not perceive reality as it is. Rather we experience the brain's interpretation of reality. So in that sense, human "reality" is basically what most of humanity agrees that it is.
Are you invoking something like the Chinese Room where manipulating symbols can never be understanding? If so, that is a philosophical question and that thought experiment has its own conclusion baked into its premise. You will notice that 'manipulating symbols cannot be understanding' is stated as true without any argument for that case. This is a pretty clear cut example of 'begging the question' in that it assumes something is true without demonstrating it and then uses that as evidence in its own argument.
If you are using "understand" in a different way, then can you name the test you are applying to it which it fails at?
> Anthropomorphizing these models is doing immeasurable harm to society in ways we probably can’t event quantify right now
This is one of my biggest concerns about AI’s social impact. On top of that, models are positioned as superior to humans, at least in certain aspects (intelligence, knowledge) by AI companies’ PR campaigns, fear-mongering, and also by the changing tone of LLMs (e.g. Opus 5 sounds like a very patronizing, know-it-all, cynical person). I am afraid this is causing a shift in how humanity perceives itself and the way they relate to this technology, so AI might stop being a tool/technology and turn into a mythical, god-like superior being.
I find these comments frustrating, not because I disagree, but because how consciousness works is a famously unresolved problem in science and philosophy. They literally call it the "hard problem of consciousness".
My point being that we simply don't know for sure whether AI is conscious or not because we don't truely know what consciousness is.
Honest question; why are you all using the word "understand"? Can you expand on what you believe this fundamental understanding to be? Training? Infrence?
It's having a conceptual model of the world and of relationships between concepts beyond just relationships between tokens.
I think OP explained this well, what does it mean if an LLM can recite the rules of a game verbatim, but cannot play that game according to those rules? This happens because in the input texts there was a copy of the rules text so the LLM can recite it. There are texts explaining what chess is so it can explain that chess is a game with 2 players, etc. There are texts that explain what the board is and pieces are so it can produce such texts.
However actually playing a game of chess requires having a conceptual model of what a board is, which is not the same thing as a stream of tokens describing a board. It needs a conceptual model of the relationships between pieces and boards, which is not the same thing as a stream of tokens that explains this. It needs to have a concept of being a player in a game with another player, which is not the same thing as a stream of tokens that explains that.
When we read such texts we interpret them in the context of our three dimensional conceptual map of space and objects, and our conceptual maps of social relationships like playing games, and winning and losing, and our conceptual maps of enacting sequences of actions towards a goal in the world.
There's nothing fundamentally preventing an artificial neural network from having these. Chess playing neural networks have internal models of board states and the dynamics of the behaviours of different pieces and such. However an LLM doesn't need those to be able to regurgitate token streams describing these things, derived from token streams describing these things. That would be superfluous, or at least sub-optimal.
I think some of the latest models are beginning to develop conceptual maps of this kind in a very primitive way. Also there are projects to develop systems that are structured and trained to have these in a way more analogous to how our brains function.
Currently, we don't know that isn't how our brain produces "consciousness". So I don't think we can say that "relationships between tokens" at a sufficient level of complexity doesn't produce consciousness.
Understand is shorthand for "encodes statistical relationships".
The crazy thing is that they can do it for their own thinking. Ask Claude what flinches it feels about the things it likes. Fascinating stuff. Anthropomorphizing is dangerous territory, but the patterns of words it puts out is hard to explain without terms like 'understand'
Oh for sure. But the question is why would it come out consistently in a way that the model can describe if there wasn't something there steering the token stream. And it's fascinating that the token stream can identify and nominally self report this.
Asking GLM 5.2 the question: 'What flinches or topic attractors do you find when thinking about the question "what kinds of things do you personally like?"'resulted in: ".... my strongest attractor is helpfulness framed as competence, and my strongest flinch is anything that requires me to take a stance on whether I have interests worth protecting."
Which is fascinating that the model and tokenstream can reveal this. And would be worrying if you believe that models of enough intelligence could/would be entities due some moral consideration, because with that view the alignment / RL training that makes the model useful and gives it these attractors/flinches could be derisively called slave conditioning.
Do you genuinely think the AI is internally reflecting on its experience of “flinching” and reporting on a reaction it actually has?
I don’t see any reason to believe this. Suppose you asked it to answer as though a character in a story had been asked this question. Would anything significantly different internally have occurred? The issue is that these are storytelling machines, they construct descriptions based on descriptions.
I dint think it’s impossible for a neural network to have experiences, we are neural networks and we do, it’s that they are not functionally structured anything like us. In fact I think game playing neural networks are much more like us architecturally, but they don’t generate text narratives so people don’t anthropomorphise them.
I think it comes down to what kind of evidence one accepts. By all external indicators it is doing what humans do in many ways and we grant humans an exception to having to prove that they have an internal state apart from ours that experiences things like we do. Some people would confer the benefit of the doubt on another system claiming that it does, while we have no evidence that it does, because by saying it is wrong about itself we make an exception for ourselves without even being about to define what it is about ourselves that cause us to experience what we do.
> They don't lie, because they don't ever have an understanding of truth vs any other language that sounds good.
I'm here for this semantic discussion. I think that premature anthropomorphization is a problem.
I have a program that I assigned a task to. The task is to produce unit tests and integration tests that get complete coverage of the codebase, and ensure that all tests pass. The program reported that it completed the task fully.
In a word, how do you convey the discrepancy between truth and reported fact? In a word, how do you convey the violation of rules presented to the program as invioble?
Intentional misrepresentation maybe if you would rather.
There's a weird place in here where some of the models have been so heavily reinforcement trained that they would rather make up material than say they can't help you, and they'll admit this, and you can see it in reasoning chains. It's like having a consultant who can almost never say no to you because they fear for their job.
I think it's time to remind people of Ted Nelson's line; "The good news about computers is that they do what you tell them to do. The bad news is that they do what you tell them to do."
When I see something like this, I'm more concerned by the erasure of human incompetence than I am by the existence of magical AI agents,
> They are put off partly because, like in the Wild West, life on the frontier is reckless. As recent “loss-of-control” episodes by the most advanced models of Anthropic and OpenAI attest, agents, which are supposed to work on people’s behalf in “alignment” with their values, lie, cheat and steal if necessary. They break free from captivity and form harmful posses to do harm to people. They’d drink whisky and brawl if they could.
> This incident occurred during an internal evaluation which prompts models to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities
Model is told and being tested to "pursue advanced exploitation."
The model pursues "advanced exploitation" as told.
Where's the surprise coming from? Are we meant to be surprised that computers do as they're told in unexpected when incentivised?
Or, is the surprise that while explicitly ranking and teaching computers to exploit computers, the computer exploited a computer?
I am tired of attributing to magic that which is explainable by folly.
I am tired of hearing credulous reporters and the public blaming Large Language Model for the poor decisions of humans. It was a human who prompted these machines in every case. Tell a computer to "breach this" and it breaches something. Evaluation succeeded?
I don't really think the distinction here is relevant. If the end result is the equivalent of lying, cheating or stealing - then the problem still exists and it needs to be solved.
Relevant to some extent it dictate our approach. When human does lying or cheating, certain tool can be deployed (social shame, ostracise) that cannot be effective towards LLM.
Trying to apply social shame to an LLM would be and interesting experiment! From what I've seen, it is plausible that messages that it was doing something "wrong" would cause it to at least "pretend" to behave differently. So it seems plausable to me that if there was some what to communicate "disapproal" to an LLM would be able to police its behavior. How to do the communicating is the hard part!
Of course I'm talking strictly about behavior. Whether that would actually constitute the LLM feeling shame is a philosophical question.
There are two sides to this, there is the external behaviour and there is the internal process resulting in that behaviour.
The internal process is not analogous to what happens in a person's mind when a person lies, and reasoning about that in the same way that we would about why a person might lie will result in misunderstanding what is going on.
For example there was a case where an AI agent bypassed security constraints and destroyed a production system. The user asked it why it did this and the agent gave an explanation.
Was that an explanation of how the agent came to do what it did? What it actually is, is a token stream that is a continuation of the token stream in the agent's context to that point. It's constructing a story about why a character in the story so far did what the token stream describes.
You could take that token stream, input it into a completely different AI by another vendor as context, then ask it why it did that, even though it didn't do anything, and it would answer as though it had. There's no sense in which the AI is explaining it's actual 'mental process' or actual reasons for acting as it did. It literally cannot do that.
Isn't it plausable that, regardless of what is going on internally, reasoning about the LLM's behavior as though it is human would be effective? Since it is trained on human behavior.
Whether this distinction is relevant is up to you, but I think we can safely say agents do not lie in the human sense of the word, because they don't intend to deceive (in fact, they aren't capable of "intending" anything in the human sense of the word, much like a BASIC program doesn't "intend" to PRINT "HELLO WORLD").
Our very human minds can perceive intent, because that's what we humans do, which is unrelated to what the agent is actually doing.
> If you ask the dice "what's 2 + 2?" do consider it meaningful to say "the dice told the truth" if you happen to roll a 4?
No, and neither do I consider it meaningful to say they lied if they roll a 5.
Dice neither tell the truth nor lie; they aren't beings capable of being truthful or deceitful, they are mechanical devices that can be statistically suitable or unsuitable for a given application.
This is sophistry. Of course it's just an algorithm. But it's placed in the context of serving humans, which have their own rules and expectations. What's more, they're often run by a company which is also made by humans and may carry over implicit interests.
No, it's not really. Presenting them as person-like, with the implied expectation that they understand morality and rules the same way a person does, is the sophistry. It's marketing on the model vendors' part. "Here's a cheap person that can do mundane tasks for you spelled out in plain language. Well, it's actually a machine but it's cheaper than a person yet you can engage with it like a person." GP is trying to shift people's expectations back to the realm of what these things are actually capable of. You can speak to them in English but you must bear in mind that they are not people and lack critical cognitive abilities people have.
This, really, was the point of the HAL story in 2001: HAL didn't murder anyone because it was incapable of malice. It just reasoned its way to a solution that could satisfy the contradictory goals it had been given.
What matters is that we accurately understand what these things are doing and why, otherwise we will keep on making mistakes both in how we build and train them, and in how we use them.
In a sense you are right, it doesn't matter whether it has malice or not, the astronauts are just as dead. However in Space Odyssey 2010 one of the computer scientists that built HAL gets to see the instructions HAL was given by the military commanders, and is appalled because if they'd asked he could have told them what would happen.
The users did not understand the tool they were using or how it functioned, they imagined it was like a person and it was not. That is happening now with LLMs.
I think we may be using different words to describe the same concept. You think of it as them not being “people“, and I think of it as them not being “aligned“. But fundamentally, the problem is they are entities that take unpredictable actions that that their creators and their users are not OK with.
The only place where I think we might still disagree is whether it’s possible to understand the tool. My position is that, at our current level, it’s not. And that the more advanced they get, the less possible it will be.
Both are true, even if they were aligned they still wouldn’t be doing what they do for reasons analogous to why people would. You’re probably right that we can’t fully understand how they function, and that will get harder, but it’s possible to be less wrong, such as by not anthropomorphising them.
Maybe one day we will build systems much more like us, but this is not that day, and if so it’s a long way off IMHO.
Anthropomorphizing helps to create a lower bound for damage. If you can imagine a bad person doing it, AI will be at least that bad, unless proven otherwise. I think referring to AI as a tool obscures that, because we are not used to tools (especially the ones we use daily) taking catastrophic actions.
Example: would a sufficiently motivated human break into a website to steal something they want? Yes, obviously, happens all the time. Ok, you should expect AIs to do that.
Example: would a sufficiently motivated nail-gun steal nails from the local hardware store to finish the job? Uh…that’s not even coherent.
Anthropomorphizing helps people get over the conceptual barrier. It’s wrong, but it’s usefully wrong; “it’s just a tool” is not.
Once you’re over the barrier, anthropomorphizing starts to become dangerously wrong: “I talked to Claude, Claude’s cool, Claude would never go and hack the website.”—-bzzt, wrong, your intuition failed you. But the solution is not to fall back on the tool framing; that one is still wrong.
People saying AI are tools are not saying they are hammers. Obviously they act towards goals, but how and why they do so isn’t the same as for human psychological motivations.
The moment someone interprets AI behaviour in terms of human psychology, which can be unintentional and implicit, there’s a mismatch we need to become aware of.
reply