Tokens are the most basic input unit of an LLM. But tokens don't generally correspond to words or letters, rather sub-word sequences. So Strawberry might be broken up into two tokens 'straw' and 'berry'. It has trouble distinguishing features that are "sub-token" like specific letter sequences because it doesn't see letter sequences but just the token as a single atomic unit. 'Straw' and 'r' are two tokens but an LLM is entirely blind to the fact that 'straw' has one 'r' in it.
As an analogy, I might ask you to identify the relative activations of each of the three cone types on your retina as I present some solid color image to your eyes. But of course you can't do this, you simply do not have cognitive access to that information. Individual color experiences are your basic vision tokens.
LLMs see tokens, not words spelled out with letters.
Imagine verbally asking someone who has never seen written text the same question: unless they memorized the answer for the specific word you're asking about, they'd have to guess.
People assume that's the reason because it's intuitive and "strawberry" is one token. But that doesn't explain why those models would also often get it wrong for "StRaWbErRy" or even "s-t-r-a-w-b-e-r-r-y", where the r's are not combined into one token.
We don't know what was going on inside the closed source GPT models, but this paper investigated on some of the open-weight models and found it's not due to tokenization: https://arxiv.org/abs/2604.00778
And circling back around to AGI, tokenization or some other underlying cause should pose no issue. A competent human would think to write a program (ie create a tool) to do the job. It's routine for a carpenter to make a jig.
i asked opus 4.5 what the problem was and it said it pattern matched too much. i asked it how it should do it, it wrote a file that told itself to stop pattern matching. it wrote a file that started with the following and then had an english language procedure for how to count letters. so it knew the algorithm already, but the "instinct" was to pattern match rather than running the algorithm.
CRITICAL: Do Not Skip Steps
Your instinct will be to "just know" the answer. This is how you get it wrong.
You don't see characters. You see tokens. Your "intuition" about character counts is pattern-matching, not counting. It is unreliable.
You MUST execute this procedure step-by-step, writing out each step visibly.
In fairness, this at most tells you the UK court system isn't party to any honeypotting.
Despite the memes, the UK is not the 51st state, which is both why it was able to make the fine (1st Amendment doesn't have jurisdiction over the UK) and why it's not able to enforce it (the UK doesn't have jurisdiction over 4chan).
Neither has the power to be enforced outside its own territory.
The entirety of the US constitution carries zero weight to the court system of the nation that the founders of those United States, ahem, engaged in unlawful violent treason against, with the assistance of foreign powers (France, then Spain and the Dutch Republic). :P
I'm not in the tech industry. Could someone explain why there are so many new coding agents, and why they're commonly upvoted on HackerNews? It seems like there's a new one in the top 10 every other day.
It's basically just a relatively simple to create piece of software that's important to get right (since you use it so much), can be made by many different design philosophies (maximal vs. minimal, customizability, etc.), and has very few good standards around it as of yet.
Because a lot of people are writing their own to get a tool that they understand and can manipulate as they like. So they like to share them and see what other people have done to learn from. As a community we are still very far from coming to a consensus on what a good harness looks like and the only way, IMO, to get a good feel for it is to write your own.
Hacker news generally follows trends, and this is the current trend.
The discussion around coding agents nowadays is steering towards harnesses (which is probably a better description of what this is). "Agent" here is doing a lot of heavy lifting and has become a bit of a catch-all term to describe a model + harness + tooling + prompt + some other things that I've probably not thought about. The harness is a part that's being explored more as many believe it's where we can get some better performance out of the models.
This one in particular is from Vercel who provide a service to use models, so they have a vested interest in providing a harness.
Because we're actively exploring the best way to remove any need to deal with code, and make it so that you don't need any real talent to make a computer do things for anyone.
We haven't quite hit on the right formula yet, but people are very excited by the possibility.
It's a brand new type of software. Nobody knows what the best way to do it is so a lot of people are trying stuff out, and a lot of people are interested in new ideas.
When a new technique or capability arrives on the tech scene, there's a point in the invention-to-diffusion story when the new thing becomes accessible (e.g. cheap and/or easy) enough for a broader audience of developers to experiment with it... but before anyone's figured out best practices, let alone polished products/projects, or calcified around a market leader.
So you get a Cambrian explosion of weird little projects. Ultimately, one of them will probably become the "market" leader... or at least the market default.
Right now there's a lot of agent harnesses and sandbox projects floating about.
Fun examples from the past: text editors, window managers, IRC clients, blogging engines (first static, then dynamic, then static again), Twitter clients... every programming language community has weird clusters of library/framework duplication in their history...
Sometimes these projects take on a rite of passage flavour... like, as every Jedi builds their own lightsaber, every developer builds their own... blog? That used to be the obvious one. Less so these days.
> Why there are so many new coding agents, and why they're commonly upvoted on HackerNews? It seems like there's a new one in the top 10 every other day.
It is widely known that upvote rings happen on this site.
> Please don't sneer, including at the rest of the community.
> Please don't post insinuations about astroturfing, shilling, brigading, foreign agents, and the like. It degrades discussion and is usually mistaken. If you're worried about abuse, email [email protected] and we'll look at the data.
It's a delicate mix of providing good system prompts, tools, workflows for agents, extensibility etc. I've used several and have yet to find the one that fits exactly how I want to work.
reply