To my understanding, LLM, by design, is unable to encode negation semantics. Neither negation "operation", nor any other "subtractive" operations are computable in LLM machinery. Thinking out loud, in your example the "Read code" and "Form hypothesis" seem to be useful instructions for what you want, while "Do not write any code" and "Not to fix the bug" might actually be misleading for the model. Intuitively (in human terms) one would imagine that, when given such "instruction", LLM would be repelled from latent-space region associated with "write any code" or "fix the bug". But in reality LLM cannot be "repelled", it is just attracted to the region associated with full, negated "DO NOT <xxxx>". And this region probably either has a significant overlap with the former ("DO <xxx>") or even includes it wholesale. This may explain why it sometimes seems to "work" as intended, albeit accidentally. My 2c.
Agree. Your post mirrors surprisingly well with mine of last year [0] - did you, by chance, happened to come across it? I said "you’d likely get approximately the same quality of candidates from a randomly selected 50 out of 5,000". And the main idea I was trying to convey is parallel to yours too: companies, consciously or not, try to find "the best candidate", while using completely inadequate "hiring funnel" approach. If that is their genuine need, they would be better off with "headhunting" approach. And the whole industry, businesses and candidates alike, would be better off if businesses recognize that the difference between "best" and "second best", or even "third best" is not meaningful (outside of C-suite) and not worth exponentially higher spend.
- [0] https://news.ycombinator.com/item?id=45271736 (believe it or not, em-dashes are all mine, even though I regret now putting my original draft through LLM - even if it was for grammar-check only)
I am in agreement with you, but regret that you missed an opportunity to swap two paragraphs around and purposefully mislabel them (i.e. the LLM-generated as your own, and vice versa). I'd be very curious if audience here would successfully pick it up!
I think you are mistaken. I remember landing upon that website some time before 2023 and being greatly impressed by that very 3D parallax effect. To my knowledge there were no AI tools available in 2022 capable of producing effects like that. There are barely now.
To my understanding it was made possible by another project of the same author [0] - check the date. Also check this [1] - dated 2003.
Is it so hard to accept that humans are more capable of doing intricate and impressive work than any LLM will ever be?
Fictional artist Feofan Kopytto, who was immortalized in "The Little Golden Calf" [0] used oats and other cereals for his paintings. On the back of the book authors' talent it became customary to refer to his artistic endeavours as "charlatanism". Having internalized it through my Soviet upbringing, I struggle to see why the same wouldn't apply to the art being discussed. LLM kindly helped me to generate hypothetical ad copy for Kopytto in the same style [1] - I honestly see no reason to not relate to both with the same reverence (or, rather, lack thereof!). I'd appreciate a human explanation (re: why?), if anyone has a minute or two. It would help me (and maybe others) to guide understanding why AI slop of all kinds may or may not deserve the same treatment as intent-driven human "output".
Interestingly, I think this HN topic is very relevant to understanding of contemporary LLM hype, as it illustrates the power of language (and propensity of human mind) to create an appearance of substance and meaning even where there is absolute emptiness (or, worse, manipulative fraud) underneath.
I am in agreement with many commenters here (https://news.ycombinator.com/item?id=47158240, https://news.ycombinator.com/item?id=47158573 and others) that this article is a clear illustration of failure on part of AI to capture the structure of material in a useful way. As addressed in the article, the effect is very visible in visual space, 3D modeling. I would argue it is very much present in LLM space too, just less prominent due to certain properties of the medium - text-based language. I also believe the effect is fundamental, rooted in the design of those models.
I'll leave here the note I've written down recently, while thinking about this fundamental limitation.
- The relationship between sentient/human thinking and its expression ("language") is similar to the one between abstract/"vector" image specification and its rendered form (which is necessarily pixel-based/rasterised)
- "Truly reasoning" system operates in the abstract/"vector" space, only "rendering" into "raster" space for communication purposes. Today's LLMs, by their natural design, operate entirely in the "raster" space of (linguistic) "tokens". But from outside point of view the two are indistiguishable, superficially.
- Today's LLMs is a brute force mechanism, made possible by availability of sheer computing power and ample training material.
- The whole premise of LLMs ("Large" and "Language" being load-bearing words here) is that they completely bypass the need to formalize the "vector" part, conceptualize in useful manner. I call it "raster-vector impedance".
- Even if not formalized, it can be said that internal "structures" that form within LLM somehow encode/capture ("isomorphic to" is the word I like to use) the semantics ("vector"). I believe the same can be said about "computer vision" ML systems which learn to classify images after being fed billions of them.
- However, I believe that, by nature, such internal encoding is necessarily incomplete and maybe even incorrect.
- Despite the above, LLM can still be a useful tool in many domains. I think language translation is a task that can be very successfully performed without necessarily "decoding" the emerging underlying structures. I.e. a sentence in source language can be mapped onto a region of latent space; an isomorphic region of latent space based on target language can be used to produce an output in the target language which will be representative of an equivalent meaning, from human perspective. All without explicit conceptual decoding of underlying token weight matrices. "Black-box" translation, so to speak. I am amazed (and disturbed, and horrified too!) that producing a viable code in a programming language from casual natural language prompt turned out to be a subset of general translation task, largely. Well, at least on lower levels.
- To me it is intuitive that such design (brute-force transforms of "rasterized" data instead of explicitly conceptualizing it into "vector" forms) is very limited and, essentially, a dead-end.
This is a very useful take, thank you. Really helped me to adjust my mental model without "antropomorphising" the machinery. Upvoted.
If I may, I would re-phrase/expand the last sentence of yours in a way that makes it even more useful for me, personally. Maybe it could help other people too. I think it is fair to say that in presence of hints like "Pretend you are X" or "Take a deeper look" the inference mechanism (driven by it's training weights, and now influenced by those hints via "attention math") is not "satisfied" until it pulls more relevant tokens into "working context" ("more" and "relevant" being modulated by the particular hint).
Upvoted, as it basically 99% matches my own thinking. Very well said. But I, personally, would not predict a breakthrough in this direction in the next 2-5 years, as there is no pathway from current LLM tech to "true reasoning". In my mental model LLM operates in "raster space" with "linguistic tokens" being "rasterization units". For "true reasoning" an AI entity has to operate fluently in "vector space", so to speak. LLM can somewhat simulate "reasoning" to a limited degree, and even that it only does with brute force - massive CPU/GPU/RAM resources, enormous amount of training data and giant working contexts. And still, that "simulation" is incomplete and unverifiable.
I would argue that the research needed to enable such "vector operation" is nowhere near the stage to come to fruition in the next decade. So, my prediction is, maybe, 20-50 years for this to happen, if not more.
Wow, sounds so familiar! I've once had to argue precisely against this very conclusion - "you saved us once in emergency, now you're bound to do it again".
Wrote to my management: "It is, by all means, great when a navigator is able to take over an incapacitated pilot and make an emergency landing, thus averting the fatality. But the conclusion shouldn't be that navigators expected to perform more landings or continue to be backup pilots. Neither it should be that we completely retrain navigators as pilots and vice versa. But if navigators are assigned some extra responsibility, it should be formally acknowledged by giving them appropriate training, tools and recognition. Otherwise many written-off airplanes and hospitalized personnel would ensue."
For all I know the only thing this writing might have contributed to was increased resentment by management.
reply