if patterns can be put into words, why wouldn't the same process happen at the word level? do you think it's impossible for a LLM to write even a simple python program that will generate millions of plausible random sentences in the general problem space we aim to generate new concepts for, then filter and rank them semantically?
More generally, do you really think humans randomly come up with new ideas, in any act of creation, such as art, science, etc.? Why do you think art studies and stuff like Oblique Strategies exist? How long should one wait for a specific random human without any formal education in physics to have 50% chance of coming up with a new workable theory of cosmology?
As you say, putting a random sequence of plausible-sounding words together is a trivial task - even a non-LLM algorithm can do it. No, I don't think humans are unique - they also absorb the ideas available to them and re-elaborate on them to produce novel ones.
An LLM might come up with a sequence of words that express a novel idea/thinking pattern, but so can I on the spot right now. The question, though, is: is that idea any good? In the case of humans, maybe the first idea is good, the second ok-ish, but inevitably they get duller.
Most people come up with very few novel (good) ideas in their lifetime.
The question, then, is whether a model can, unlike humans, *repeatedly* generate new ideas, continuously.
My guess/assumption is that NO, a model might come up with SOME ideas that are somewhat original [1], mixing the existing sources available to it (similarly to how a human would do it, just with more sources and more efficiently), but it cannot continuously generate novel ideas.
If that's true, I think there will be a space for humans to be thinkers, even if models are super smart, as a different unique perspective and a different reconfiguration of previous ideas will be more unique than raw intelligence.
Please note: I wasn't refusing the notion that LLMs don't produce anything original, just noting some fallacy in your example about word generation. The jury is still out on whether AI models will own thinking in the future - or will simply be partners to humans - and I don't pretend to have the answer.
[1] More should be said about what makes an idea original (even if derived from others), as opposed to a transposition of an existing concept. I admit I'm not a mathematician or anywhere near an expert on the topic, but from what I've read, the latest AI math achievements seem to belong to the latter case.
> I'm ready to bet that is untrue. Sure, no one can recite the code, but people can quickly scan the module and refresh their memory as well as understanding the new changes, even after a while. If not, collaboration would have been impossible.
I don't understand this part. When I was doing freelance consulting, my job was quite often "come to our office, scan this 40 million lines codebase and fix stuff for the next week". It was never a problem to read and get up to speed completely unknown codebases if you have basic programming ability. Much harder to do something that fits floaty business objectives though.
For one person that understands a proof of some matrix multiplication algorithm lowering its current complexity bound in an useful way (e.g. not with absurdly gigantic <= 2nd order polynomial term), there's likely thousands of companies willing to pay for the improved efficiency
And if anything that useful does come out of this output/"slop dump" then that might begin to make a strong argument for it. But every other time it was "that's the theorem done for the headline figuring out everything important and how to use it is an exercise for the reader".
Which language do you think doesn't need it? For instance for me I wish I had such a tool whenever I used any interpreted or JIT language, to understand clearly to which semantics whatever is happening maps and how their respective front-end optimizes. Things as simple as "a" + "b" in JavaScript: are you sure you know how every js runtime maps it? Do they fold the strings as an optimization during parsing ? Do they preallocate a larger native string type and push? Do they just do something that ends up an equivalent of std::u16string("a") + std:: u16string("b")?
> (most of the time it’s actually the comments that are the problem, not the code).
I wonder if it's just a consequence of a gigantic training set full of comments completely out-of-date with the code, leading to the model considering this "normal"
I don't usually see Claude leaving comments that are obviously out of date compared to the code. The problem is that the comments are just a dumping grounds for Claude's stream of consciousness, verbosely recording random bits of history and context that are true and at least somewhat relevant, but without cohesively explaining why the code is the way it is. And some sentences in the comments will be beginner-style restating what's obvious from reading the code.
Exactly. Comments are basically its scratchpad for whatever it wants. Ask it to change a number in a TOML config and it'll also add 2 new comment lines above it with some Claudeslop about it being changed and what that accomplishes, as a useless note to itself.
I now make sure to do a big decommenting pass before every PR.
This kind of stuff annoys me too, and I have instructions in agent.md to avoid it, which helps.
But I also am starting to just let go and stop caring. It’s not clear to me that it causes problems down the road, it’s easy to strip out en-masse if needed, and in my experience, agents now are really good at read git blame, the commit log, even prior agent transcripts if available to sleuth out when a change was made and why. So yeah, it’s annoying, but the code agents write for me is increasingly never read by a human, so does it matter?
This. Be honest: who reads the human-written comments? ;)
When I read, I skip most comments, especially the larger "Javadoc" style. My brain sees them colored differently in the editor and it doesn't even take mental effort. Then, when I have a question about the code, I look back up for a relevant comment. That doesn't happen very often.
If Claude writes great code and leaves a garbled Claudese-but-accurate comment ... I can read and comprehend (with like 5x the effort of a human comment) ... that's a small price to pay.
Yes, comments might be my biggest gripe with CC due to what you said. 9 out of 10 of my revisions to Claude's work is deleting or rewriting comments. Such a silly thing for it to fail on.
(I do have detailed instructions for it on how to comment (or not) but it has not fixed this.)
Nah, it’s the typical Claude-isms, and way too much info, info in the wrong places, and putting lots of planning discussions into comments that are completely irrelevant.
It’s always “you explain only what but not why” or “this is way too much prose” or “these comments don’t belong here, they should be inline comments” or “this is completely redundant as it’s already obvious from the code”.
I do find that once I beat it into submission and the codebase is “clean”, the new code it generates gets better and better, which makes sense gives its pattern-prediction nature. But it seems like there is work to do for Anthropic in terms of getting Claude to not confuse code comments with dumping its interactive discussion state into there.
> The credits for my code are protected (with the GPL you have to tell where your code originates from)
But that's literally anathema to the spirit of GPL. Copyleft exists only as a reaction to copyright which is sadly ingrained in legal systems, but the original thought about free software, at the time of GPL inception, is that in an ideal world, copyright shouldn't exist for software ; it's leveraged by GPL only to protect against abuse of copyright holders that could close open code, which is thus made impossible "legally" with the GPL. AI makes this distinction fal into "practically" as pretty much anything is "open" for individual use now (e.g the only use that matters).
Attribution and copyright are separate. Who says that, even if copyright didn't exist for software, attribution also wouldn't?
If anything, the remark that I have to make to the parent is that in principle attribution is also required by non-copyleft licenses. However, I doubt it's respected for the hundreds of crates or npm modules in a typical Rust or JavaScript project...
Even the true copyright abolitionists (who want companies like Oracle to be able to fork their software and make a billion dollars releasing the binaries) still use MIT license which requires attribution.
I'm a copyright abolitionist, but I think Oracle should be required to release the source code with their binaries as a basic consumer protection. I also don't care about attribution.
> In both photoshop and painting, the human using the tool decides what strokes to use.
except for the million posters and album covers unironically made with a real photo passed into one of photoshop's "Artistic" filters to make it look like some impressionist painting
Well what's better ? I have so many pictures of posters of old events or storefronts saved on my phone because they were so astronomically bad it'd have been better to not have any poster at all. Today's AI makes it look even worse but I'm hopeful that in five years we'll all live in a world generally more agreeable to look at and not "concrete with pink and yellow fluo paper smashed on top of it" like https://www.alamy.com/stock-photo/music-poster-wall.html
or
https://www.alamy.com/stock-photo/france-poster-wall.html?ps...
because the cost of making things enjoyable with more colour and detail will have come down. Maybe we can finally go back to Art Nouveau if the cost of making it comes down!
As I said, "Today's AI makes it look even worse". But the post makes me hopeful than in a couple years, actually good graphic design can come out from AI. It still looks terrible and AI even for the best posters it highlights, but noticeably less than the random ice cream or coffee shop where I live with fully ai-generated interior design, memorabilia and menus.
More generally, do you really think humans randomly come up with new ideas, in any act of creation, such as art, science, etc.? Why do you think art studies and stuff like Oblique Strategies exist? How long should one wait for a specific random human without any formal education in physics to have 50% chance of coming up with a new workable theory of cosmology?
reply