I'm very skeptical of that graph. What does it mean for code to be AI-generated vs human-generated?

If I merge a PR that was opened by Claude do all the changes in it automatically count as AI-generated? How about the individual commits? If I'm reviewing locally and make manual changes but then have Claude create the commit it appears to be AI-generated but might not have been.

The point is that these metrics are easy to game and I've definitely wasted time and tokens refining code with Claude that would have been easier to just edit by hand. It can be kind of fun and when the goal is just "use AI" I don't find it surprising that graphs showing 100% switch to AI-generated code could be defensibly generated without really saying anything about how much manual intervention is happening or how efficient the process is.

One of my least favourite feature of our AI era is that I can't even trust people to do basic math anymore. It used to be that a graph like that would mean someone had a (probably broken) metric that they put in an excel sheet and got a lineplot out of it. The metric might be broken but at least additions and subtractions etc were correct.

Now I don't even know! Colleagues send me graphs they made with Copilot and then we discover that the LLM did mental arithmetic or whatever to produce the results and they are wrong. Computers used to be able to do math !

The crazy thing is how cheap it is to spin up a cloud server, but how little actual processing power you're getting for your money. Even if the business manages to keep their cloud spend in check it's probably going to be at the expense of a terrible user experience because everything in the web app will feel slow and all that matters is individual P95 request latency is within some arbitrary number of seconds.

Isn't it also true that the deeper and thicker the quagmire, the more tokens one will have to use to wade through it?

This seems like a path to eventual LLM lock-in once the codebase gets messy enough. These things could end up being like 0% interest credit cards for technical debt. I guess it all depends on how the token usage scales over time. My guess is it will be steeper than linear.

I don't think there are good analogies to physical tools. It would be something like a nondeterministic version of a replicator from Star Trek which to me would feel much closer to a slot machine than a CNC mill.

Lots of hair splitting in the comments. The service is so unreliable at this point that I don’t trust them to not train on private repos even accidentally. You’re one vibe-coded PR away from having all your data scooped up regardless of any policy or intention.

I can't even imagine what these exams would look like. The entire profession seems to boil down to making the appropriate tradeoffs for your specific application in your specific domain using your specific tech stack. There's almost nothing that you always should or shouldn't do.

All engineering professions are like that. NCEES has been licensing Professional Engineers for over a hundred years. The only thing stopping CS/SE is an unwillingness to submit to anything resembling oversight.

Results over time are important. Or at least they should be.

Great read: https://www.newyorker.com/tech/annals-of-technology/chatgpt-...

Interested to see how this plays out. ESPN is the only reason I subscribe to YouTubeTV.

interesting, isn't there cheaper ways to watch ESPN? I have youtube tv and i could care less if they ever bring it back...tho i do want monday night nfl games

Exactly! All LLMs do is “hallucinate”. Sometimes the output happens to be right, same as a broken clock.

I find this way of looking at LLMs to be odd. Surely we all are aware that AI has always been probabilistic in nature. Very few people seem to go around talking about how their binary classifier is always hallucinating, but just sometimes happens to be right.

Just like every other form of ML we've come up with, LLMs are imperfect. They get things wrong. This is more of an indictment of yeeting a pure AI chat interface in front of a consumer than it is an indictment of the underlying technology itself. LLMs are incredibly good at doing some things. They are less good at other things.

There are ways to use them effectively, and there are bad ways to use them. Just like every other tool.

The problem is they are being sold as everything solutions. Never write code / google search / talk to a lawyer / talk to a human / be lonely again, all here, under one roof. If LLM marketing was staying in its lane as a creator of convincing text we'd be fine.

This happens with every hype cycle. Some people fully buy into the most extreme of the hype, and other people reverse polarize against that. The first group ends up offsides because nothing is ever as good as the hype, but the second group often misses the forest for the trees.

There's no shortcut to figuring out what the truth of what a new technology is actually useful for. It's very rarely the case that either "everything" or "nothing" is the truth.

I think a lot of problems will be solved by explicitly training on high quality content and probably injecting some expert knowledge in addition

Yeah but that's not easy, which is why it wasn't done in any of the cases where it's needed.

>I find this way of looking at LLMs to be odd.

It's not about it being perfect or not. It's about how they come about with the responses they do.

>Very few people seem to go around talking about how their binary classifier is always hallucinating, but just sometimes happens to be right.

Yeah, but no one is anthropomorphizing binary classifiers.

You imply that, like a stopped clock, LLMs are only right occasionally and randomly. Which is just nonsense.

Although I get what you're saying, it's still true that if something is wrong randomly at any point, it is always "randomly wrong".

Sure, but that chance might be 99.7%. 'Random' isn't a pejorative.

Same is true of humans fwiw.