This is purely anecdotal, and I try to keep it to myself but its very difficult when at least half of the HN homepage is AI related: LLMs like ChatGPT do so utterly terribly at any non-trivial job I throw at it that I seriously consider people who use it daily to either be straight up incompetent, or maybe their domain is so trivial that the LLM actually does well.
From asking LLMs to solve a highly difficult async C++ parallelism problem, to german language specifics, it just fucks up at a fundamental level. I understand that LLMs cannot solve these issues and why, but then I do not understand the heavy focus on AI by so many tech people.
Is day to day programming job so trivial that LLMs do a good job, while at the same time being too difficult for you to do it yourself? I really, really want to know exactly what the use case is.
Do people just throw simple problems at it to validate their own preconceived notion of how cool and useful LLMs are? Whats the deal?
I had a similar take until about a week ago. A friend showed me his workflow with Copilot and whatever Jetbrains AI assistant is.
Use it as a tool: what if instead of opening up a new tab, searching for the API docs for the library you're trying to find a function in, find the function, re-read the parameter arguments for the 400th time, and then use it, you could just highlight a snippet and say "Paginate the results from S3 using boto3" and the code would just populate?
You have to have the clarity of thought to know what you're doing, but the time it takes to write every line for basic stuff you've done 1000x before can be greatly compressed if it's inlined with your IDE.
I think this is the move for most LLM tools: integrate it with existing tooling. An LLM for Excel for corporate bookkeepers, CPAs, etc will be great. A Word/PDF summarizer that's tuned for attorneys will also be fantastic. Highlight a paragraph, ask for relevant case law, etc.
I thought ~2 years ago the results were... not great. Now I'm pretty happy with it.
SecureFrame (helps with compliance regimes like SOC2) recently added the ability to generate Terraform templates to automatically generate infrastructure that will fix specific platform risks for AWS, Azure, GCP, etc.
It definitely needs someone at the helm since it does hallucinate, but I have found it to cut down my time on mundane tasks or otherwise niche/annoying problems. When was the last time you visited 4+ StackOverflow posts to find your answer? Copilot, so far, has always hit a pretty close answer very quickly.
I also had to build intuition for when it will be appropriate versus not. It's hard to describe but one very positive signal is certainly "will any hallucination be caught in <30s"? Even in ChatGPT Plus you can have it write its own unit tests and run them in the original prompt (even in the profile's Custom Instructions so you don't have to type it all the time).
So a mistake was using it for something where runtime performance on dozens of quirky data files was critical; that nearly set my CPU on fire. But str>str data cleanup, chain of simple API calls, or some a one-off data visualization? chef kiss
> to write every line for basic stuff you've done 1000x before
There are ways to avoid writing basic stuff you've done 1000x before that are better than LLMs though...
Put it in a well-thought-out function or package or other form of shared/reusable code. You can validate it, spend the time to make sure it covers your edge cases, optimize it, test it, etc. so that when you go to reuse it you can have confidence it will reliably do what you need it to do. LLM-generated code doesn't have that.
(When you think about how LLMs are trained and work, you realize they are actually just another form of code reuse, but one where there are various transformations to the original code that may or may not be correct.)
Where LLMs shine for coding is in code-completion. You get the LLM output in little chunks that you can immediately review correctly and completely, in the moment: "yeah that's what I want" or "no, that's no good" or "ok, I can work with that". Not surprising, since predicting completion is what LLMs actually do.
I don't know exactly how you use it, but this isn't my experience at all. If you ask a LLM anything too specific, that isn't obvious and a common issue/discussion ( something that I almost never need to do), it just makes up nonsense to fill the space.
Equally, if you ask it general questions it misses information and is almost always incomplete, leaving out slightly more obscure elements. Again, I need comprehensive answers, I can come up with incomplete ones myself.
What's really obvious to me when I use it is that it's a LLM trained on pre-existing text, that really comes through in the character of its answers and its errors.
I've very glad others find them useful and productive, but for me they're disappointing given how I want to use them.
That's fair, it might not be for you. In 'old school ML', for a binary classifier, there's the concept of Precision (% of Predicted Positive that's ACTUALLY Positive) and Recall (% of ACTUALLY Positive that's Predicted to be Positive).
It sounds like you want perfect Precision (no errors on specific Qs) and perfect Recall (comprehensive on general Qs). You're right that no model of any type has ever achieved that on any large real-world data, so if that's truly the threshold for useful in your use cases, they won't make sense.
I just want something useful. I'm not talking perfection, I'm talking about answers which are not fit for purpose. 80% of the time the answers are just not useful.
How are you supposed to use LLMs if the answers they give are not salvageable with less work than answering the question yourself using search?
Again, for some people it might be fine, for technical work, LLMs don't seem to cut it.
Sorry if this is sophmoric, but when you said "you have to have clarity of thought" - what jumped to mind was the phrase "you have to speak to the code"... I thought it encapsulated your clarity of thought quite saliently for me.
Stop using it for things that are in you area of expertise but are too difficult for you. Use if for things where you think "this is probably easy but I have no idea how to do it". For example, I needed to do some pretty trivial task in powershell but I have never used it so I got chatGPT to do it for me and it worked first time. Obviously I checked the commands looked plausible before I ran them, but it still probably took 2 mins to do something that would have otherwise taken 30.
> Use if for things where you think "this is probably easy but I have no idea how to do it"
I had exactly the same reaction as OP (LLM's suck what's with the all the hype). These people are using it differently. For me it's often something like, asking it to put together a specific sequence of matrix transformations in ThreeJS or some other library.
This is not a difficult task but it's often one I waste a lot of time getting right. It's sort of about finding the right level of abstraction you need to ask it.
That just means you are ignorant of how wrong it guides you. You need to first build trust before taking it new places. You do that with topics and concepts you are familiar with.
This has always been true of anything anyone has ever googled or looked up on stackoverflow
I copy paste code from stackoverflow all the time. I used to agonize over making sure I fully understand every line it's copying. Now I have the discretion of making that decision: sometimes it does really matter, sometimes all you need to know is that it produces the right result for your limited use & test case of it. (it's no different than relying on a 3rd party library in that way)
I think we need to apply the same discretion to LLM output. The answer "it depends". Sometimes using its output blindly leads to disaster. Sometimes using it without fully understanding all the details is a great way to make progress.
This is no different from my coworker who regularly copy/pastes from stackoverflow to do things he doesn't have any idea how to do himself, and just as awful, unproductive, and problem inducing.
This is an observation I've seen a lot around here. Underneath it is the assumption that "if I can't figure out how to get meaningful use out of a tool, the tool must be useless".
OpenAI didn't sign up 100M users without somebody somewhere finding it to be useful. Like any other tool, it's utility is limited mostly by the person wielding it.
The tools seem useful, but I'm not sure they are. too often they will confidently make up an answer that is wrong. When I use them they do great on trivial problems but can't help on hard ones.
Reframe your thinking. You’re approaching it like other computer systems, where a given input yields a determined output. Instead, treat it like a junior dev whom you can unload an unlimited amount of work to, but the result still requires review.
We’re all used to working this way in human systems, people that sound confident might also be wrong, and you learn where you might trust them more or less as you work with them over time. Until you are confident that they are always “right” in a given problem domain, you need to apply some level of review.
Finally, keep in mind that there are "smarter" and "dumber" LLMs. If you didn't pay for what you were doing, you were talking to a "dumber" model. The quality does go up if you have $20 in your pocket.
The junior engineers I know tend to ask questions not be confidently wrong. That isn't to say they are always right but they make a very different class of errors.
Again, this is a tool you can use. You can complain that it doesn't work in the way you expect, or you can learn how it operates and how best to use it. If you can't figure out how to apply it to your work, that's fine, but loads of other people are doing exactly that with or without you.
> When I use them they do great on trivial problems but can't help on hard ones.
That sounds super useful! The tools free you up from wasting time on trivial problems so you have more time to focus on the hard ones. What's not to love?
Does your job involve solving complex, challenging problems all the time?
I am a CS professor, I don't think most people would class that as a trivial job, but I find myself needing to do plenty of trivial tasks every day: mixed bureaucracy (periodic reports, grant requests, various evaluations, etc.), trivial programming (a Seaborn chart to show some Excel results), text polishing (need to cut a text to 500 words without altering meaning), writing student assignments, writing emails in (non-Native) English for sensitive requests with the right tone, etc... all of those are things I have found LLMs to do fairly well and save me a lot of time.
I wouldn't use them to do the core job of designing novel algorithms, doing experiments, writing the bulk of a paper or teaching students. But most of my working hours are not really that "core" stuff. And I would assume it's the same for most professionals.
If you have an environment where you are constantly challenged by difficult tasks... wow. I don't know if I should envy you (because I love difficult problems and hate mindless chores) or it would be too stressful.
PS: I don't think "being too difficult for you to do it yourself" is the right litmus test for LLM usefulness. I can draw charts with Seaborn, of course. But the LLM does it much faster, and I don't think doing it myself would make me grow, hone useful skills or anything. I'd rather devote my time to something else. So (in my view) it's clearly better to have the LLM do it.
They're good autocomplete, they can help search for solutions sometimes better than Google (SEO spam), you can use it as a rubber duck, and you can make it auto fill trivial stuff that would take you a few minutes to write out manually, like test scaffolding. I would never use it to actually complete a non-trivial task and I always confirm it's answers. And yeah, sometimes it sucks - it's a tool with a learning curve about knowing it's limitations.
The reason there's so much money and time is that even semi-competant AI is relatively new and the methods are still extreme crude, and yet it's this advanced. This seems like the path to an AGI, and if someone were to even approach that point, it would radically change the world forever and could lead to either really good things or really bad things.
Now, GPT-4 isn't considered the best at specialized tasks. It's a master of many, but there are much smaller models that can do things like incredibly complex symbolic/geometric math proofs, write code, perform translations, etc better. A lot of ideas are on making expert systems using many of those specialists combined with a generalist, like the segmentation of a brain.
Anyway:
> I seriously consider people who use it daily to either be straight up incompetent, or maybe their domain is so trivial that the LLM actually does well.
These kinds of radical lines of thinking about a significant proportion of enthused professionals (in any industry) who aren't showing the same experience as you, is a red flag for introspection. It's so easy to fall into the "enlightened me" trap.
There are plenty of jobs where people have to complete various tasks that are outside of their domain or otherwise tedious on a daily basis. For example, plenty of devs have to set up or change the configuration of remote hosts. Some LLMs are pretty good at generating devops scripts to speed up this work.
Exactly. Example: maybe 1% of the code I generate is bash. I used to try to memorize patterns, but of the top 20 I'd use each less than once per year. Now, instead of that 1% taking 5% of my time, it takes 2%. It's all "simple stuff", and I can verify it instantly.
I have ~10 similar use cases. So it hasn't revolutionized my life, but it's been well worth $20/mo ChatGPT Plus and $3/mo API calls.
Even if half of it is garbage it’s a net win. At least in domains where I can distinguish the two.
There are also cases where the cost of failure is very low. Eg
I could spend half an hour reading an api spec or I could make an AI give me a curl command and test it out in 30 seconds. If it works great if not oh well time to read spec
I signed up for Open.AI’s monthly subscription. Its performance on non-trivial tasks is abysmal. It’s a regurgitation machine. One might mischievously argue the average tech worker isn’t much better than an LLM, thus the interest? On a related note, we are deluged daily with firms offering AI services. I see a bubble.
You should treat LLMs the same way you treat any other smart entity, human or otherwise: realize that they can be both immensely useful and fundamentally wrong at the same time. Intelligence is not equivalent to correctness.
Why do you presume that people commonly use it for non-trivial things? It excels at trivial things. That's what most people use it for, probably. Like google search. Is there something that leads you to think otherwise?
And when i do a little bit of go coding once a week (i'm a java developer by trade), i don't have the time to learn go well enough to just type stuff down without looking things up. Instead of googling, i tell it "I need a struct with the following attributes..." and it doesn't just ell me how i do structs in go, it also creates them for me.
Also: There are a TON of issues were i would write a short script to do something (formatting text into a table, searching for specific lines etc.) were a normal person doesn't even have those tools at hand.
For companies overall: Its not just what an LLM can do, LLM can do things for you but its also a very very good interface to your application. The demos i saw in my company are really good and totally make sense and do reduce the entry barrier for people.
I know a friend whos job is to create reports with sql. She doesn't do anything else just reports across the whole datawarehouse. Why? Because every normal non dev person can't just write SQL or automate things.
The gap between tech people and management is huge.
Technology is complex and hard to make sense of. That is why most non-experts have a strong wish for a kind of mythical technology, which you can just pour onto your problem and it magically knows what you wanted (and which things you did not want).
For a certain class of problems LLMs achieved new, never before seen, almost magical results. Now imagine you were someone who hates dealing with the constant complexity of solving problems with technology and something comes along that seems to carry promise of lifting that off your shoulders. Then you know why people react like they do. Recall the block-chain-craze? There were people who declared that this somehow magically solved any IT-security problem there ever was – instead of seeing it as a good solution for a very specific set of circumstances, nearly nobody faced in practise.
In reality of course also LLMs have limitiations, e.g. above mentioned ambiguity that is inherent to any magical technology: To be true magic the technology would have to be able to read the thoughts of those who apply it and somehow infer from that the true thing they want or need. Now LLMs are in the end still just very good guesses based on statistical data, that means the guess could just be what you want, but it lacks an actual understanding of what it is doing.
Those applying the technology for things it is actually good at (e.g. classification problems etc) will put it to good use, but there will be a lot who will apply it and have things fall apart Canada Airlines style.
Your view on LLM usage is too narrow. Yes, they are pretty shit for me too in solving coding problems, but they are still useful for bespoke information extraction, classification and creative applications. The interest is justified, we're just having a hard time understanding the limitations.
1. having ChatGPT generate boilerplate, because I’m lazy;
2. having ChatGPT attempt something I don’t know as a starting point, eg JavaScript; or,
3. having ChatGPT give a reference rather than Google myself, eg of a config option.
ChatGPT makes 1 less tedious, 3 less a game of “what magic phrase finds the right SO post?”, and means I do 2 at all, eg trying out JS features on my blog.
I think it does alright at composition if you break down the task sufficiently, but it struggles with higher order structure — particularly if you’re using multiple responses.
That said, I suspect we need a theory shift to get AI to comprehend higher order structure in composition.
When you say ChatGPT, are you referring to GPT4? I find a huge and avoidable miscommunication happens when two people both think they are using “ChatGPT” but talking about two different models which vary in size by a factor of 10.
Assuming you are talking about GPT4, for the sake of argument, the answer is speed. Of course I can write a small parser script that deals with some data I received from a client. It will take me an hour and be a tedious task far distant from my actual expertise. An LLM can do it in 45 seconds, including the time it took me to describe the task.
I daily drive KDB/Q. This is readily extendable for example in C, which was my previous daily, and Python which I use sporadically.
I don’t use LLMs for C or KDB, I do use them for Python.
ChatGPT is good in Python. I guess as Python programmers rely on stack exchange so there is lots to learn from, and Python anyway is largely an exercise in finding the correct library.
If the only thing ChatGPT did was listen to my problem and suggest which imports to use/manuals to read, that would be good enough to use regularly. If I wasn’t after a library/pre existing code I wouldn’t be using Python!
I've definitely noticed ChatGPT generally writes better Python than it writes Scala, presumably for the same reason of there being a fair bit more Python code in the wild.
The actual reason probably has to do with the fact that LLM developers and academics are more familiar with Python than other programming languages, and therefore have policed it's correctness better.
Boilerplate, test code, and general tedium. Most software just needs to handle IO.
The next time you want to use SQL to compute a rolling sum try asking ChatGPT 4 instead of searching through documentation or search engine results for windowing functions.
Competency at programming along with very good technical communication skills (with a touch of learning how to not hold the tool backwards) and you should find the appeal.
Yes, the free one. I find it almost a fallacy to say "oh, yeah it sucks, but you should try the expensive one! Its proper good."
Oh, your BMW keeps having issues? You should have bought the one thats 2x the price, that one is perfect!
Of course its better, but both are being sold as a car, or, in this case, an LLM. Theyre both LLMs, theyre both by the leading AI company, if one sucks such supreme ass, why would the other one be amazing? ;)
We're not in control here and we didn't build them, they just happened. Every LLM is completely different and some of them are bad. You can't generalize from one of them at all.
Every other query I've given to ChatGPT came up with an utterly wrong answer. Followup always yielded "sorry, I made an obvious mistake, here's another wrong answer". Confident and stupid is a very bad combination.
From asking LLMs to solve a highly difficult async C++ parallelism problem, to german language specifics, it just fucks up at a fundamental level. I understand that LLMs cannot solve these issues and why, but then I do not understand the heavy focus on AI by so many tech people.
Is day to day programming job so trivial that LLMs do a good job, while at the same time being too difficult for you to do it yourself? I really, really want to know exactly what the use case is.
Do people just throw simple problems at it to validate their own preconceived notion of how cool and useful LLMs are? Whats the deal?