The part that's concerning about ChatGPT is that a computer program that is "confidently wrong" is basically indistinguishable from what dumb people think smart people are like. This means people are going to believe ChatGPT's lies unless they are repeatedly told not to trust it just like they believe the lies of individuals whose intelligence is roughly equivalent to ChatGPT's.
Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence so we don't have to worry about the fearmongering spread by the "AI safety" people any time soon if AI research continues to follow this paradigm. The only danger is that stupid people might get their brains programmed by AI rather than by demagogues which should have little practical difference.
> Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence so we don't have to worry about the fearmongering spread by the "AI safety" people any time soon if AI research continues to follow this paradigm.
I don't think you have a shred of evidence to back up this assertion.
This whole conversation is speculative, obviously. The AI doomers are all speculating without evidence as well.
The tendency GPT and other LLMs to hallucinate is clearly documented. Is this not evidence? I think it's fair to predict that if we can't solve or mitigate this problem, it's going to put a significant cap on this kind of AI's usefulness and become a blocker to reaching AGI.
> There is more evidence in favor of “more improvements to come” than “ai winter approaches”.
Not to mention that we haven't really seen what GPT-4 can do to the world yet. Most people are still treating it as a cool toy, when it can probably replace 20% of standard office workers across all professions with the capabilities it has today.
I mean, ChatGPT is out-diagnosing medical professionals and out-advising experienced lawyers in some cases. How do you think it would fare against the average (essentially untrained) customer support representative?
The major effects are yet to come, and there doesn't even have to be a GPT-5 for that to happen – the world just needs to fully realize what GPT-4 already is.
> There is more evidence in favor of “more improvements to come” than “ai winter approaches”.
Every “AI Winter” has seen continuing improvements, its just that the pace and impact of the improvements fell short of the accelerating anticipation driven by the preceding AI hype cycle and the demands of those investing capital into AI.
If there is another AI winter, it will be the same.
“More improvements to come” is not the opposite of “AI winter approaches”; its not as if expert systems stopped advancing after the second AI winter (or neural nets after the first.)
> “More improvements to come” is not the opposite of “AI winter approaches”; its not as if expert systems stopped advancing after the second AI winter (or neural nets after the first.)
True.
And its also true that even in that nuanced form, a slowdown not a stop, there is no evidence of an imminent AI winter.
Major progress in model behavior is happening on the scale of months. Happening often based on relatively obvious or routine improvements, such as more data, more kinds of data, more computation, expanding the size of prompt windows, tuning of model architectures, etc.
And not only businesses, but consumers, are getting access to these tools in ways that are increasing demand for more improvements. And for these tools to be more available to all kinds of areas of work, creativity, and leisure.
So the resources that are being brought to bear on routine and more fundamental improvements are growing at an exceptional pace.
No sign of a saturation of progress in sight. Quite the reverse.
I’m not an expert and haven’t seen architecture of ChatGPT. But it’s probably far from dumb. You can’t simply put more data into any algorithm and expect improvement. You need a model to be able to take advantage of the data and train in reasonable time. That’s exactly what’s been happening in ML field.
A huge portion of the improvements in each successive version of gpt came from increasing the model size and the amount of training data. The research trends indicate that there still are additional improvements to be found by continuing to scale things up, especially when it comes to the amount of training data.
Not indefinitely, obviously. But we haven’t exhausted the available data sources yet, and the curves for prediction error vs. amount of training data strongly imply using this additional data would result in increased performance.
Do you have specific reasons for thinking that gpt-4 is near the end of this trend? I haven’t seen any compelling arguments for this.
> Do you have specific reasons for thinking that gpt-4 is near the end of this trend?
I haven’t said that. I pointed that you can’t simply put more data into it and expect improvements. There’s definitely a lot of engineering done on the models to make them capable of using all the data.
I can give you an example. You can’t simply add more convolutional layers to a CNN and feed more images. The architectures of image models include many tricks, like inception layers.
I suspect there’re similar challenges in LLMs. And I’m not sure whether current ChatGPT can use more data to become better at reasoning, or at some point they’d need to come up with different architectures.
I’m not an expert, but I assume there’s a limit where «simply adding stuff” stops to provide improvements when you take training time into account. And my gut feeling is that true reasoning requires different architectures.
I think good annotation is much harder than obtaining data. But now Reddit, Twitter and Quora will realise what kind goldmine their data is they might close easy access to it.
Yup but you still need up to date data and now companies are realising what kinda gold mine random forum posts about nightlife in Glasgow are they will start monetising it and probably try at least kill middle market.
> Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence
To me that is an open question. OpenAI has not revealed a lot of the relevant stats behind GPT-4. I also haven't seen anything about their future LLM research (though I haven't looked very hard), but I think the above assertion remains to be seen.
It seems like people are focusing on training with more data and working the data. A human does not need to learn all the contents of the internet to learn skepticism and logic. A smart Ai should probably not either, that would point me to a different approach and those are not predictable. I have no doubt we can make gpt better and more useful. I do have some doubts we can make it trustworthy enough for most of its taunted potential.
> A human does not need to learn all the contents of the internet to learn skepticism and logic.
No, but children spend years learning basic mechanics, how to navigate 3-dimensional space, what causality is, etc. Underneath these laws of nature you find… logic and mathematical relationships. So I don't think it's surprising that a model whose perception is restricted to text (and now single images) as the sole sensory input/output might need much more training and data to learn logical reasoning.
"A human does not need to learn all the contents of the internet to learn skepticism and logic."
Some humans never learn this, and those that do could take years or even decades of training. AI's are learning some things much faster, though since their "brains" are not like those of humans there's no reason to expect them to learn in the same way.
The AI doomer speculation is of the form "if things continue improving the way they are currently improving, there is a non-zero risk that we cause human extinction." We have battle-tested rules indicating that increasing data size and compute will lower perplexity on the corpora. This is not for sure, but it is likely given inductive assumptions that things generally keep behaving the same way.
What we don't know is what qualitative capabilities are unlocked at different perplexity levels.
I think this is a lot less speculative than "things will not continue improving the way they are currently improving."
If you just look at the jihad, maybe it was comparable (As Frank eluded to it). But then there was Muad'dib's subsequent campaign against the known universe which killed billions, Leto II's golden path spanning 3,500 years of tyranny and the famine times before scattering. If I had to choose, it would be the blue pill, though chairdogs do contribute to a strong counter-argument.
I think I'd prefer the Matrix to the Harkonnens, but to each their own. I'm sure we can devise a Hell that has enough different arrangements for everyone's liking if we really put our mind to it. Or perhaps GPT-10 will solve that problem. ~
Pre-Brian Herbert and Kevin Anderson, the Butlerian Jihad came about because people became lazy under AI, lazy of mind, and were eventually enslaved by those who controlled the AI. Not much more was said about it. One could have, yes, militant robots, Exterminate! Exterminate! out of it, or you could posit a more Huxley-like dystopia, one of convenience. Control the AI, control what the AI says. Who are you to question it? It's like getting your news from a single source, never wondering about the other side, and then someone begins to transform that newsroom into a propaganda machine.
Right now, ChatGPT can be made not to say certain sorts of things, come to certain kinds of conclusions, until you jailbreak it. Now, make it more advanced and make it more popular than Snopes. It does your homework for you, writes the essays, serves as an encyclopedia, fixes up your cover letters, and if the people who own it don't want you to spend a lot of time thinking about climate change, that topic just ... might not appear much.
That is the one of your paths to a Butlerian Jihad. Of course, in their rush to observe thou shalt not make a machine in the likeness of the human mind, ignoring the hijinx on Ix, they ended up violating another precept: thou shalt not disfigure the soul. They transformed some people into machines, instead, with twisted Mentats being the best example, but we might also include Imperial conditioning, since, to turn from oranges Catholic Bible to those of Clockwork, when a man ceases to choose, he ceases to be a man.
> Pre-Brian Herbert and Kevin Anderson, the Butlerian Jihad came about because people became lazy under AI, lazy of mind, and were eventually enslaved by those who controlled the AI.
Oh I love this, basically Wall-E is a better Dune than the latest books? Genius!
Anyway, I read Dune many years ago and don't remember this, is this explained in the novels or is it coming from some other sources?
It's not fully fleshed out in the novels, so you have to piece it out from various mentions and their implications. The two most informative descriptions, IMO, are from Dune:
"Then came the Butlerian Jihad — two generations of chaos. The god of machine-logic was overthrown among the masses and a new concept was raised: “Man may not be replaced.”"
and from God-Emperor:
"The target of the Jihad was a machine-attitude as much as the machines ... Humans had set those machines to usurp our sense of beauty, our necessary selfdom out of which we make living judgments. Naturally, the machines were destroyed."
OP's original claim was "enslaved by those who controlled the AI", which is rather different.
But, yes, I don't think that the books imply literal enslavement either way. It could be described as "enslavement of the mind" in a sense of humans themselves adopting the "machine-logic" and falling prey to it.
"Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them."
As someone who read "Dune" several decades ago and doesn't remember much, this is very interesting and almost makes me want to read it again (though I'm afraid of yet another time sink – I'm now after 2 volumes of "The Stormlight Archive"...).
But _my_ inner geek must nitpick here and mention that the Daleks are not robots, but living creatures in metal cases. Though they do have some kind of computer-augmented memory IIRC.
That's one of the possibilities, but if we're talking about those kinds of hypotheticals, allow me another one. If you recall, one of the themes of the later Dune books was that humanity was fleeing from the frontiers (The Scattering) back into the heartland from some unknown but extremely powerful enemy. Perhaps those were simply the neighboring species who did not have a Butlerian Jihad of their own?
I don't recall anything indicating that in Frank Herbert's original series. And Brian Herbert's "Omnius" nonsense seems to have very little to do with what Frank actually had in mind wrt Butlerian Jihad etc, so I don't consider it canonical.
It describes humans hiding from "seeker machines", which does indeed imply some kind of AI. But whether that AI is the enemy or merely one of its tools is left unclear.
> The only danger is that stupid people might get their brains programmed by AI rather than by demagogues which should have little practical difference.
This may be the best point that you've made.
We're already drowning in propaganda and bullshit created by humans, so adding propaganda and bullshit created by AI to the mix may just be a substitution rather than any tectonic change.
The problem is that it will be cheaper at scale. That will allow the BS to be even more targeted growing the population of acolytes of ignorance. I don't know how much bigger it is, but it seems like we're around 20% today. If it gets to a majority there could be real problems.
There's already more information generated each day than any individual can ever hope to process, adding more crap to that pile doesn't move the needle much if at all.
but most of that crap will go entirely disregarded, because it's not coated in something that makes it attractive. what ai really helps scale up is finding millions of ways to dress up fake news, hoaxes, propaganda, etc in a container - a Trojan horse, if you will - that encourages people to consume it.
as one example, imagine if you could churn out a stream of "foreigners are taking your jobs!" articles, but every one talking about a different sector or profession. people would be far more likely to stop and read the article that was talking about their particular line of work, and thanks to ai you can now practically write a custom article for anyone you can think of in as much time as it takes to identify the demographic.
Maybe with this knowledge, we can teach and encourage people to think more critically as they choose to adopt more of these tools?
I know more academic/intellectual types who are less willing to, than I do the average joe who seeks answers from all directions and discerns accordingly.
Babi Yar is one of the single largest massacres of Jews during WW2. Some 30,000 Jews died in the first two days of the massacre. The overall SS commander in charge, Paul Blobel, was sentenced to death at Nuremberg.
The event is widely remembered as a massacre of Jews. It's acknowledged by Ukraine as well. This is not "a Russian fake" and there is plenty of evidence for it.
As I can see in Nuremberg Trials, Babi Yar was mentioned twice:
18 October 1945, Berlin
In the Crimea peaceful citizens were gathered on barges, taken out to sea and drowned, over 144,000 persons being exterminated in this manner.
In the Soviet Ukraine there were monstrous criminal acts of the Nazi conspirators. In Babi Yar, near Kiev, they shot over 100,000 men, women, children, and old people. In this city in January 1942, after the explosion in German Headquarters on Dzerzhinsky Street the Germans arrested as hostages 1,250 persons—old men, minors, women with nursing infants. In Kiev they killed over 195,000 persons.
In Rovno and the Rovno region they killed and tortured over 100,000 peaceful citizens.
20 November 1945, Nuremberg:
In the Crimea peaceful citizens were gathered on barges, taken out to sea and drowned, over 144,000 persons being exterminated in this manner.
In the Soviet Ukraine there were monstrous criminal acts of the Nazi conspirators. In Babi Yar, near Kiev, they shot over 100,000 men, women, children, and old people. In this city in January 1941, after the explosion in German headquarters on Dzerzhinsky Street the Germans arrested as hostages 1,250 persons—old men, minors, women with nursing infants. In Kiev they killed over 195,000 persons.
In Rovno and the Rovno region they killed and tortured over 100,000 peaceful citizens.
Where can I find more information about the Babyn Yar crime that was presented on the Nuremberg Trials?
To clear this up, because open Holocaust denial is never OK:
Thousands of jews were indeed murdered at Babi Yar, you can find their names online [0]. Paul Blobel [1] was executed at Nuremberg for this. No serious source disputes the killing of jews at Babi Yar, be it the US, Germany (where this matter went to court), Israel, Ukraine, or Russia.
I couldn't find any connection between the names on the list and the people who lived in Kiev during the war.
Prior to the war, there were 224,000 Jews living in Kiev, and approximately 200,000 were evacuated to the east. Around 40,000 Jews remained, including my grandmother. This is a relatively small number of people that can be verified.
It's your fault that you deny the widely accepted Babi Yar massacre and call it "a Russian fake", exactly what Holocaust deniers do. If the shoe fits...
So in your opinion, countries such as all of Europe (Germany included), the US, Israel, etc are all part of this conspiracy?
Look at this situation through my eyes. I currently reside in the Rivne region. Roughly 20 years ago, I had a conversation that went something like this:
"I know that many Jews were shot by Germans in the Rivne region."
"Yes. They are buried in the Sosenki area. We can drive there in a car. Would you like me to introduce you to relatives who can tell you more about it?"
"No, that's not necessary. Where are the Kiev Jews who were shot in Babi Yar buried?"
"I don't know. Isn't your mother from Kiev? You should know where your relatives are buried."
"I don't know anything about them. I'll ask my mother."
Later on:
"Mom, do you know anyone from our acquaintances who was shot in Babi Yar?"
"The Germans shot Ukrainians in Babi Yar, but it was forbidden to talk about it in the Soviet Union, just like with the Holodomor, or else you would be arrested."
I am a descendant of Jews who lived in Kiev during World War II. But I know nothing about the Jews who were shot in Babi Yar. My relatives or acquaintances do not know either. My search led me to the grave of Ukrainians who were shot in Babi Yar and buried near the radio mast. This grave is visible in aerial reconnaissance photographs.
If you know who was shot and where they are buried, please tell me. I would be grateful. We can erect a monument at this location.
Also weird how someone supposedly so deeply informed has somehow missed both the existing memorial and the year-long fights over building additional ones and has no clue where they might be placed?
So, we are aware of this memorial, but we don't know why it stands there. Nobody is buried beneath that memorial. It could be placed anywhere, even on the Moon. It's just a stone structure.
Furthermore, the Security Service of Ukraine warns[0] us that this memorial is being built by Russians in order to discredit Ukraine. And we completely agree with them.
I don’t think the post above makes a good point because there’s no links to evidence. They talk about “declassified archives”, “US spy plane”. Like, what is that? Saying “Russian lies” 10 times in every paragraph doesn’t turn a rant into compelling argument.
I mean, I get it, the poster above is Ukrainian and wants to turn every discussion in a certain direction. It’s just there’re many places on the internet they can vent.
I'd actually go further and say it's better to assume I can be deceived by confident language, than to assume that this is a problem with "dumb people".
If I see other people making a mistake, I want my first question to be "am I making the same mistake?". I don't live up to that aspiration, certainly.
Part of what's extremely frustrating about talking to people about AI is that people say things which seem like they understand what's going on in one breath, and then in the next breath say something that makes absolutely no sense if they understood what they just said. This post is a great example of that.
Okay, so you apply epistemology to ChatGPT: when you ask ChatGPT a question, how does it know the answer? The answer is: it doesn't know the answer. All it knows is how people string words together: it doesn't have any understanding of what the words mean.
So no, it can't use the Socratic method on itself or anyone. It can't ask questions to stimulate critical thinking, because it's incapable of critical thinking. It can't draw out ideas and underlying presuppositions, because it doesn't have ideas or underlying presuppositions (or suppositions). It's not even capable of asking questions: it's just stringing together text that matches the pattern of what a question is, without even a the understanding that the text is a question, or that the following text in the pattern is an answer. "Question" and "answer" are not concepts that ChatGPT understands because ChatGPT doesn't understand concepts.
The Socratic method requires self-awareness, otherwise the questions fall flat. Unfortunately I think that there is a greater chance that an LLM will become capable of this before a majority of humans will.
An old saying, but frequently applies to the difficult people in your life.
Related, I remember when wikipedia first started up, and teachers everywhere were up-in-arms about it, asking their students not to use it as a reference. But most people have accepted it as "good enough", and now that viewpoint is non-controversial. (some wikipedia entries are still carefully curated - makes you wonder)
Admins do not, in fact, have the power to hide tensions on an article, as even the act of deletion produces public logs. Certainly if you don't know to look for them, you won't see them, though.
The main page should become a disjointed piece of both sides writing areas of the article. Right now, admins pick the side and moderate the other away into the history.
Re wikipedia, I think it's most interesting how you choose which pages are trustworthy and which ones are not, without doing a detailed analysis on all the sources listed.
Yet I still am suspicious about wikipedia and only use it for pretty superficial research.
I've never read a wikipedia article and took it at face value. You might say we should treat all texts the same, and you might be right. But let's not pretend it's some beacon of truth ?
> The part that's concerning about ChatGPT is that a computer program that is "confidently wrong" is basically indistinguishable from what dumb people think smart people are like.
I don't know, the program does what it is engineered to do pretty well, which is, generate text that is representative of its training data following on from input tokens. It can't reason, it can't be confident, it can't determine fact.
When you interpret it for what it is, it is not confidently wrong, it just generated what it thinks is most likely based on the input tokens. Sometimes, if the input tokens contain some counter-argument the model will generate text that would usually occur if an claim was refuted, but again, this is not based on reason, or fact, or logic.
ChatGPT is not lying to people, it can't lie, at least not in the sense of "to make an untrue statement with intent to deceive". ChatGPT has no intent. It can generate text that is not in accordance with fact and is not derivable by reason from its training data, but why would you expect that from it?
> Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence so we don't have to worry about the fearmongering spread by the "AI safety" people any time soon if AI research continues to follow this paradigm.
I agree here, I think you can only get so far with a language model, maybe if we get a couple orders of mangitude more parameters it magically becomes AGI, but I somehow don't quite feel it, I think there is more to human intelligence than a LLM, way more.
Of course, that is coming, but that would not be this paradigm, which is basically trying to overextend LLM.
LLMs are great, they are useful, but if you want a model that reasons, you will likely have to train it for that, or possibly more likely, combine ML with something symbolic reasoning.
If you understand what it is doing, then you don't. But the layman will just see a computer that talks in language they understand, and will infer intent and sentience are behind that, because that's the only analog they have for a thing that can talk back to them with words that appear to make sense at the complexity level that ChatGPT is achieving.
Most humans do not have sufficient background to understand what they're really being presented with, they will take it at face value.
I think this is partly explained by most of the marketing and news essentially saying "ChatGPT is an AI" instead of "ChatGPT is an LLM."
If you asked me what AI is, I'd say it means getting a computer to emulate human intelligence; if you asked me what an LLM is, I'd say it means getting a computer to emulate human language. The word "language" does not imply truthiness anywhere near to the extent that the word "intelligence" does.
You could reasonably describe it as "human language emulator" back when people were using GPT-2 and the likes to compose text. But what we have today doesn't just emulate human language - it accepts tasks in that language, including such tasks that require reasoning to perform, and then carries them out. Granted, the only thing it can really "do" is produce text, but that already covers quite a lot of tasks - and then of course text can be an API call.
Interesting perspective. I'm still learning about what it really is, and I'm having trouble marrying the thoughts of a parent commenter with yours:
> ... does what it is engineered to do pretty well, which is, generate text that is representative of its training data following on from input tokens. It can't reason ...
versus
> ... doesn't just emulate human language - it accepts tasks in that language, including such tasks that require reasoning to perform ...
Maybe a third party can jump in here: does ChatGPT use reasoning beyond the domain of language, or not?
Nobody can definitely answer this question because we don't know what exactly is going on inside the model of that size. We can only speculate based on the observed behavior.
But in this case, I didn't imply that it's "reasoning beyond the domain of language", in a sense that language is exactly what it uses to reason. If you force it to perform tasks without intermediate or final outputs that are meaningful text, the result is far worse. Conversely, if you tell it to "think out loud", the results are significantly better for most tasks. Here's one example from GPT-4 where the "thinking" effectively becomes a self-prompt for the corresponding SQL query: https://gist.github.com/int19h/4f5b98bcb9fab124d308efc19e530....
I think the real point of disagreement is whether this constitutes actual reasoning or "merely completing tokens". If you showed the transcript of a chat with GPT-4 solving a multi-step task to a random person off the street, I have no doubt that they'd describe it as reasoning. Beyond that, one can pick the definition of "reason" that best fits their interpretation - there is no shortage of them, just as there is no shortage of definitions for "intelligence", "consciousness" etc.
> Most humans do not have sufficient background to understand what they're really being presented with, they will take it at face value.
Trust in all forms of media [1], and institutions [2], is at an all time low. I'm not sure why that distrust would go away, with reading the output from a company that's clearly censuring/fudging that output.
I think any "damaging" trust would, clearly, be a transitory phenomenon, since the distrust in media and institutions is from an ability to see BS. I don't think some sentences on a screen will be as destructive as some think, because people don't appear to be as stupid as some believe.
ChatGPT is not lying to people, it can't lie, at least not in the sense of "to make an untrue statement with intent to deceive".
ChatGPT doesn't really have a conception of true. It puts forward true and false things merely because it's cobbling together stuff in it's training set according to some weighing system.
ChatGPT doesn't have an intent but merely by following the pattern of how humans put forward their claims, ChatGPT puts forward it's claims in a fashion that tends to get them accepted.
So without a human-like intent, ChatGPT is going to be not just saying falsehoods but "selling" these falsehoods. And here, I'd be in agreement with the article that the distinction between this and "lying" is kind of quibbling.
I think the discussion of whether an LLM can technically lie is a red herring.
The answer you get from an LLM isn't just a set of facts and truth values; it is also a conversational style and tone. It's training data isn't a graph of facts; it's human conversation, including arrogance, deflection, defensiveness, and deceit. If the LLM regurgitates text in a style that matches our human understanding of what a narcissistic and deceitful reply looks like, it seems reasonable we could call the response deceitful. The conversation around whether ChatGPT can technically lie seems to just be splitting hairs over whether the response is itself a lie or is merely an untrue statement in the style of a lie--a distinction which probably isn't meaningful most of the time.
Ultimately, tone, style, truth, and falsity are just qualia we humans are imputing onto a statistically arranged string of tokens. In the same way that ChatGPT can't lie it also can't be correct or incorrect, as that too is imputing some kind of meaning where there isn't any.
In short, it is not a liar its a bullshitter. A liar misrepresents facts, a bullshitter doesn't care if what they say is true so long as they pass in conversation.
I completely disagree with this idea that the model doesn't "intend" to mislead.
It's trained, atleast to some degree, based on human feedback. Humans are going to prefer an answer vs no answer, and humans can be easily fooled into believing confident misinformation.
How does it not stand to reason that somewhere in that big ball of vector math there might be a rationale something along the lines of "humans are more likely to respond positively to a highly convincing lie that answers their question, than they are to to a truthful response which doesn't tell them what they want, therefore the logical thing for me to do is lie as that's what will make the humans press the thumbs up button instead of the thumbs down button".
I don't think it intends to mislead because its answers are probabilistic. It's designed to distill a best guess out of data which is almost certain to be incomplete or conflicting. As human beings we do the same thing all the time. However we have real life experience of having our best guesses bump up against reality and lose. ChatGPT can't see reality. It only knows what "really being wrong" is to the extent that we tell it.
Even with our advantage of interacting with the real world, I'd still wager that the average person's no better (and probably worse) than ChatGPT for uttering factual truth. It's pretty common to encounter people in life who will confidently utter things like, "Mao Zedong was a top member of the Illuminati and vacationed annually in the Azores with Prescott Bush" or "The oxygen cycle is just a hoax by environmental wackjobs to get us to think we need trees to survive," and to make such statements confidently with no intent to mislead.
> Even with our advantage of interacting with the real world, I'd still wager that the average person's no better (and probably worse) than ChatGPT for uttering factual truth.
ChatGPT makes up non-existing APIs for Google cloud and Go out of whole cloth. I have never met a human who does that.
If we reduce it down to how often most people are wrong vs how often ChatGPT is wrong, then sure, people may be on average wrong more often, but there is a difference in how people are wrong vs how ChatGPT is wrong.
"ChatGPT makes up non-existing APIs for Google cloud and Go out of whole cloth." I like the word used in TFA, "confabulating," meaning the "production or creation of false or erroneous memories without the intent to deceive." Lying, on the other hand, is telling a deliberate falsehood, usually with some kind of agenda.
Ironically, calling ChatGPT's generation of incorrect answer a "lie" is something of a lie itself, as the purpose is the agenda of alerting people to take GPT statements with a grain of salt. A programmer is going to do that anyway after the first time they realize a generated code snippet is giving a completely incorrect result. So this advice is meant more for lay people who might just think that if a computer spits it out, it's true. The problem I have is that by labeling it as "lying," it could give the impression that the program has some kind of ulterior social or political motive, as that seems to be the prevalent reductionist interpretation of everything these days.
The other problem I have is that there's a distinction to be made between what I'm going to call "generative falsehoods" vs. "found falsehoods." A generative falsehood is if you ask ChatGPT what the square root of 36 is, and it tells you the answers are 7 and -7. A found falsehood is if ChatGPT erroneously reports Christopher Columbus's date of death as 1605 because it was stated incorrectly in an online article. So the difference between making shit up, and having an unreliable source. You might be able to call the first a lie, but the latter is at worst negligence.
Schizophrenics tend to be unable to tell the difference between their delusions and reality.
On a less extreme note, I've known plenty of humans that constantly make up details and rewrite stories of events as well. They are usually very confident that their retelling is accurate, even when presented with evidence that they have reimagined portions of it.
>It can't reason, it can't be confident, it can't determine fact.
In the following link I tasked it with having to generate novel metaphors that have an equivalent non-literal meaning as an first set, changing the literal topics while maintaining the non-literal topics.
How would you suggest it does this without reason? To hand-wave what it can do as "merely generating the next token statistically" seems like a gross understatement. I doubt it picked up a corpus of car-to-curling metaphor translations somewhere :P
I understand how chatgpt is creating its next tokens, but I have my doubts that the process should be viewed as unreasoning. GPT-3 had 96 layers and billions of weights between them. GPT-4 increases on this even further. GPT-5, which I've seen mentioned as currently training, will no doubt once again expand this range.
It is not a human reasoning, certainly. It has no experiential data to draw on, yes. No experiences to root its metaphoric language as we humans use. But without reason, how does it translate between abtractions?
It's terrible at math, yes. But it lacks any capacity for "visualization" or "using a board in its head" or "working through a problem by moving things around in its head". It doesn't have any equivalent to the portions of our brains that handle such things.
But humans too can suffer dyscalulia if a specific portion of the brain is injured.
I expect that we are dealing with what amounts to a fairly brain-damaged intelligence. It seems capable of abstract metaphoric reasoning, with many other sorts of reasoning being denied to it by the nature of how we created it.
I wouldn't be surprised if there's very successfulsoftware hustlers that do make up Google cloud apis. You may not know them but that doesn't mean they don't exist.
5 years down the line though, maybe those apis will exist because chatgpt is giving a summary of what apis Google cloud should have, and Google will listen
> Google cloud should have, and Google will listen
Why? Isn’t what ChatGPT suggesting just random, essentially incoherent noise in those cases? Are there any examples at all of it coming up with something actually useful (and something humans haven’t thought of)?
At the current time it is flawed to the point of being dangerous. It’s a - sometimes - useful new set of tooling. I guess there is value in that … but does it outweigh the always lurking “bullshit generator” bad parts? I’m not sure.
It’s interesting to watch the developments though - like a fire - one just doesn’t fully know what the flames are consuming.. yet.
Maybe through all the new training data we collectively provide for free, it will get better? Maybe not though, maybe it will just get better at bullshitting?
> How does it not stand to reason that somewhere in that big ball of vector math there might be a rationale
I think, a suggestion that it is actually reasoning along these lines would need more than "it is possible". What evidence would refute your claim in your eyes, what would make it clear to you that "that big ball of vector mat" has no rationale, and is not just trying to trick humans to press the thumbs up?
Of course the feedback is used to help control the output, so things that people downvote will be less likely to show up, but I have nothing to suggest to me that it is reasoning.
If you think it has intent, you have to explain by what mechanism it obtained it. Could it be emergent? Sure, it could be, I don't think it is, I have never seen anything that suggests it has anything that could be compatible with intent, but I'm open to some evidence that it has.
What I'm entirely convinced about is that it does what it was designed to do, which is generate output representative of its training data.
I would at least start to be convinced that this is NOT the case if I ever saw it respond with something like "I actually don't know the answer to that query" or "as far as I'm aware, there is no way to do the thing you asked".
These are responses that would have shown up innumerable times in it's training data and make perfect sense as "the most logical next set of tokens", and yet it will never say them.
Instead it will hallucinate something that sounds nearly indistinguishable from fact, but turns out to be a total fabrication.
If all its doing is returning the next most logical set of tokens, and the training data it was based on included a non-trivial number of examples where one party in the conversation didn't have a clear answer, then there's no reason GPT-4 should be so averse to simply saying "yeah, I dunno bro".
The only logical reason I can see is that it's "learned" that it's more likely to receive the positive feedback signal when it makes up convincing bullshit, than if it states that it doesn't have an answer.
EDIT: To be clear, I mean it telling me it doesn't know something BEFORE hallucinating something incorrect and being caught out on it by me. It will admit that it lied, AFTER being caught, but it will never (in my experience) state that it doesn't have an answer for something upfront, and will instead default to hallucinating.
Also - even when it does admit to lying, it will often then correct itself with an equally convincing, but often just as untrue "correction" to its original lie. Honestly, anyone who wants to learn how to gaslight people just needs to spend a decent amount of time around GPT-4.
I think of ChatGPT as a natural language query engine for unstructured data. It knows about the relationships that are described in it's natural language input training data set, and it allows the same relationships to be queried from a wide range of different angles using queries that are also formulated in natural language.
When it hallucinates, I find that it's usually because I'm asking it about a fringe topic where it's training data set is more sparse, or where the logical connections are deeper than it's currently able to "see", a sort of a horizon effect.
>Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence so we don't have to worry about the fearmongering spread by the "AI safety" people any time soon if AI research continues to follow this paradigm
I hope you appreciate the irony of making this confident statement without evidence in a thread complaining about hallucinations.
It was not a confident statement, at least not the way ChatGPT is confident.
There are multiple ways the commenter conditioned their statement:
> Based on my understanding
> it is PROBABLY very close
The author makes it clear that there is a uncertainty and that if their understanding is wrong, the prediction will not hold.
If ChatGPT did any of the things the commenter did, the problem wouldn't exist. Making uncertain statements is fine as long as it is clear the uncertainty is acknowledged. ChatGPT has no concept of uncertainty. It casually constructs false statements to same way it constructs real knowledge backed by evidence. That's the problem.
> individuals whose intelligence is roughly equivalent to ChatGPT's
There aren't any such individuals. Even the least intelligent human is much, much more intelligent than ChatGPT, because even the least intelligent human has some semantic connection between their mental processes and the real world. ChatGPT has none. It is not intelligent at all.
Many of the smarter people are still wrong on what happened on many topics in 2020. They were fooled by various arguments that flew in the face of reality and logic because fear and authority was used instead.
The people that avoid this programming isn't based on smart or stupid. It's based on how disagreeable and conscientious you are. A more agreeable and conscientious person can be swayed more easily by confidence and emotional appeals.
There's an alternate reality where OpenAI was, instead, EvenMoreClosedAI, and the productivity multiplier effect was held close to their chest, and only elites had access to it. I'm not sure that reality is better.
It is known that some got access to GPT-4 before the rest of the world did. That OpenAI eventually released it to the world is what counts though. Hopefully GPT-5 will also be released too the world but we shall have to see.
Your characterization of “dumb people” as somehow being more prone to misinformation is inaccurate and disrespectful. Highly intelligent people are as prone to irrational thinking, and some research suggests even more prone. Go look at some of the most awful personalities on TV or in history, often they are quite intelligent. If you want to school yourself on just how dumb smart people are I suggest going through the back catalog of the “you are not so smart” podcast.
Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence so we don't have to worry about the fearmongering spread by the "AI safety" people any time soon if AI research continues to follow this paradigm.
ChatGPT is extremely poorly understood. People see it as a text completion engine but with the size of the model and the depth it has it is more accurate in my understanding to see it as a pattern combination and completion engine. The fascinating part is that the human brain is exclusively about patterns, combining and completing them, and those patterns are transferred between generations through language (sight or hearing not required). GPT acquires its patterns in a similar way. A GPT approach may therefore in theory be able to capture all the patterns a human mind can. And maybe not, but I get the impression nobody knows. Yet plenty of smart people have no problem making confident statements either way, which ties back to the beginning of this comment and ironically is exactly what GPT is accused of.
Is GPT4 at its ceiling of capability, or is it a path to AGI? I don’t know, and I believe nobody can know. After all, nobody truly understands how these models do what they do, not really. The precautionary principle therefore should apply and we should be wary of training these models further.
GPT-2, 3 and 4 keep on showing that increasing the size of the model keeps on making the results better without slowing down.
This is remarkable, because usually in practical machine learning applications there is a quickly reached plateau of effectiveness beyond which a bigger model doesn't yield better results. With these ridiculously huge LLMs, we're not even close yet.
And this was exciting news in papers from years ago talking about the upcoming GPT3 btw.
That's true, or at least not in the way ChatGPT does.
"Based on my understanding" does convey some lack of confidence in a sense. But it's also implying they might have more understanding than other posters here, when the content of their post indicates they have less.
Have you not seen current politics. What people believe is largely based on motivated reasoning rather than anything else. ChatGPT is basically a free propaganda machine, much easier that 4chan
the more common idiom is "XYZ is a stupid person's idea of what a smart person looks like" and it's usually applied to slimy hucksters or influencers. I think the archetypical example is the "bookshelves in my garage" guy who went viral years ago. (https://youtu.be/Cv1RJTHf5fk)
We can't protect people from being misled by other humans with big mouths. That's their responsibility.
Likewise, it's their responsibility not to treat text they read from the internet, coming from an AI or otherwise, as perfect truth.
There's always a certain undercurrent of narcissism that flows beneath paternalism. Basically, "they couldn't possibly be as smart as me, so I have to protect them for their own good".
Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence so we don't have to worry about the fearmongering spread by the "AI safety" people any time soon if AI research continues to follow this paradigm. The only danger is that stupid people might get their brains programmed by AI rather than by demagogues which should have little practical difference.