You are underestimating the scale of the tectonic shift.
A computer program that can pass the god damned Turing test is not "a mildly useful parlor trick", it is the single most impressive computer program ever made. It exhibits reasoning, it can think with analogies. You can give it complicated requests in natural language. Given suitable prodding, it's creative. Everyone just woke up to the fact that we're a sneeze away from AGI.
>The issues with these systems are obvious to everyone
They are not. Already people routinely ask ChatGPT for factual info, and it doesn't bother them that it will simply make things up. It walks and quacks like a duck, so people assume it's a duck.
>addressing those issues is a deeply technical and deeply difficult problem
Alignment is a deeply difficult problem for philosophical reasons - we don't know how to reliably "align" humans either - but getting LMMs to output "roughly" what we want is fun and easy (they're like virtual humans!) and they're going to be deployed everywhere, problems notwithstanding.
Things are about to change, big time. In the short term we're only talking about something like the smartphone revolution, where the parameters of social interaction are fundamentally reshaped. In the long term it could get real weird...
I think it's pointless arguing about "slowing down", the cat is out of the bag. I would just like to see some transparency rules about weights and prompts - we're rapidly reaching a stage where a company hosting a highly used language model could do extreme violence to culture and business. Language models aren't paperclip maximizers - corporations are paperclip maximizers, and language models are the HypnoDrones.
People used to anthropomorphize Markov chains like this. I've been through this rodeo before, at least a few times.
I did a little lab on GPT with middle schoolers as part of a Science enrichment activity. Without prompting, the entire class was making it say nonsense inside of the one hour session.
Try using a GPT model to do something humans actually do, other than random bullshitting on the internet. Field customer service requests. Even carefully SFT'd models struggle to beat decision trees, and are sometimes worse, at actually solving the customer's problem. They sound more human / less robotic, but who cares if the customer's problem isn't solved and the dialog quickly diverges into insanity?
Just because you want to see God on a piece of toast doesn't mean that the average human has completely lost their capacity for critical thought.
> doesn't mean that the average human has completely lost their capacity for critical thought
The average person, maybe everyone, is very selective about where they apply their critical thought processes.
And groups of people ... groups don't have critical thought like individual people do.
Corporations, political parties, social groups don't "think" coherently, but wield most of the power by being centralized or synchronized.
Groups survive and grow by developing feedback and incentives that push back against individuals with critical not-aligned-with-the-group thought.
If we are depending on unorganized mass critical thought to save us, we are surely doomed. We are going to need our best systems for getting along to date, and upgrades.
A computer program that can pass the god damned Turing test is not "a mildly useful parlor trick", it is the single most impressive computer program ever made. It exhibits reasoning, it can think with analogies. You can give it complicated requests in natural language. Given suitable prodding, it's creative. Everyone just woke up to the fact that we're a sneeze away from AGI.
>The issues with these systems are obvious to everyone
They are not. Already people routinely ask ChatGPT for factual info, and it doesn't bother them that it will simply make things up. It walks and quacks like a duck, so people assume it's a duck.
>addressing those issues is a deeply technical and deeply difficult problem
Alignment is a deeply difficult problem for philosophical reasons - we don't know how to reliably "align" humans either - but getting LMMs to output "roughly" what we want is fun and easy (they're like virtual humans!) and they're going to be deployed everywhere, problems notwithstanding.
Things are about to change, big time. In the short term we're only talking about something like the smartphone revolution, where the parameters of social interaction are fundamentally reshaped. In the long term it could get real weird...
I think it's pointless arguing about "slowing down", the cat is out of the bag. I would just like to see some transparency rules about weights and prompts - we're rapidly reaching a stage where a company hosting a highly used language model could do extreme violence to culture and business. Language models aren't paperclip maximizers - corporations are paperclip maximizers, and language models are the HypnoDrones.