I don't know why the invention of node.js tickled me so much but here we are. Thanks for writing this comment. Paired perfectly with my morning coughey.
What I don't understand is where is the memory? How does GPT-3 or ChatGPT remember so much information with just that architecture? It would seem that the maximum it could remember is 2048 words.
EDIT: Maybe it's 2048 x 96? Still seems low for what it can do.
Yes, but how does it remember the stuff you told it earlier in the conversation? Those 1.2TB is the trained model, and I assume that those weights are not changed by the conversation?
I'm fairly sure GPT-3 can't do any of that, because it was trained only with text. That is, it was trained without the context we are exposed to. Now imagine if it was trained with a body (similar sensors to what we have) in the real world. I'm not so sure we wouldn't get something indistinguishable from a human.
That would be a different story. Imagine its parents trained it or trained itself on data derived from the environment mixed with other bits we could already call its experience. Imagine it then went to some kind of kindergarten with others, then school, them started to look for a partner then a job and so on. Yeah, in that sense we’d be going full circle replicating the life of humans on silicon. If I were still around Id be befuddled as why we went on this track
I just want to correct some comments that say we are certain there are no local variables and that they have been ruled out experimentally.
This is false information and even Bell himself knew it:
"There is a way to escape the inference of superluminal speeds and spooky action at a distance. But it involves absolute determinism in the universe, the complete absence of free will."
Regarding unfalsifiable, from [0] "If engineers ever succeed in making such quantum computers, it seems to me that the CAT is falsified; no classical theory can explain quantum mechanics." By "such quantum computers" he means computers that can run Shor's algorithm. "...but factoring a number with millions of digits into its prime factors will not be possible – unless fundamentally improved classical algorithms turn out to exist."
Or right in the page I linked: "This makes it possible to construct a local hidden-variable theory that reproduces the predictions of quantum mechanics, for which a few toy models have been proposed."
From Wikipedia about superdeterminism:
"By postulating that all systems being measured are correlated with the choices of which measurements to make on them, the assumptions of the theorem are no longer fulfilled"
By assuming superdeterminism you can indeed construct a valid local hidden variable theory. But there is no way you can say superdeterminism is by itself a local theory, and no physicts will call it so. It's as far from "local" theory as possible, it's actually global.
So, what is supposed to be wrong with "the complete absence of free will"?
It reads like some sort of religious objection. If the data leads you there, you go there.
Sabine, particularly, goes there. Apparently there are no actual problems with giving up the illusion of "free will", whatever the hell it was supposed to mean in the first place.
Distaste seems pretty rich coming from people promoting MWI.
Chess is superdeterministic as all the moves are known in advance, but free will is still there. The "reality" might be such a superdeterministic playground that simply tells what the possible next moves are, while the players keep a pointer to one "chess" position and advance it one step at a time with their free will.
Perhaps the most important notion of semantics is that of referent. Pretty sure you can't develop that without an agent that can interact with the environment.
I did something like you. I also faced many of your issues and to solve it, I was also super focused on me, me, me. Trying to be better, more motivated, healthier, more exercise, better sleep etc etc.
Now that I know myself better, I realize I need to be integrated in a community with similar objectives, even when doing individual work. Otherwise, creative and/or research work is too demanding. However, everyone is different.
Depends on the community. For research, check your local university. You can ask around in relevant departments. Don't be afraid, some researchers are more than happy to collaborate.
"Ancient history covers all continents inhabited by humans in the period 3000 BC – AD 500."