I'm sure you don't have the brain of a gnat, and, even if you did, it probably wouldn't prevent you from understanding this.
As for whether these definitions have a clear meaning that one can relate to 'the world': I think so. To take just one example (I could do more), finite-dimensional means exactly what you think it means, and you certainly understand what I mean when I say our world is finite (or three, or four, or n) dimensional.
Commutative also means something very down to earth: if you understand why a*b = b*a or why putting your socks on and then your shoes and putting your shoes on and then your socks lead to different outcomes, you understand what it means for some set of actions to be commutative.
And so on.
These notions, like all others, have their origin in common sense and everyday intuition. They're not cooked up in a vacuum by some group of pretentious mathematicians, as much as that may seem to be the case.
Opposing view (that I happen to hold, at least if I had to choose one side or the other): not only is mathematics 'reality'; it is arguably the only thing that has a reasonable claim to being 'reality' itself.
After all, facts (whatever that means) about the physical world can only be obtained by proxy (through measurement), whereas mathematical facts are just... evident. They're nakedly apparent. Nothing is being modelled. What you call the 'model' is the object of study itself.
A denial of the 'reality' of pure mathematics would imply the claim that an alien civilisation given enough time would not discover the same facts or would even discover different – perhaps contradictory – facts. This seems implausible, excluding very technical foundational issues. And even then it's hard to believe.
> To the best of our knowledge, such cases are basically coincidence.
This couldn't be further from the truth. It's not coincidence at all. The reason that mathematics inevitably ends up being 'useful' (whatever that means; it heavily depends on who you ask!) is because it's very much real. It might be somewhat 'theoretical', but that doesn't mean it's made up. It really shouldn't surprise anyone that an understanding of the most basic principles of reality turns out to be somewhat useful.
I think you're not even disagreeing with me, we're just using different definitions of the word "reality". I meant it to use specifically "the physical world" - which you are treating as distinct from mathematics as well in your second paragraph.
Mathematics is an abstract game of symbols and rules invented by humans. It has nothing to do with reality. However it is quite useful for modelling our understanding of reality.
So, the hidden mental model that the OP is expressing and failed to elucidate on is that llm’s can be thought of as compressing related concepts into approximately orthogonal subspaces of the vector space that is upper bounded by the superposition of all of their weights. Since training has the effect of compressing knowledge into subspaces, a necessary corollary of that fact is that there are now regions within the vector space that contain nothing very much. Those are the valleys that need to be tunneled through, ie the model needs to activate disparate regions of its knowledge manifold simultaneously, which, seems like it might be difficult to do. I’m not sure if this is a good way of looking at things though, because inference isn’t topology and I’m not sure that abstract reasoning can be reduced down to finding ways to connect concepts that have been learned in isolation.
Not the OP, but my interpretation here is that if you model the replies as some point in a vector space, assuming points from a given domain cluster close to each other, replies that span two domains need to "tunnel" between these two spaces.
Sometimes things that look very different actually are represented with similar vectors in latent space.
When that happens to us it "feels like" intuition; something you can't really put a finger on and might require work to put into a form that can be transferred to another human that has a different mental model
Yes, that also happens, for example when someone first said natural disasters are not triggered by offending gods. It is all about making explanations as simple as possible but no simpler.
I hadn't planned on spending my evening googling the pay grade of government officials, calculating the time taken to change a font on Microsoft Word and extrapolating that over a year.
That's because a lot of commenters here are not hackers in any real sense; rather, they're software engineers. Perhaps this hasn't always been the case.
Your argument fails right here because you're supposing something that isn't true. LLMs are better than search engines for some things, but you're speaking as if they're a replacement for what came before. They're absolutely not. Reading books — going to the original source rather than relying on a stochastic facsimile — is never going to go away, even if some of us are too lazy to ever do so. Their loss.
Put another way: leaving aside non-practical aspects of the experience, the car does a better job of getting you from A to B than a horse does. An LLM does not 'do a better job' than a book. Maybe in some cases it's more useful, but it's simply not a replacement. Perhaps a combination is best: use the LLM to interpolate and find your way around the literature, and then go and hunt down the real source material. The same cannot be said of the car/horse comparison.
That tends to happen when you're the one defining the curve.
reply