- The physics of the universe can be completely modeled as computation, and
- It's possible to pose undecidable problems about the way the universe unfolds
This is intrinsic to the idea of undecidability even for Turing machines, e.g. "we equate computation with the functioning of Turing machines, but there are real processes executable in Turing machines that are undecidable".
A key thing about the undecidability problem wrt physics is preparation of the initial state. In math and computer science it is relatively straightforward to prepare such problems now (though this represented an enormous leap conceptually), but the "undecidability" of all physical problems relies on construction of materials that are clearly unconstructable - systems of infinite negentropy (eg Turing machines), infinite mass (the lattice), bespoke local interactions etc. Problems standing in the way of physics decidability are typically chaos, far from equilibrium mechanics, elementary SNR considerations and so forth, not problems of logic.
Of course, if our universe is undecidable it must be the case that computable processes can be executed within it, and it might be the case that all of the processes that are ever executed within it are computable... but it might be that some of the processes that are executed are not computable... because the machine may.. or may not?
I think there's an equivocation of "computable" going on here. Mathematicians talk about a lot of things like "uncomputable sequences" but that is usually making a statement about the sequence, not necessarily any individual member. The Busy Beaver sequence is uncomputable. You can, however, quite trivially compute BB(2), even in your head if you're a bit careful. You can set up individual elements of an uncomputable sequence in our universe, and you may be unable to state in advance what the system would do with anything less than simply letting it run and see what happens due to the complexity of the system, but being a member of an uncomputable sequence doesn't mean that you can't in fact set those things up and watch them run. The Universe doesn't throw an "UncomputableCircumstance" exception or anything. It just keeps advancing to the next state. Your inability to make certain statements about that next state or some future state is not its problem.
There's no way to empirically spot an uncomputable process, since it would require infinitely-many observations.
For example, if aliens claim their machine solves the halting problem, we could test it on millions of inputs whose halting/not-halting behaviour we already know; but even if it works for all of them, there's no way to know that it works for all inputs. For all we know, it might be a huge lookup table which happens to cover all of those inputs we tried.
My point is, there would be no way to empirically test this; and therefore, it would make no observable difference, there would be no way to exploit/utilise such effects, etc.
In essence: there's no way to tell the difference between a real halting oracle (which would imply an undecidable universe), versus a computable approximation which just-so-happens to be more powerful/sophisticated than the approximations we compare it against.
Sure, we can prove that some abstract systems are undecidable and that others aren't. Yet that distinction is inherently unfalsifiable, and hence physically "useless".
Deterministic processes can be modeled probabilistically and are computable, so existence of a probabilistic model doesn't say much about computability.
Interestingly I have found many tech circles in SF would rarely discuss ultra local things like supervisor politics. I don’t even think it’s taboo, I just have found folks tend to not be super informed on any of it. Lurie does seem to capture plenty of political mindshare though.
That area still has some 'gold rush' mentality to it, where people want to get in, make their money and get out. That doesn't foster much of a long-term attitude that pays attention to local politics. Also some of the local politics are totally psychotic, like the guy harassing Scott Wiener.
This was long before. Google had conversational LLMs before ChatGPT (though they weren’t as good in my recollection), and they declined to productize. There was a sense at the time that you couldn’t productize anything with truly open ended content generation because you couldn’t guarantee it wouldn’t say something problematic.
I'm having a hard time believing this, or at least understanding the decision (not on your part). Why wouldn't they just continue R&D on it rather than drop it entirely?
Many products we use every day start out unsafe and dangerous during the early stages. Why would this be any different?
Both would be understood and are roughly interchangeable.
"Sequential" feels more appropriate to me for the task runner scenario where we wait for one task to finish before running the next.
"Series" suggests a kind of concurrency to me because of the electrical circuit context, where the outputs of one are flowing into the next, but both are running concurrently. Processes that are Unix piped into each other would be another thing that feels more like a "series" than a "sequence".
You might be using JSON mode, which doesn’t guarantee a schema will be followed, or structured outputs not in strict mode. It is possible to get the property that the response is either a valid instance of the schema or an error (eg for refusal)
I assume you know how to program in Python? I would start with just the client libraries of the model providers you want to use. LLMs are conceptually simple when treated as black boxes. String in, string out. You don't necessarily need a framework.
I’m not as familiar with the Baby Bells, so this is a surprising comparison to me. Bell Labs was famously so productive while it had the monopoly money hose, and not as much came from it after Bell was broken up.
What are the most noteworthy accomplishments of the Baby Bells?
While I lament the decline of Bell Labs and unfettered research in general (unfettered research labs need protection from market forces that only a monopoly, government, academia, or the very wealthy can provide), I also believe that the breakup of the Bell System was overall a good thing for society. For example, there was a time when AT&T customers had to rent their phones; they couldn’t own them (https://memorial.bellsystem.com/bell_system_property.html). Customers were finally allowed to purchase their own phones once the divestiture was underway. In addition, I’m not sure if we’d have a competitive cell phone market in America today had the Bell System remained in place, not to mention how I haven’t heard anything about long-distance calling charges in about 15 years due to how many modern cell phone plans work.
Ironically, we're nearly back at the "renting" phone stage. Sure the companies selling the phones don't use that terminology, and it's a one-time payment for the life of the device, but full control of the device is never transferred to the user. The company holds the keys and will only allow you to do what they want you to do. This certainly describes iPhones and most Android phones to date, and it's getting worse on the Android side as root becomes harder and harder.
I just don't see any positives here though. Apple will be given 100% free-reign to take complete monopolistic control of the smartphone market without Google.
Calling people became cheap. Think about making a cross-country phone call in the pre-broken up AT&T era. It was like 25¢/min. Now I pay $35/month and can literally call most countries for up to 500min before I get metered (Visible+).
it's hard to compare without a control, but a few key points:
* none of the Baby Bells failed
* because they segmented regionally, integration was super important and you could argue paved the way for the modern internet
* consumer services under Bell was incredibly expensive and tightly controlled
In hindsight maybe they should have split up horizontally, nationalizing the natural monopoly components/infrastructure (ex: the physical lines)? It's interesting to see what looks like a reconsolidation of wireless now, I wonder what the future will look like.
My opinion is that Bell Labs created great technology, but had no real incentive to make products and bring them to the public. The Baby Bells needed to compete however, and so they did.
I think even with firehose monopoly money that Bell Labs would have eventually succumbed to cuts and general enshittification as the CEOs and shareholders wanted ever increasing pay and dividends. "Do more with less guys! You're smart you can figure it out! The Board really needs this 10,000% pay raise, they have families you know."
The size of the cached internal state of the network processing the book is much larger than the size of the book. The resource that is preserved with caching is the compute required to recreate that state.
But it may very well be slower than just recompute it. At least for ordinary MHA and even GQA.
So, either a model arch woodoo significantly reducing kv cache size (while keeping roughly the same compute cost), or some really careful implementation moving kv cache of upcoming requests to devices in background [0].
[0] My back of envelop calc shows that even then it still does not make sense for, say, Llama 3 70B on H100s. Time to stare at TPU spec harder trying to make sense of it I guess.
It depends on how large the input prompt (previous context) is. Also, if you can keep cache on GPU with a LRU mechanism, for certain workloads it's very efficient.
You can also design an API optimized for batch workloads (say the same core prompt with different data for instruct-style reasoning) - that can result in large savings in those scenarios.
If you can pipeline upcoming requests and tie state to a specific request, doesn't that allow you to change how you design physical memory? (at least for inference)
Stupid question, but why wouldn't {extremely large slow-write, fast-read memory} + {smaller, very fast-write memory} be a feasible hardware architecture?
If you know many, many cycles ahead what you'll need to have loaded at a specific time.
Or hell, maybe it's time to go back to memory bank switching.
The throughput of the PCIe link between the CPU and GPU, is far less than the aggregate throughput of the internal interconnects between neighbouring tensor cores.
Matrix operations might flow a lot of data around — but that data flow is akin to a bunch of individual people travelling along the individual residential streets they live on. There's a lot of movement there, but also a lot of capacity for movement, because there's no bottleneck of everyone needing to go to the same place or come from the same place.
Persisting the data out of the GPU and then loading it back in, is more like all those people commuting to work and then going back home. Big fan-in onto the PCIe "highway" over to the CPU and into RAM; then big fan-out back. Traffic jams for miles.
In the time it takes to restore a 1GB state snapshot from RAM into VRAM, you can probably chew through the equivalent of 1TB or more of intermediate matrix states.
I don’t know of any public details on how they implement Context Caching, but that is presumably exactly what they are doing. Just caching the text would be a minimal savings.
But isnt the information somehow cached when you start a new chat and build context with say GPT4? If the caching was so large as you say so many chat sessions in parallel would not be possible.
That's not my understanding. We can't be sure how OpenAI does things themselves, but adding messages to a conversation in the API means just rerunning the history through the prompt every time
And the internal state of a JPEG decoder can be an order of magnitude larger than the JPEG file (especially progressive JPEG that can't stream its output).
You can make any lossy compression scheme into a lossless scheme by appending the diff between the original and the compressed. In many cases, this still results in a size savings over the original.
You can think of this as a more detailed form of "I before E, except after C, except for species and science and..." Or, if you prefer, as continued terms of a Taylor-series expansion. The more terms you add, the more closely you approximate the original.
- The physics of the universe can be completely modeled as computation, and
- It's possible to pose undecidable problems about the way the universe unfolds
This is intrinsic to the idea of undecidability even for Turing machines, e.g. "we equate computation with the functioning of Turing machines, but there are real processes executable in Turing machines that are undecidable".