Also its answering the question of what gonna happen if you wake up tomorrow and datacenters are gone. Or internets are gone.
Some people on our globe live in countries with no internet whatsoever. Of course most of them dont have Macbook with 64GB RAM either, but it's much much easier to get than internet connection or rack of GB200.
SOTA LLMs are efficiently compression of all the knowkedge humanity has built. Having ability to run it at home to extract said knowledge is important no matter the speed.
A post-apocalyptic desert landscape. The hero descends into the ruins, now a temple. Crude drawings of times before, people riding dinosaurs and mobile phones with arms and legs swinging a whip at the oppressed. He reaches the altar, a old MacBook, bottlecaps and fidgit spinners glued as a mosaic onto it. In the corner three starved men, riding on power generating bicycles providing some artificial light and supplying the machine with divine electricity. He proceeds and types: "we are starving. How can we get food?". Everybody waits in silence. Hours pass. Days. Slowly, letters emerge on the cracked screen: "you are absolutely right —" the screen flickers and fades. Panic. The link to God severed cynically. The hero has to seek out a new device, into distant lands. Maybe this time it will save them
That's not a great argument for having an llm in an apocalyptic scenario.
People tend to think they are the thriving hero but most won't survive the first year.
I'd much rather have a USB stick of prepper disk. It works on any shitty machine I find, and it's not going to take days for an answer on how to make potash or fix a solar panel.
Also its answering the question of what gonna happen if you wake up tomorrow and datacenters are gone. Or internets are gone.
Some people on our globe live in countries with no internet whatsoever. Of course most of them dont have Macbook with 64GB RAM either, but it's much much easier to get than internet connection or rack of GB200.
SOTA LLMs are efficiently compression of all the knowkedge humanity has built. Having ability to run it at home to extract said knowledge is important no matter the speed.