That's most likely what's happening. SNR will constantly decrease as LLM content is so much quicker and cheaper to generate, which makes it more statistically significant, which will make it more "relevant" for future models. A positive feedback basically.
"Huffing their own farts" comes to mind. Instead of building Artificial Intelligence, they were building Artificial Human Centipede.
I'd never considered that the 'Gray Goo' scenario[1] or the Paperclip Maximizer, might be informational and digital rather than physical. LLMs might turn every bit of computer storage into slop (goo) and then cause us to go mad turning the Earth's resources into making more computers (AI datacenters) to hold more goo. The GrAI Goo catastrophy.
Alan Kay performs an improvisational avant garde layered audio feedback loop about Claude Shannon, live online during Kristen Nygaard's 100-year birthday celebration.
Alan was scheduled to talk about how encountering Simula sparked his early thoughts about objects. During setup, somebody had the live stream playing out loud near an open Zoom mic, so his own voice kept arriving back in his ears about 21 seconds late, over and over.
What he said was not random: "Shannon gave us a way of dealing with noisy channels." And: "I think about that almost every day. I realize what the fuck is going on and it's just so amazing." Shannon's noisy channel coding theorem is the math for exactly the kind of channel that was garbling him as he praised it.
Alan joked it was "being rerouted to Mars and back." At the speed of light, a 21 second round trip is about 3 million km one way -- eight trips to the Moon and back, not even a twentieth of the way to Mars at closest approach.
This is an accidental Zoom performance of Alvin Lucier's "I Am Sitting in a Room" (1969), where Lucier re-recorded his own voice in a room until only the room's resonance remained. Here what remains is the network: delay, compression, dropouts.
Bonus noise: YouTube's auto-transcript bleeps Alan's enthusiasm into [ __ ]. So the full chain is: Alan's voice, Zoom, stream, room, Zoom again times three, my screen recording, YouTube's speech recognizer, a censored transcript.
The fix: "Just turn off the audio at your end on Zoom."
I have argued with managers and it's so boring and frustrating. At least with an AI I can start a conversation from scratch, or even practice with a locally running instance.
The only reason managers exist is because of their power to fire people. They are essentially clerks that "coordinate" work, as if work needs that much coordination.
My bet is that engineers will have to go down, closer to the silicon level. The higher-level stuff will become more and more accessible to the non-tech consumer, so building apps and stuff will no longer require tech-savvy people, everyone will be able to make their own app.
i'm a low-level developer but I disagree, low-level C or chip design or firmware development are actually a lot simpler than (complex) app development, AI can take over low-level way more easily than upper level sw devel.
reply