I'm in no way familiar with LLM/ML architecture, but an N trillion parameter model is an enormous amount of training data... itself? When you predict a token you have the neurons/weights that were activated. Couldn't you run that backwards and put it in it's own training data?
How much issue do you take with animal testing in the rest of medicine? We already, more or less, torture an unfathomable amount of animals to get where we are.
I think it's definitely not as "cool", but it's almost a different problem class entirely since there's no correct answers here.
Knowing TCP connection protocols or deep compiler infra is impressive insofar as it's a mountain of knowledge that you've conquered and can stand on with a nice view, whereas "context engineering" and current AI engineering is a murky domain with no correct answers because... nobody alive knows the correct answers.
There's something different about wading forward into unexplored territory, or being able to establish techniques that nobody has attempted before. A lot of it will be noise or garbage (especially given a lot of it is LLM code), but I'd be careful to not mistake clever greenfield engineering with garbage simply because there's not a book written about it you can use as an oracle.
I understand the desire for something like this, but it's very naive. Different countries had unimaginably different histories which directly propagate to different cultures and values. *This is not natively bad thing*.
The problem arises when people bring in normativity when talking about these differences. It's okay to acknowledge that other people are different than you. This is a good thing. I don't want a grey homogenous world.
> I want lower energy bills, lower rent, better understanding of health etc. more than I want theorems.
These efforts aren't mutually exclusive. I hate to be snide, but a lot of people would criticize you for being a mathematician because they want lower energy bills, lower rent, better understanding of health etc
I don’t think you hate to be snide even a little bit, if I’m honest. The difference being that the entirety of mathematics research has never pulled in the CapEx that anthropic or OAI is. Mathematicians in academia are mostly paid to teach, mind you.
I’m personally pretty excited by the results coming from these labs, but trying to dismiss the capital allocation objection with “but what about” is really silly.
You can't just look at the funding for math departments. I think the value generation potential of every mathematician in the world could exceed that of the capex of these companies. How much is "lost" when a brilliant mind goes into pure math research instead of nuclear engineering, or civil engineering etc. Cumulatively, a lot.
Edit: I'm fine saying we're spending too much money on AI, in fact I agree, but "theorems" are not the only output of these companies. It's almost definitely not going to be worth the expenditure, but the benefits are not going to be only lean proofs
> I think the value generation potential of every mathematician in the world could exceed that of the capex of these companies. How much is "lost" when a brilliant mind goes into pure math research instead of nuclear engineering, or civil engineering etc. Cumulatively, a lot.
This is already the situation that we have: because of the fierce competition for temporary academic positions, and very little hope for permanent positions, large quantities of really good mathematicians leave academia and work in a job that has basically nothing with mathematics (many mathematicians consider these jobs as bullshit jobs, but they pay the bill).
One of the problems is the power dynamic westernized/developed societies have. The article talks about just that: Australians brought shells in and basically one-shotted a rich culture for their own gain via hyper inflation.
If a missionaries message was basically "Believe in God and you'll have limitless shells", and backed it up with a crate of shells, they would all be converts.
The juxtaposition of someone's instinct to be thinking of bitcoin when reading about indigenous trade is at the best a bit funny, and at the worst small-minded when you consider the havoc those Australians brought onto their culture. Basically wiped out a foundational part of their way of life in an instant for their own gain. It's sad. That was my takeaway.
Depending on the age, people are spending between 4-6.5 hours a day on their phone. How much are people spending on computers? Their mattress? 2,000 for the cutting edge version is frankly reasonable for what a phone provides in this day and age, especially considering the last 3 years of inflation in the US...
reply