Is AI at the stage that it can also point out all interesting things to study and then solve them? Otherwise humans would still be able and needed for progress.
Good question. I'm not a professional mathematician, but I would be interested in seeing what results the mathematicians get from trying to prompt it to come up with new and interesting problems.
My intuition is that it'll be at 'smart undergrad' levels today, though as the cryptographers say, attacks always get better.
Why is not like changing to a different type of vessel for the voyage? Unless and until it is all AI in closed system, could direct machines where to look etc.
Things that are not intelligent could still warrant very strong safety measures.
Depending on the risks associated, the complexity and effort of certain containment measures would also not neccessarily be something that could be used to brush them aside.
Yes, but I'd say there is also responsibility attached to the deployment and use (be that, e.g., for quant fund performance, behaviour on exchanges, ...). In a way, using an algorithm is the decision humans make that matters.
At the same time, I've been involved in hundreds of After Action Reviews of incidents and it's not always simple or easy to both figure out which party was ultimately responsible and also how to allocate the "dollar error budget" to the parties overall.
Why would that need a new way? Or do you mean that no-one ever spun up anything at all (e.g., agent zero was never spun up or allowed to be spun up by anyone)?
That's the billion dollar question. Nvidia currently wants to deploy chips with the sole purpose of monitoring agentic workloads and that doesn't seem farfetched but they obviously have a financial motive to sell more products. It's like a cat and mouse game, like cybersecurity in general.
It isn't some law of nature. Humans/societies have agency - what AI should or should not be used is up for debate and decisions. It might even wind up the other way around that using AI is the special case - who knows.
> what AI should or should not be used is up for debate and decisions.
Of course; but it's very hypocritical to raise these feelings only when mathematicians are affected, whereas all the above professions are just told to adapt to the new way of things.
For sure though, translators don't have the same clout and social status as mathematicians do.
In some form and shape, yes. Humanity's creativity is a lot of marginal copying and remixing.
But obviously, it adds up to something greater than went in; in aggregate, our contributions are something to awe.
But my point is, if you zoom in at the marginal, incremental contributions of any individual human in this process, it's really hard for me to say LLMs are not at the same level already.
On this topic, people like to compare LLMs to Einstein, but as far as I know, Einstein did not zero-shot special relativity in an afternoon. He built it up incrementally over time, it took him three times longer than the time between first ChatGPT release and today, and it depended on centuries of prior art, culminating in the right observation and right notation being available to him in his moment of greatness.
What would you accept as evidence there? Are, for example, the first names/words for colors from scratch?
So your view is that everything was there at the creation of the universe (it's a possible view, of course)? Or are there any "things" that can create ideas from scratch?
Recently I watched a documentary on the tanzanian Hadza tribe, one of the last hunter gatherer tribes on Earth. Their language is a distinct click and pop language and they regularly imitate animal calls (monkeys, baboons, birds) when they hunt but also when they communicate with each other, tell stories etc.
I think it's not impossible that words evolved as adaptations of the environmental sounds with which our ancestors lived. The human creativity producing DNA is also a remix of preexisting molecules formed under evolutionary pressure, so the view that it's turtles all the way down, unintuitive as it is, may not be so indefensible after all.
I mean what is your criterion on invention here? On the one hand, each specific word could be seen as a new invention. On the other hand, all languages basically correlate strongly with the environment of their users - it's why LLMs turn out to be universal translators - and pattern-matching is hardly an invention, isn't it?
I don't know, but my current intuition is that invention = more or less preexisting ingredients + some tiny spark of creativity ("connecting the dots") rather than this grand myth of "creating something from nothing". And then the tiny spark could itself be a remix of previous tiny sparks (connecting the dots between dot connections, a bit like maps between n, n+1, n+2, ...-dimensional manifolds), and so on.
> and pattern-matching is hardly an invention
Exactly. That's what I meant with turtles all the way down, until the tiny spark disappears into a dot and it turns out that creativity is just an illusion - akin to how nobody would call the gene (the spark, the pattern matching) formed by evolution (the brain connectome) "creative".
My view is that LLMs meet the standards by which we judge human creativity/inventiveness, and thus that one cannot claim LLMs "just repeat, never invent" without the same being true about humans.
At least in Sid's case, it went from the oncologist saying "I have no more drugs I would recommend, no trials available" (slide 7) to "I currently have no evidence of disease" (slide 18). I don't know beyond that or beyond Sid's case - or a similar story of an Australian who treated a cancer tumour their dog had with a similar AI / personalized vaccine process.
My understanding of what Sid's describing is that you do RNA sequencing, a whole genome sequencing, feed that into frontier AI (if it will still let you), and somewhere along the way give the information the AI finds to people who can use it make a personalized mRNA vaccine, specifically for you and your cancer.
Another link here about Sid's case, it explains it didn't go through trials: "made possible through a compassionate use allowance from the U.S. Food and Drug Administration (FDA)".
reply