> Using Starlink, a civilian commercial service, for offensive drone strikes to kill people has never been allowed, nor should it be.
Starshield, a different system, is controlled by the US government. What they decide to do is up to them.
> In a response to Musk on Sunday, Ukrainian presidential adviser Serhiy Beskrestnov said: “It’s very strange that Elon Musk doesn’t know about the restrictions on Starlink in Ukraine.”
> “Starlink does not work over Russian territory,” he said in a post on Telegram. “So we PHYSICALLY CANNOT USE COMMERCIAL STARLINK for deep strikes and mid-range strikes.” [1]
But you already have Claude-directed videos that utilize OP's manim project (which he uses himself to do 3b1b) that anyone else can create. Grant open-sourced manim and now anyone can point a model at it to create their own 3b1b-like videos. They're popping up all over the place as Opus 5.5 demo videos just in the last few weeks. So... AGI?
I guess I wasn’t clear, but using manim to make videos is not the hard part. The hard part is making them engaging and enlightening, and whatever else you want to attribute to 3b1b videos which make them a joy to watch and learn from.
> Why not pay taxes and let us, the people decide how to use it instead?
Maybe because the current administration has no interest in doing this and Anthropic doesn't want to wait for this cluster-f of a situation to sort itself out. Even if the current admin was for progressive taxation it might take them years to sort through all the bureaucracy to use this money effectively which might be much too late.
> Looking up at the stars in wonderment, or enjoying a complex puzzle game, or being inspired by an encouraging teacher, and dreaming that one day an unfeeling corporate robot[1] will burn a rainforest to compile a list of 700 highly obscure barely-readable facts that might be true that nobody can understand as part of a marketing gimmick to raise hype for an IPO[2].
But if I am going to critique it's main point it would be this part:
> in the pursuit of advancing humanity’s knowledge as quickly as possible, many academics will recoil in anger at the suggestion that it is worth spending any time making results understandable to humans. It is inefficient to explain or check a result when those resources can be spent generating another.
While obviously ridiculous, I think this does not go far enough. The problem isn't that people don't care and resources should not be spent to explain AI results to human minds, the problem is that AI will soon be better at explaining itself than humans. The puzzle then changes from can AI formally verify and explain a proof to: do human minds trust the proof. Do we believe that the proof is correct and not an elaborate AI hallucination using tricks that are beyond our understanding. In order to trust it a formal verification is not enough unless it is human understandable. In fact human understandability is the only thing that matters. That is something that AI can never do on its own.
>In order to trust it a formal verification is not enough unless it is human understandable.
The whole point of formal verification is to remove the necessity of understanding most of the work. We only need to trust the formalization of the problem statement, the proof verifier, and the formalization of the result. We already have proof verifiers that humans understand to a high level, and the formalizations of the problem statement and the result are much simpler than the full proof.
I'm confident that the Navier-Stokes equations can blow up because I'm confident that Lean works correctly, and I'm confident that the mathematicians verified the problem statement and result correctly. This is despite the actual proof not being human verified yet (as far as I know), and despite the "human-readable" version not actually corresponding to the formal proof. The formal method still worked.
Some mathematicians are not confident in the NS verification because it is non-trivial to tell if the Lean proof matches NS. I would argue this requires "human understability" to make this claim. The Lean proof is a very useful tool, but it does not replace human understanding.
This is patently absurd. Science is based on reproducible experiments. If you can't reproduce the experiment and are asked to blindly believe instead, we're back at religion.
Beneath the sarcasm it's making a rather simplistic argument that OpenAI cares little about human understanding of the results or their correctness.
The author appears to argue that this in service of a pointless goal, that no one will even care to read the results, and that all this is taking away from human creativity and removing the fun in scientific discovery.
I don't see a convincing argument here that OpenAI does not in fact aim to further humanity's understanding of mathematics, or that using AI for scientific discovery is comparable to automating away the fun parts of life.
The sarcasm is rhetorically strong, but profound? I don't think so.
OpenAI certainly does not really care about human understanding of these math problems. We've always said solving the hardest math problems would be a remarkable demonstration of AGI, and here is their demonstration. But it's not really about the math, this is about the power of LLMs and the path to AGI/ASI. Solving math problems is just a parlor trick for OpenAI.
Saying that OpenAI does this only to show off their models, rather than also advance mathematics or at the very least their model's mathematical capabilities, seems speculative.
I understand OpenAI published write-ups for each problem.
It's unclear to me what other expectations mathematicians have that would demonstrate OpenAI's commitment to advancing mathematics.
Last I heard they asked to completely stop solving math problems with AI, which seems just ridiculous.
It may not be the most prescient or innovative sci fi series, but I read The Three Body Problem a few years ago and it's really stuck with me. Specifically the description of how humanity reached an inflection point in math & physics research that almost immediately unlocked novel innovations in applied science ... which then made space exploration & settlement practical. Moreover, that the antagonist society in the series decided their absolute best bet at avoiding complete annihilation was to hamstring humankind's scientific research -- and that's it.
It feels like frontier models are starting to approach an asymptote where what you suggest could become reality, and we'll begin seeing not just AI research but AI-designed & led applications of that research into fundamental parts of our society & culture. For better and for worse.
I'm all for advancing human understanding, even if it might lag behind machine understanding, but there is a way to have machines prove they understand something we don't: ask them to build something functional that we can't. If your hypothetical future comes to pass, and we ask the machine to build a cancer-curing healing ray, and it works, I'd say that's its own win. Then we can spend however long it takes trying to understand how it works. It's depressing and disempowering in a way, but I think overall I'd be happy with that future.
I once visited a town in Spain over which an ancient Roman aqueduct runs, elevated above most of the buildings, even today. It carried an essential ingredient foundational to life in the area, but after the fall of the Empire and the deaths of the architects and engineers who could understand it, the aqueduct stood in the midst of people who could not have repaired it. Relying on mission-critical technology divorced from human understanding is exceptionally risky. Obviously, people have done it before and will happen again, but it makes me nervous.
I don't think the aqueducts were beyond the understanding of those who came after the romans. But the romans were extremely organised and could cooperate.
Chess has already been this way: computers are far better than humans, yet the humans still play and learn/understand new position evaluations from the computers.
There also will never be a Mikhail Tal type of player again and most classical matches at the highest level are just draws now. I'm not sure if you could say AI is a benefit for the highest level but it is an equaliser in the sense that you don't need to pay for a tutor to analyse your mistakes (though they can still be helpful) which helps the average player. Chess is also predominantly for entertainment making it not the best example for AI not replacing people.
> the problem is that AI will soon be better at explaining itself than humans
(I assume you mean explain itself to humans here.)
I keep seeing people saying this as fact. It might, but it might not, and there are various reasons one might be skeptical that it will, using current techniques. Not least among these is that explaining something well to a human requires a degree of empathy and sophisticated theory-of-mind that LLMs have not yet exhibited yet*, and that is an area where humans are uniquely advantaged by being humans.
Someone will almost certainly reply by just stating as fact that they will achieve this goal, or pointing out that past predictions that LLMs wouldn't be able to do particular things have failed, and that's all very well. I certainly wouldn't be rash enough to claim it's impossible, but there are still models of all this available that are consistent with all the successes so far but don't predict "LLMs can do anything and will do everything". We'll find out in due course.
(I should clarify that the benchmark here is good human explanations. Of course many humans are capable of producing "explanations" worse than Markov chain text, let alone LLM output.)
* My idle speculation is that a lack of decent theory-of-mind is why LLMs still suck at using the "it's not X it's Y" construct. I doubt they actually use it any more frequently than humans when writing on the same topics, but they use it wrong and it's jarring. This particular rhetorical flourish works when there's some reason to think that X is an idea that might actually be present or forming in the mind of someone reading the passage (or in some other mind that they in turn might be imagining, eg. a fictional character), and that explicitly calling it out and negating it will have a particular dramatic effect. LLMs probably use it with about the expected frequency, but they use it for that reason (frequency) and then come up with something to put in the X and Y slots. The result often jars because the X it uses isn't something pertinent to the reader's thought process and/or the Y isn't a convincing, persuasive or relevant counterpoint to it. I suspect the result is that they stick out and seem more frequent than they are.
fair points but i disagree. LLMs are very good about explaining and summarizing complicated material. You can test this today with any modern LLM about any topic or paper you like. One thing that makes them so powerful is that they can customize their explanation to the specific querier.
Today, to explain and communicate these math proofs, it makes sense that a human working with an LLM would be the most effective at human explanation. However, given LLMs current abilities with explanation, it seems reasonable that LLMs will become equal to human experts in explanation. Again part of their power is that they can scale to 1-on-1 explanations for the particular audience which humans cannot do.
> "it's not X it's Y" construct.
I agree this is annoying, but this is not a fundamental feature of LLMs, this is "tick" of current models, like the em-dash. There have been countless of these "ticks" that have been refined and this process will continue.
100% agree. In fact, this issue of verifiability is not at all unique to AI. It was a profound and growing issue before the advent of useful AI. Perhaps the issue was best elucidated by Dr. Anthony Fauci when he suggested that we "Believe the science" as a rational for why people should follow his covid prescriptions. I side with those who feel that Dr Fauci was really just trying to do his best. Regardless, with that statement he cut the foundation out from under the whole scientific project! Science (and math) are not to be believed. They are to be tested!
We have moved quite a long way down the "believe the science" road already, and for good reason. Belief is much (!) faster and cheaper than proof, so those who are fed good science(math) and believe it tend to outcompete people who get stuck trying to prove things all the time.
But fraud is fraud regardless of the platform (meat or silicon) committing it. People who believe in frauds often end up worse off for it, though it hard to know which piece of BS has led you astray or who's to blame for it or how to deal with the fall out.
Obviously, we shouldn't believe AI any more than we believe humans. That said, the world was already suffering greatly under the weight of human generated BS long before useful AI came on the scene. Now that AI has reduced the cost of producing plausible BS by orders of magnitude, we may need some new methods for BS detection.
I feel this way about code. Writing things by hand is the quickest way for me to fully understand it, and that is what I am paid to do, to understand it. It will be executed millions and millions of times for every time it is modified, so even if writing it by hand is 10x or 100x as slow, it is still a better payoff.
I think here the question is what is the goal? If the goal of the code is to learn and teach, then understandability is critical. If the goal to efficiently produce useful software, then that is not necessarily the case, because the end-users could not care less if the code is understandable. They just want it to work. And maybe understandability and "working" are correlated, but not necessarily.
> an unfeeling corporate robot[1] will burn a rainforest to compile a list of 700 highly obscure barely-readable facts that might be true that nobody can understand
He'd have more of a point if a huge number of humans weren't occupied doing exactly this, and much worse on top.
the problem is that AI will soon be better at explaining itself than humans
But will it? What's the trend on the metrics for this? Everything else has metrics, why not this?
I mostly work on getting ChatGPT to come up with algorithms, which it is moderately good at. When I ask it to summarize it's findings, the results are less understandable to me than what it said originally.
I don't believe that it's always getting better at everything everywhere. It's no doubt getting better at many things but even there it seems clear the rate vary.
Understanding math isn't just about "having a good explanation ready".
People fundamentally misunderstand the nature of mathematics: it's not "flat" like most of common people's daily pursuits. It consists of building upwards, on top of previous concepts.
In other words, AI is like an oracle: it can tell you all kinds of stuff, but since you probably don't understand, you won't act properly.
There are plenty of things we use and found useful before we understood them. We knew how to make it, how it works was for the most part inconsequential.
Well, we can try tying it to ground truth. AI does the math for magnetic field containment? Test it. AI predicts a particle? Test it. AI says P=NP? Ok, factor this in polynomial time.
And if it fails, and AI goes, "I don't know what happened. I relied on these papers" then I guess one of those must be wrong.
I'm not saying it'd be easy, but I think AI can also find the pieces of the puzzle that broke a pyramid of "discoveries" culminating in a disproval,
And the only other option is to simply never understand as much. AI has the potential to understand things humans cannot. Not necessarily a strict superset.
Humans can deal in 5+-2 things at once. AI doesn't have to, and can cut through leaky abstractions.
If AI says a new invention will do X, and it doesn't, it'll say "you're absolutely right" and try again until it does the thing.
This is interesting but sounds insane to me. At my startup the code has been 100% written by AI agents for the past 6 months. It’s like John Henry vs the Steam engine, time to give it up. The human job which is harder to replace is product manager. Understanding what is valuable to the customer and what the product needs to focus on.
In my experience, AI code is really hard to maintain and overly complex which creates unexpected behavior and hard to catch bugs.
I inherited some vibe coded repos in my company and it's been really painful removing all the unnecessary complexity
I would have agreed completely with this 12 months ago. 6 months ago maybe. but not today. Of course anyone can have AI agents generate a big pile of shit, but that should no longer be the case for professionals. It actually does require a harness where agents are reviewing code with simplicity in mind, and running static analysis checks to reduce complexity.
Often times the constant (2 in this example) is a conjectured minimum, so anything below that is a noteworthy result. Think of it as breaking through some theoretical limit.
Does anyone else remember in the early 2000s the concerns around privacy with cell phone cameras? [0][1][2][3]. How long until people feel the same way about camera glasses?
I was a student who would wear my Google Glass out and about (in the UK) and no one ever said anything negative. Of course, they might've looked at me and thought I was a glasshole. Most people I interacted with were intrigued by the novelty.
This reminds me of during covid, when one of the biggest public proponents of vaccines in the UK was having an affair. The anti-vaxers then, instead of focusing on whether vaccines were helpful, focused on the affair. That allowed them to dismiss the pro-vaccine point of view without actually engaging on the merits of their argument.
In conducting his affair he was breaking the lockdown rules that he had written, which harmed their credibility. The antivax crowd (I am not a member) quite reasonably extrapolated that if the architect of lockdown was not locking himself down then he wasn't too worried about the virus, ergo vaccines were being oversold.
When you are a public figure like that then the personal is political.
I think the thing is engaging with “ML is mostly math and science engineering, AI is getting better at autonomous math and science, if it gets fully super human the rate of change could be mind boggling” is hard to fully dismiss in the same way you can fully dismiss weird Bay Area people. So that’s why we are seeing the pivot towards the personal attacks on these people.
reply