”An external incident response firm reviewed PingCAP’s production systems and found no evidence that the attacker ran commands or maintained persistent access in the systems reviewed”
The evidence is in the blog post explaining how they did it. However seeing at how bad their operations were I doubt they have evidence of anything themselves
> We even stood up a replication feed of their own control-plane database using credentials committed in their own repositories — GCP service-account keys and an Azure service principal — and let their infrastructure deliver it to us. Every six hours, for the last twenty-two days of the campaign, from 8 September to 30 September, that pipeline faithfully re-exported every table of the operational heart of TiDB Cloud — tenants, users, API keys, cluster credentials, changefeeds, migrations — and pushed it out. Eighty-eight consecutive exports, not one detection event.
Haven't tried it yet. It looks like sol is a hair more expensive on input, a hair less expensive on output ($2/in, $10/out per 1m, K3 = $0.82/in, $13/out per 1m).
Technically kimi k-3 weights license is not open weight (it has a lot of restrictions). I would classify it as ‘weight open’ similar to the bsl and fsl ’source open’ licenses.
Off topic:With sol pricing drop tbh kimi k3’s value prop has not been that great. For our internal use case/testing/benchmarks sol come out with way better quality and much cheaper costs.
Kimi really needs to drop their pricing (I heard it’s set by them across all the neoclouds)
Sol is at 2/10 vs kimi’s 3/15
Sol pricing dropped but so did the quality few days ago. I wonder when these companies are sued for making the terms from their side to go downwards while taking the same subscription cost.
They're impossible to objectively track by design and intent. Intelligence is such a nebulous target that one can find any number of metrics to support any premise: that the model is smarter, and that the model is dumber. We use benchmarks to attempt to standardise comparisons but these are quickly ingested into the training data and then become effectively useless. You might have heard the term "bench-maxed." Meaning that a benchmark has an effective lifespan in months.
They've been using the Pelican test in the /r/Codex subreddit with some success. One major finding is that OpenAI has been silently degrading the model while charging Astra prices. Another finding is that even when the model has not been silently degraded, pelican quality is significantly lower. Sometimes comically so. The general consensus right now is that the new GPT-6 Sol model is an updated Terra model. Many intelligence metrics are roughly similar. Meaning the most recent model updates were an attempt to rebalance compute rather than improve intelligence.
Ultimately I've never seen users as upset about GPT-6 Sol/Luna than I have right now. Even Astra has been noticeably degraded for me and everyone else I have asked. This is compounded by the fact that Opus 5.5 is a generational improvement at an affordable price. There is currently no competition.
Sure they are, but do you think they are continuing training to improve a model after release without bumping the version number, presumably only to game the benchmarks?
I am glad I am not the only one to notice. I feel like I've gone back to Sonnet 4 levels of incompetence!
With Sol 6 I am back in a world where the model writes bad code because it is lazy ("You're absolutely right, I did not [do it properly] because I did not want to edit [a normal amount of files]").
- it actually failed to correctly understand a simple English grammar and logical implication of it, then when challenged it admitted its mistake but couldn't explain why it made it.
- for the code I am working on, I asked to create two PRs for the two small features (couple lines of code). It created one in upstream, as intended, and other one in my own fork. Just like that, out of nowhere, and called the job done.
- it said it would ask me to approve/ammend the suggested PR message, it never did and fired off right away
- it keeps forgetting the changes it did itself; no context compaction was used
- it said it tested the change visually, but it did not even try
On top of that, it ignores all of my AGENTS.md, which is short and concise. I mean I point it at ignoring it, it acknowledges and ignores again.
This is astonishingly bad and it is nowhere close to Sol 5.6, or even DeepSeek 4.1! I swear even Gemini 3.8 is slightly better.
To me, Sol6 is what Opus5 was for Claude.
I keep saying that with self-hosting, you at least know what to expect and don't have to trust they nerf their models as they go. I was a skeptic and considered nerfing a conspiracy theory, but at this point with enough experience, I have experienced enough to fully see this being a thing.
Probably only a matter of time before some class action happens.
100%, I wish for a legislation which would require the providers to give you at least a unique hash identifying the model (and infra running it, if it affects output) - such that the same hash must give the same output given the same seed. Right now it's all just vibes
I was surprised by that. I run my benchmark [1] every couple of days and was sure this model will be ath the pareto frontier, if not THE pareto frontier. But no:
Ember isn't picked yet. In planning, Opus 5.5 wins under the planning weights. In code, GPT-6 Sol dominates it: also 10/10, but with a higher quality score and a lower estimated cost. Ember has no intelligence index, so its starting score is only 0.73, which holds its 10/10 down to 0.954 against Sol's 0.975.
Agreed, I think the only place where it’s still interesting is ui design. Visually kimi and muse feel much nicer than frontier models to me, but maybe it’s an artifact of everything terrible being Claude Design
Wow, this was really fun to play around with! Amazing execution and launch!
Quick feedback: it’d be great if the page updated when the URL changes, so you can explore different positions without reloading.
Right now, if I change the URL/hash of the cube position (eg: incrementing by 1 https://everycube.alen.is/?view=3d#5319001606686939024
), the app doesn’t update and I have to open the new URL in a separate tab to see the new position. [Chrome 150 on Mac OS X 10.15.7]
i’d like to revise my earlier comment: 2022 may have been the last time we had pure humans win a Fields Medal.
I’m fairly certain this batch's winners used LLMs for research, lit-revews, reviewing work, and calculations... perhaps not enough to count as a co-author, but still enough to handle a lot of the grunt work.
Who would have imagined the pace of progress in LLM-powered math..
- winners in the 30s were the last time we have pure human to win (before computer)
- winners in the 70s were the last time we have pure human to win (before internet)
- winners in the 90s were the last time we have pure human to win (before search engine)
Why can't we treat LLMs as just another tool like computers, search engines, computing libraries? Why do people keep trying to anthropomorphizing these binaries?
People in the 1800s used to win awards and acclamation by simply hand-cranking numbers for popular calculations (Pi, error functions, etc.) and printing them in a book. This will just be the same thing.
But it's not the same thing. I went through this conversation between Terry Tao and ChatGPT about the Jacobian Conjecture counterexample [0] and it looks a lot more like a conversation between peers than him using a tool.
Well, how many humans do you think are able to prompt this to the LLM?
"The homogeneity in x is an intertwining between a dilation (x,r,u) to (lambda x, r,u) and a dilation (P,Q,R) to (lambda^-2 P, lambda^-1 Q, lambda R) which seems to collapse the 3d jacobian to a sort of twisted 2d jacobian. Is there a general theory of such twisted jacobians and do you have any sense why those particular dilation weights were used?"
"I can see why the five-dimensional Jacobian has a nice monomial form in rho. Why does this make the three-dimensional Jacobian after restricting to c_2 = rho = 1 and eliminating the delta, eps variables also a monomial (now in x)? Is there some block-diagonal structure or something in the 5D Hessian that allows for a nice reduction? I would have expected some sort of Schur's complement type operation to appear."
I, and probably most people on here, won't be able to get the LLM to write such a detailed conversation, because we are not experts in this field. They are tools.
Because the 4 prompts were actually still prompting for the original prompt (ugh.) in that the computer did not come up with a "complete unconditional counterexample" and the human expertise is required to discern that.
There was an article on Rudi Mandl who pestered Einstein into working on gravitational lensing.
This happened because Rudi was persistent and Einstein was kind enough.
In future, citizen scientists have a chance to work on their ideas using AI, eventhough they don't have the deep domain skills. Of course, an expert would still need to review it as usual. But it is a useful tool to democratize science further.
I don't get the argument. It sounds to me like you're saying that because my own chess skills are insufficient to allow me to have a proper game with a grandmaster and I end up losing on move 10 because after a stupid blunder, then I should conclude that this grandmaster is just a tool?
"Looks like" being the operative keyword there. Do you feel like you're having a conversation with a peer when you prompt an LLM in the topic you're an expert of? For the love of God, I'd hope not. The whole point is that, even though these things are really good at generating what looks like human output, they are still just regular software algorithms.
> Do you feel like you're having a conversation with a peer when you prompt an LLM in the topic you're an expert of? For the love of God, I'd hope not.
If I were to anthromorphize my experience with frontier models, it would be as a mentally challenged child with complete memorization of an encyclopedia and thesaurus. It has the ability to rapidly experiment and potentially succeed at tasks through trial-and-error, but not without constantly corralling it in the correct direction because it would stick a fork in an outlet if unattended for five minutes.
Tao's chat certainly doesn't give me a vibe of talking with a peer. Do you much often have conversations with colleagues where you write one sentence and then get five pages dumped on you, repeating ad infinitum? LLMs can be useful for rubber ducking, and sometimes the plausibly-related word-soup it generates so quickly will help your thinking along faster, but that's not the same thing as a genuine conversation. And it mostly looked like Tao was using it as an advanced calculator, firing off his own ideas for it to quickly do calculations on. I don't know why we need to anthromorphize these tools just because they generate sentences.
If I were to anthromorphize my experience with frontier models, it would be as a mentally challenged child with complete memorization of an encyclopedia and thesaurus.
Memorizing an encyclopedia is not going to help you solve open high-level math problems, is it?
> It has the ability to rapidly experiment and potentially succeed at tasks through trial-and-error, but not without constantly corralling it in the correct direction
It kind of could, given the above. Part of an LLM's advantage is that, much like a calculator or a Chess engine, it can iterate over a finite problem space far, far faster than a human can. That much is expected of a useful computing tool.
It is worth noting that we know literally nothing about how the counterexample was achieved. Technically speaking, the person who tweeted it could have solved it themselves with zero LLM assistance and then attributed it to Fable to boost their IPO and ensuing payday. I'm not saying that's actually what happened, but it's hard to draw conclusions without any transparency about the degree of human involvement.
My daily experience certainly does not reflect that of prompting a superhuman intelligence when it routinely flubs commands and destructively drops the PATH of its vm, or bypasses an instruction about passing tests by burning millions of tokens constructing a completely new test suite that rubberstamps its own work when it can't pass the real tests.
If you say "find some unsolved graph theory problem and counterexample for it" and LLM actually does it, is it really you that solved the problem? That's the difference vs other tools.
It's not, it's basically something that happened. One unsolved problem was solved with the only human input being telling LLM to work harder and to actually solve it after few failed attempts.
Hey @netvarun, sorry about that. I originally read it in one of your comments, and I should have credited you when I mentioned it. My apologies.
There was never my intention to plagiarize. That's also why I wrote "this prediction" rather than "my prediction" but I still should have mentioned the source. My apologies again.Hope you understand.
What do you base that certainty on? I'm not saying you're wrong, but I am also skeptical you are correct and since it is four people you can probably look into if any of them have talked about it instead of just deciding that what you think is true.
“The Anthropic team for their incredible coding models (Fable-5 wrote every line of code in this project), and the Claude Code harness.”
Source: the repo
[0] https://news.ycombinator.com/item?id=50173716
reply