> There were some technical findings along the way—for example, Kavish found that DDPM, while theoretically ideal for denoising in diffusion models, diverged, and that flow matching performed much better.
When they say "theoretically ideal" they mean a specific thing: the DDPM loss is information theoretically the number of error correction bits needed to recover an image. At very low noises, this quantity goes singular. This is not actually a problem—you should bound the log-SNR of your schedule by the image quantization level (usually 9 bits) anyway, and a trained bound lands within 10% of this.
Flow matching does not diverge because it removes the log-SNR rate from the loss. Since the rate can span several orders of magnitude over your schedule, you should importance sample during training. My guess is Kavish forgot to do this, leading to divergence.
Chess has always been "just a game". Sure, it teaches valuable life lessons, and it is fun to defeat your opponents through superior training and strategy, but it is "just a game".
Mathematics was historically "just a hobby". Much theoretical work is still "just a hobby". But, for the past few hundred years, humans that better understood the theory could find very profitable applications.
That is no longer true. Computers have solved mathematics, like they did chess two decades ago. A human will never find a better application than a computer, just like they will never find a better chess move. Yet people still play chess, and people still make money tutoring chess.
That is the future of humans in mathematics. Mathematics will become mathematics competitions. It will be just another intellectual sport. Sure, a sport that teaches valuable life lessons, but still "just a game". If you are going into mathematics today, your future career is as a competition mathematics coach.
What do you think mathematics is? "Just a hobby"? Developments in mathematics made possible the industrial revolution, the information age, everything! Do you think the world will be the same when math is "solved"?
Maybe don't take words out of my mouth. I studied math when I was younger, and got to the top 0.001% in mathematics competitions. Why did I study so hard? In order:
1. I recognized that understanding math had incredibly profitable applications.
2. It was a sport.
3. It was a hobby.
Do I think the world will look the same when math is "solved"? Absolutely not! And we're seeing it transform right before our eyes. I routinely ask Claude, "here's the general idea for a math/cs/ai problem I'm working on, can you help me figure out the maths? oh and then implement it too."
Could I figure this out myself? Of course. Having the idea is halfway to formalizing it in math. But I remember even last year working on GPU kernels by hand and having a bunch of little arithmetic or indexing errors burn me, because I'm just slower and less perfect at doing the math correctly. Now I basically just tell Claude, "go optimize a GPU kernel for this problem". I still have to tell it what to look out for (architectural, algorithmic, memory improvements), but I'm sure the next model won't even need that.
Who decides whether or not there was intent behind your words?
What do we do when people disagree about the intent behind a word?
Who decides what is antisocial?
What do we do when people disagree about what is antisocial?
I don't believe there are neat clean answers to any of these. The closest we'll get will involve mutual respect, assumption of good faith, and dialogue. We can't have any of that if our starting point is, "I believe your words are secretly not what you say they are" or "I will hold you responsible for anything bad that anyone does that might be tenuously connected to your words"
Yes, people can always claim that wasn't their intent, but what you see quite often nowadays is people openly saying their intent is to get people to take action. "I wouldn't personally... but I have massive respect for those who do. We need more people with the balls to do something about it."
This is probably less restrictive than you are advocating for in your previous comment: as long as there is plausible deniability, the speech goes free.
I skimmed through the book, and it's lacking the information theory foundations. For example, "trust region methods" come from maximizing the policy's relative entropy (to a reference policy) under a tournament system where high-scoring agents are exponentially likely to survive. In general, a reward is the negative bits it costs an environment to propagate an agent (multiplied by some temperature).
It's just another way to frame it. It's as foundational as the many other ways to frame it. I'm not aware of any major insight you get specifically from this framing. Is there one?
Didn't Standard Intelligence release a duplex model two years ago? Sounds disingenuous to market this as a new generation of voice models, when it is really OpenAI finally catching up to the current generation after two years.
When I was a young man I took three philosophy courses from a very old man who, when he was a young man, was the dissertation director of Alan Turing. The latter, by the way, was an habitué of the seminar of Wittgenstein.
Connections are far more important IMO. The opinion itself is there for the plebs so they don't revolt when the high-IQ trickery flagrantly mismatches the plain language of the constitution. The courts are really the main thing nowadays that can provide legitimacy to the acts of the state since it doesn't follow the people or the documents authorizing it.
Nondeterminism is also a feature, not a bug. If you don't want people to optimize against your filtering process, you have to make it somewhat nondeterministic. For example, better candidates are exponentially more likely to pass the filter, instead of a hard cut-off at the top-100. Then it becomes no longer worthwhile to Goodhart the filtering process, because it barely increases your chances and there are so many more places you can use your time better.
> If you don't want people to optimize against your filtering process, you have to make it somewhat nondeterministic.
I'm sorry, I'm not following this at all. When you say "better candidates are exponentially more likely to pass the filter", we're still are talking about a metric, yes? A metric that can be optimized? Why would switching from a hard cutoff to some sort of stochastic filter weighted by this metric discourage optimization?
It is the case that most metrics are logarithmic: it takes just as much effort to decrease one bit of error as the next bit. So
log(score) \propto (opportunity cost) \propto expected gain
Thus, for them to be agnostic, you should filter candidates proportional to their log-score on the metric (where 0 is a perfect score). Because generally applicable skills are generally applicable, they will still benefit from improving those, they just no longer benefit from adversarial optimization, unless your score function looks very similar to others who have not adopted this filtering process.
The issue with a hard cutoff is that people near the boundary are extremely incentivized to adversarially optimize, as it is usually cheaper than working on generally applicable skills and actually pays off for them. You see this phenomenon on AoPS where (esp. Californian) students talk about grinding for MATHCOUNTS instead of learning calculus.
> There were some technical findings along the way—for example, Kavish found that DDPM, while theoretically ideal for denoising in diffusion models, diverged, and that flow matching performed much better.
When they say "theoretically ideal" they mean a specific thing: the DDPM loss is information theoretically the number of error correction bits needed to recover an image. At very low noises, this quantity goes singular. This is not actually a problem—you should bound the log-SNR of your schedule by the image quantization level (usually 9 bits) anyway, and a trained bound lands within 10% of this.
Flow matching does not diverge because it removes the log-SNR rate from the loss. Since the rate can span several orders of magnitude over your schedule, you should importance sample during training. My guess is Kavish forgot to do this, leading to divergence.
reply