What’s after generative models?

Dwarkesh Patel

Yeah, fair enough. What's after generative models? Before, you were working on reinforcement learning. Is this basically it? Is this the paradigm that gets us to AGI? Or is there something after this?

Ilya Sutskever

I think this paradigm is gonna go really, really far and I would not underestimate it. It's quite likely that this exact paradigm is not quite going to be the AGI form factor. I hesitate to say precisely what the next paradigm will be but it will probably involve integration of all the different ideas that came in the past.

Dwarkesh Patel

Is there some specific one you're referring to?

Ilya Sutskever

It's hard to be specific.

It is insane how secretive OpenAI are being in general. Meta just open sourced their planning AI, how hard is it to talk about work in progress on planning and LLMs combined with each other? Especially when that's the purpose of your non profit.

Supposedly Q* combines LLM with A* search, which I think could be "easily" reproducible by other tech giants, even if they have just a vague idea of what OpenAI is researching.

I created an account just to chime in . Had the same experience happen to me around 30 years ago in a rural area. I think the only reason I was able to distinguish the round shape against the night sky was because there was almost no light pollution. In my case it was a small group of them flying in a v shape Formation just passing by.

Thanks for chiming in. It is oddly comforting to hear that others have had similar experiences. When it was happening, the first words out of my mouth were “You guys see that, right? I’m not hallucinating?” It was so obviously not normal. I rubbed my eyes, questioning whether it was me or not and it was a relief when they saw it too.

The size of the one I saw was large and it was already extraordinary and alarming. A formation would be astounding.