Ryan Greenblatt – What happens once AI can automate AI research?

Paper · Source
Frontier AI Risk & RSI

Source: Dwarkesh Patel · 2026-08-11

This might be the most important question in the world right now – whether within a year or so of achieving human-level intelligence, you slingshot towards having 10s of billions of superintelligences, each of which is dramatically more competent than human experts across all fields.

If, because of RSI, we got a jump as big as GPT-3 to a Mythos (i.e. 6 years of AI progress) within a single year of achieving AGI, then the thing we get there at the end of that year is definitively and wildly superhuman.

We hashed it out, and I think Ryan made a pretty good case that this kind of speedup is plausible. FWIW, Ryan’s median for when we automate AI R&D is 2031.

We then discussed the alignment implications of this scenario. Who should these superintelligences be aligned to? In the future, our capacity to steward our votes and our capital, and to make sense of what’s happening in the world, will all be titrated by superintelligences. And I worry that specs like the Claude Constitution are not shaping these ASIs to truly be my personal advocates and guardian angels.

I think once you have AIs which are roughly matching the top human experts in AI R&D, that could kick off a feedback loop where the AIs are doing AI research. That produces smarter AIs. That feeds back in. That feedback loop could be strong enough that you end up with a lot of progress in a short period of time. Maybe my median expectation is something like four or five years of AI progress in a single year. This requires really overcoming a huge amount of diminishing returns in research and basically doing the equivalent of the progress we would have gotten after a really large compute scale-out. So this is a pretty impressive, big thing.

It’s worth keeping in mind that five years of AI progress, four years of AI progress, even three years of AI progress, is really a lot of fucking AI progress. A little over three years ago, GPT-4 had come out. Right now, of course, we have Mythos 5 or whatever, and maybe a somewhat better model that Anthropic has internally. That is just a huge amount of progress in a bit over three years. If we’re talking about five years, then maybe we’re talking more about a jump from GPT-3 to Mythos 5 or whatever.

I think this argument has three different parts. Now I want to evaluate each one of them. First is the argument that AI R&D is very verifiable. Second is the argument that if you automate AI R&D, you could get four or five years of progress in a single year. Third is the argument that what comes out the other end of four or five years of AI progress at the current pace, starting at the point whenever AI R&D is automated, is an AI where you can drop it on the job at basically anything you can imagine.

Basically, there’s this whole class of containerizable, verifiable, small-scale AI R&D tasks that we can aggressively RL the AIs on. Already companies are presumably doing some RL on these sorts of tasks, and you could just keep scaling that up, keep making more of these small-scale AI R&D tasks, and then the AIs could keep getting better at this. Implicitly, I’m claiming this will transfer to extremely load-bearing aspects of AI R&D. But maybe let’s stop there for a second and then get to that part.

So let’s talk through what this concretely looks like. You can imagine that we have GPT-7.5. We say, “GPT-7.5, we want to make you so good at AI R&D that you help us train GPT-9.”

Lines of inquiry this paper opens 24

Research framings built by reading the notes related to this paper — the questions it feeds into.

Can AI research automation sustain progress through accelerating feedback loops? What human oversight must AI research systems have? What limits recursive self-improvement in autonomous AI systems? Do individually safe AI actions create unsafe outcomes in integrated systems? Does AI-assisted research sacrifice exploration breadth for productivity gains? Do AI coding tools measurably improve developer productivity and code quality?