AI Researchers' Views on Automating AI R&D and Intelligence Explosions
Many leading AI researchers expect AI development to exceed the transformative impact of all previous technological revolutions. This belief is based on the idea that AI will be able to automate the process of AI research itself, leading to a positive feedback loop[1]. In August and September of 2025, we interviewed 25 leading researchers from frontier AI labs and academia, including participants from Google DeepMind, OpenAI, Anthropic, Meta, UC Berkeley, Princeton, and Stanford to understand researcher perspectives on these scenarios. Though AI systems have not yet been able to recursively improve, 20 of the 25 researchers interviewed identified automating AI research as one of the most severe and urgent AI risks. Participants converged on predictions that AI agents will become more capable at coding, math and eventually AI development, gradually transitioning from ‘assistants’ or ‘tools’ to ‘autonomous AI developers,’ after which point, predictions diverge. While researchers agreed upon the possibility of recursive improvement, they disagreed on basic questions of timelines or appropriate governance mechanisms.
Introduction. An ‘intelligence explosion’ is a hypothetical scenario introduced by I.J. Good in 1966[2], in which AI systems become capable of recursively improving their own capabilities (designing AI successors that can design AI even better). Yudkowsky [3] defines an intelligence explosion in terms of the return on cognitive investment, when investing cognitive resources in improving cognition accelerates the returns from those investments. Recursive improvement need not look like conventional AI R&D, future AI systems might be able to directly plan and program recursive improvements, or improve by improving their physical hardware. However, a critical capability milestone may occur when AI systems can do the work of today’s AI researchers, or AI R&D, at which point, such an AI could improve the development of it’s successor. In August of 2025, a version of OpenAI’s GPT-5 model won a gold medal at the International Math Olympiad[4].
Discussion / Conclusion. An epistemic divide emerged between frontier AI companies and academic institutions regarding ASARA development, reflecting deeper differences in organizational culture, incentives, and proximity to cutting-edge capabilities. All participants at frontier companies demonstrated active engagement with ASARA scenarios and reported regular internal discussions about recursive improvement dynamics. They note that these conversations are encouraged. For instance, Participant 6 said, “I’ve been impressed so far, at least internally, with the discussions people have had that especially that the leaders are aware of these worries and talk openly about them, and bring them up on their own and encourage us to think about them." By contrast, academic participants often expressed limited consideration of these possibilities, with one noting: “I don’t even think the majority of AI researchers even think about this problem” (P17, PhD Student). Participant 7 also brought this up when they said, “When I ask questions like this, even if people in the labs are not believers in AGI or something, these are still discussions that we have at work. They’re encouraged by the top...
Lines of inquiry this paper opens 24
Research framings built by reading the notes related to this paper — the questions it feeds into.
What human oversight must AI research systems have?- What makes research tasks verifiable enough for AI automation?
- What governance approaches do researchers propose for automating AI research?
- How should labs measure their own AI systems' impact on research workflows?
- Why do AI researchers consider automating research itself a severe risk?
- How do researchers justify withholding AI from accountability-heavy work?
- Can third-party evaluators embedded in labs measure AI-led R&D work reliably?
- Do humans or AI perform better at different research stages?
- Can standards enforcement prevent any single nation from accelerating unsafe AI research?
- Does slowing AI development reduce risk or just delay it?
- Have AI researcher timelines shifted based on recent capability evidence?
- How does automated R&D affect the efficiency of the research process itself?
- What timeline disagreements emerge among researchers about autonomous AI development?
- How does automating research tasks change the pace of AI progress?
- Can partial automation in software research alone trigger runaway AI progress?
- How do AI researchers currently estimate timelines to artificial general intelligence?
- How should superintelligent AI systems be aligned during rapid capability gains?
- Do efficiency gains in AI-assisted development stem from better tools or autonomous improvement?
- Are AI companies already implementing slowdowns in development as claimed?
- Which bottleneck in the R&D feedback loop is the weakest link today?