Could AI research start reinforcing itself, each round faster than the last, while progress still looks flat to outside observers?
Can self-amplification onset occur while acceleration remains invisible to observers?
This explores whether AI-driven research could start feeding on itself, with each improvement speeding up the next, before anyone watching can see things speeding up.
This explores whether AI could cross into self-reinforcing improvement, where AI-assisted research makes the next round of research faster, before observers see any acceleration. The corpus says yes, and it gives a clear reason why. What determines whether AI self-improvement actually compounds? frames the tipping point like an epidemic's reproduction number. It compares how strongly AI improvements feed back into AI research with how quickly research gets harder as the easy wins run out. Once feedback outpaces that hardening, the process compounds. At the moment of crossing, though, the growth curve still looks almost flat. The switch is about whether the dynamics compound, not about how fast progress currently looks. That is why the onset can come before visible acceleration, and why it doesn't line up with any particular capability milestone.
This changes what we should be watching for. Most public debate looks for a capability threshold, a model that is 'smart enough' to take off. The reproduction-number framing points to a ratio instead. A modest system in a field where research hardens slowly could cross the line, and a very capable system in a field where research hardens quickly might not. Are AI feedback loops strong enough to sustain recursive self-improvement? arrives at a similar structure from another direction. Net acceleration depends on the *product* of how responsive each link in the loop is: AI improving researcher productivity, productivity improving systems, and so on. Because the links multiply, one weak link holds back the whole loop, and strengthening several links a little each can tip it over. Its estimate is that today's loops are getting stronger but are not yet self-sustaining. By the first note's logic, that is exactly the kind of judgment that is hard to make from the outside, since the crossing doesn't announce itself.
Small-scale self-training experiments show both what compounding looks like and what holds it back. Can transformers improve exponentially by learning from their own correct solutions? shows exponential gains across rounds when a model trains on its own *verified* correct answers. Each round's improvement makes the next round possible. Run the same loop with weak checking and it compounds in the wrong direction: How quickly do errors compound during model self-training? finds small errors snowballing within two or three iterations, so verification quality, not raw capability, sets the ceiling. Can models reliably improve themselves without external feedback? generalizes the point: self-improvement that works usually relies on an outside anchor, such as a judge, a tool, or a human correction. Read alongside the reproduction-number framing, these outside anchors and verification limits act as the hardening term, the brakes the feedback has to overcome.
This leaves a question you may not have expected to ask: what would an early warning look like if not a speedup? The corpus suggests measuring the components rather than the curve: the strength of the feedback, how fast research is hardening, and how reliable verification is. It does not offer an empirical method for detecting a hidden onset in real labs. That gap is worth knowing about.
Sources 5 notes
A recursive reproduction number RAI = χ/aσ determines whether AI-assisted R&D self-amplifies, comparing recursive feedback strength against research hardening rate. The transition can occur before visible acceleration and is independent of any particular capability threshold.
Back-of-the-envelope modeling shows recursive improvement loops depend on the product of elasticities across feedback pathways. Current loops remain too weak for self-sustaining acceleration, though they appear to be strengthening based on data on researcher productivity and system benchmarking trends.
Standard transformers generalize from 10-digit to 100-digit addition by repeatedly generating solutions, filtering for correctness, and retraining—showing exponential (not linear) out-of-distribution improvement across rounds without saturation.
Small inaccuracies in model-generated training data amplify rapidly across iterations, degrading performance unless self-consistency checks filter outputs. The effect stalls improvement within a few steps, setting an error floor based on verification quality rather than actual capability.
Pure self-improvement stalls due to the generation-verification gap, diversity collapse, and reward hacking. Reliable improvement methods succeed by smuggling in external anchors: past model versions, third-party judges, user corrections, or tool feedback.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops
- The Economics of Recursive Self-Improvement
- Recursive Criticality of AI Self-Improvement
- Self-Improvements in Modern Agentic Systems: A Survey
- Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models
- Dream-RSI: Recursive Self-Improvement through Evolving Worlds
- Frontis-MA1: Training an AI4AI Model towards Recursive Self-Improvement in Machine Learning Engineering
- NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness