Boundless Socratic Learning with Language Games
An agent trained within a closed system can master any desired capability, as long as the following three conditions hold: (a) it receives sufficiently informative and aligned feedback, (b) its coverage of experience/data is broad enough, and (c) it has sufficient capacity and resource. In this position paper, we justify these conditions, and consider what limitations arise from (a) and (b) in closed systems, when assuming that (c) is not a bottleneck. Considering the special case of agents with matching input and output spaces (namely, language), we argue that such pure recursive self-improvement, dubbed ‘Socratic learning,’ can boost performance vastly beyond what is present in its initial data or knowledge, and is only limited by time, as well as gradual misalignment concerns. Furthermore, we propose a constructive framework to implement it, based on the notion of language games.
On the path between now and artificial superhuman intelligence (ASI; Morris et al., 2023; Grace et al., 2024) lies a tipping point, namely when the bulk of a system’s improvement in capabilities is driven by itself instead of human sources of data, labels, or preferences (which can only scale so far). As yet, few systems exhibit such recursive self-improvement, so now is a prudent time to discuss and characterize what it is, and what it entails.
We focus on one end of the spectrum, the clearest but not the most practical one, namely pure self-contained settings of ‘Socratic’ learning, closed systems without the option to collect new information from the external world.
We set out to investigate how far recursive self-improvement in a closed system can take us on the path to AGI, and are now ready to conclude on an optimistic note. In principle, the potential of Socratic learning is high, and the challenges we identified (feedback and coverage) are well known. The framework of language games provides a constructive starting point that addresses both, and helps clarify how a practical research agenda could look like.