Can language help agents imagine goals they've never seen?
How might compositional language enable artificial agents to target outcomes beyond their training experience? This matters because it could unlock open-ended exploration without hand-coded reward functions.
Goal generation is a bottleneck for open-ended learning agents. Fixed hand-defined goal spaces limit agents to predefined objectives. Learned generative models of states constrain goals to the distribution of known effects. Neither enables creative discovery — imagining outcomes the agent has never experienced.
IMAGINE models how children leverage language compositionality to overcome this: language descriptions can specify goals the agent has never seen because language is compositional — familiar words recombine to describe unfamiliar outcomes. An agent that has seen "red ball" and "blue box" can imagine "red box" without having encountered it.
Three mechanisms enable this:
- Language as goal representation — descriptions of outcomes serve as target goals during play, enabling targeting of out-of-distribution states
- Social peer guidance — a social partner provides language descriptions, bootstrapping the imagination process (paralleling how children's exploration is scaffolded by adult language)
- Modularity — decomposition between learned goal-achievement reward function and policy (using deep sets, gated attention, and object-centered representations) enables generalization from imagined goals to actual exploration behavior
This challenges purely intrinsic motivation approaches to exploration. Since Can communication pressure drive agents to learn shared abstractions?, language is not just a communication tool but a cognitive scaffold: it structures the agent's goal space, enabling combinatorial expansion of what can be targeted.
The connection to Can LLMs reason creatively beyond conventional problem-solving? is direct: IMAGINE implements combinational creativity (recombining familiar concepts into novel goals) as a concrete mechanism. Exploratory and transformative creativity would require additional mechanisms, but the combinational foundation enables a rapid expansion of the search space.
The social guidance requirement echoes Can human-AI research teams improve faster than autonomous AI systems? — the most effective exploration is not fully autonomous but socially scaffolded.
Source: Agents
Related concepts in this collection
-
Can communication pressure drive agents to learn shared abstractions?
Under what conditions do AI agents develop compact, efficient shared languages? This explores whether cooperative task pressure—rather than explicit optimization—naturally drives abstraction formation, mirroring human collaborative communication.
language as cognitive scaffold, not just communication medium
-
Can LLMs reason creatively beyond conventional problem-solving?
Explores whether large language models can engage in truly creative reasoning that expands or redefines solution spaces, rather than just decomposing known problems. This matters because existing reasoning methods may miss creative capabilities entirely.
IMAGINE implements combinational creativity as goal imagination
-
Can human-AI research teams improve faster than autonomous AI systems?
Explores whether keeping humans actively involved in AI research collaboration accelerates paradigm discovery compared to fully autonomous self-improvement, and what safety advantages this preserves.
social scaffolding as an enabler of exploration
-
Can neural networks learn compositional skills without symbolic mechanisms?
Do neural networks need explicit symbolic architecture to compose learned concepts, or can scaling alone enable compositional generalization? This asks whether compositionality is an architectural feature or an emergent property of scale.
compositionality as a capacity that enables out-of-distribution generalization
Click a node to walk · click center to open · click Open full network for a force-directed map
Original note title
language compositionality enables agents to imagine out-of-distribution goals — social guidance combined with modularity drives open-ended exploration