Why do LLMs generate ideas the research community already explores?
LLMs inherit the distribution of published literature, concentrating ideation where researchers have already invested conceptual effort. This raises a core question: can AI ideation complement rather than duplicate human research directions?
Scientific discovery is constrained not only by what is true but by what is cognitively available to the researchers currently in a field. Many directions are coherent in light of the literature yet unlikely to be proposed, because no existing community occupies the right combination of concepts, methods, and intuitions. The paper's key move is to distinguish two quantities that are usually conflated: whether an idea is scientifically plausible, and whether the current community is likely to think of it.
Modern LLMs are strong at the first and structurally biased on the second. Trained on the literature and prompted through language, they inherit the distributional shape of that literature — they recombine high-density regions, searching where the community has already placed conceptual mass. The result is ideation that is fluent and useful but not complementary to human researchers. The method targets the complementary region — the "alien space" — by decomposing papers into idea atoms, then ranking atom combinations that maximize a coherence model while minimizing an availability model (whether any author community is positioned to produce them).
This is the sharpest available framing of why LLM ideation underwhelms, and it complicates Why do LLMs generate novel ideas from narrow ranges?: diversity collapse is not just homogeneous sampling, it is sampling from the community prior. The honest caveat the paper keeps is that alien ideas are not guaranteed breakthroughs — most research ideas fail. But it relocates the value of AI ideation from "generate plausible ideas" (which the community can already do) to "generate plausible ideas the community structurally cannot reach."
Related concepts in this collection 3
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Why do LLMs generate novel ideas from narrow ranges?
LLM research agents produce individually novel ideas but cluster them in homogeneous sets. This explores why high average novelty coexists with poor diversity coverage and what it means for automated ideation.
diversity collapse reframed as sampling from the community availability prior
-
Why do LLMs generate more novel research ideas than experts?
LLM-generated research ideas are statistically more novel than those from 100+ expert researchers, but the mechanisms behind this advantage and its practical implications remain unclear. Understanding this paradox could reshape how we use AI in creative knowledge work.
alien-space targeting is a constructive response to the novelty-without-evaluability problem
-
Can LLMs generate more novel ideas than human experts?
Research shows LLM-generated ideas score higher for novelty than expert-generated ones, yet LLMs avoid the evaluative reasoning that characterizes expert thinking. What explains this apparent contradiction?
coherence vs availability is a cleaner decomposition of what ideation actually optimizes
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- The Ideation-Execution Gap: Execution Outcomes of LLM-Generated versus Human Research Ideas
- The Alien Space of Science: Sampling Coherent but Cognitively Unavailable Research Directions
- Agent Laboratory: Using LLM Agents as Research Assistants
- Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers
- Has the Creativity of Large-Language Models peaked? —an analysis of inter- and intra-LLM variability —
- Unlocking Varied Perspectives: A Persona-Based Multi-Agent Framework with Debate-Driven Text Planning for Argument Generation
- Interesting Scientific Idea Generation Using Knowledge Graphs and LLMs: Evaluations with 100 Research Group Leaders
- AI Can Learn Scientific Taste
Original note title
LLM ideation inherits the literature's distribution so it searches where the community already is — novelty requires targeting coherent but cognitively unavailable directions