INQUIRING LINE

Who is working near a research topic may predict its next discovery better than the papers alone do.

Why does expert density around topics signal near-future discoveries?

This explores why knowing *who* is working near a topic (which scientists, with which expertise and collaborators) helps predict what will be discovered next. Prediction from the papers' content alone works less well.


This explores why mapping the people around a research topic, and not just the topic's literature, helps predict where the next discoveries will come from. The clearest evidence in the collection comes from a materials-science study. It built a network linking papers, materials and authors, then sent simulated 'random walks' through it to trace the steps a real scientist might take: from a collaborator, to a material that collaborator knows well, to a property nobody has tested yet. Adding the people made forecasts 43% more precise than models that read only paper content, with gains of up to 400% in some settings. The advantage was largest where the literature was thin Can predicting scientists improve discovery forecasts?. The reason is simple once stated: discoveries don't come out of ideas floating freely. Someone has to be near enough to make the next inference. A cluster of relevant expertise tells you which untested connections are within someone's reach.

This also explains why reading the text alone misses something. A separate line of work argues that an argument's force depends partly on who is making it: their reputation, track record and standing in a field. Language models that see only text lose that social layer Can language models distinguish expert arguments from common assumptions?. The hypergraph approach puts that layer back. It treats science as a social system where expertise sits with particular people, not as a pile of documents. When papers on a topic are scarce, the people around it are often the best signal you have.

There is a tension here. If expert density predicts what will be found next, it also marks the limits of what is likely to be found. In a study of 100+ NLP researchers, ideas generated by language models were rated more novel than ideas from human experts, though somewhat less feasible. Expert knowledge seems to narrow the range of ideas people consider Do language models generate more novel research ideas than experts?. So the same expert clusters that make discoveries predictable may also make them conventional. The parts of the map with no experts nearby could be where surprising work is waiting.

Other studies in the collection treat scientific prediction as something machines can do without modeling people at all. Fine-tuned language models beat neuroscience experts at picking which experimental results actually happened. The tendency to blend patterns that causes hallucination when a model recalls facts becomes useful when it forecasts Can LLMs predict novel scientific results better than experts?. Retrieval-augmented models now forecast real-world events about as well as competitive human forecasters Can retrieval-augmented language models forecast like human experts?. These results raise a fair test question: was the model simply exposed to the future? Architectures that block knowledge from after a question's date are one way to keep these tests honest Can routing mask future experts to prevent knowledge leakage?.

The collection has one strong study on this exact question, not a broad body of work. The takeaway is still useful. Predicting discoveries works better when you model science as people making inferences rather than as text piling up. That same model also shows you where people are *not* looking.


Sources 6 notes

Can predicting scientists improve discovery forecasts?

Random walks over hypergraphs of papers, materials, and authors forecast discoveries 43% more precisely than content-only models, especially when literature is sparse. The mechanism simulates plausible scientific inference steps like collaboration and material expertise.

Can language models distinguish expert arguments from common assumptions?

LLMs lose the social context that gives expert claims their force—reputation, track record, and standing—because they process only text, not the social world where expertise is built and evaluated.

Do language models generate more novel research ideas than experts?

A statistically significant study of 100+ NLP researchers found LLM-generated ideas rated as more novel than human expert ideas (p<0.05), though slightly lower on feasibility. Expert knowledge constrains novelty, while LLMs explore wider conceptual combinations.

Can LLMs predict novel scientific results better than experts?

BrainBench benchmarks show fine-tuned LLMs outperform neuroscience experts at predicting which experimental results actually occurred. The same pattern-integration tendency that causes hallucination in retrieval tasks enables genuine prediction in forward-looking scenarios.

Can retrieval-augmented language models forecast like human experts?

A retrieval-augmented LM system achieved near-parity with competitive human forecasters on real forecasting questions published after model training cutoffs, sometimes surpassing human crowds. Newer model generations naturally improved forecasting without domain-specific tuning.

Show all 6 sources
Can routing mask future experts to prevent knowledge leakage?

TiMoE pre-trains experts on disjoint two-year slices and masks experts whose windows postdate the query, cutting future-knowledge errors by ~15% while guaranteeing strict causal validity. This shows temporal grounding can be an architectural property, not just a retrieval patch.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.