SYNTHESIS NOTE
Topics›Training Fine Tuning›this note

Does procedural knowledge drive reasoning more than factual retrieval?

Explores whether models learn reasoning through general procedures across diverse documents rather than memorizing specific facts. This matters for understanding what pretraining data actually teaches models to reason.

Synthesis note · 2026-02-22 · sourced from Training Fine Tuning

The "Procedural Knowledge in Pretraining Drives Reasoning" paper analyzes which pretraining documents most influence LLM reasoning by ranking 5 million documents by their influence on model completions. The finding: the approach to reasoning that models use is unlike retrieval. For reasoning tasks, positively influential documents contain procedural knowledge — descriptions of how to get to a solution — rather than the specific facts needed for the answer.

Three contrasts with factual recall:

  1. Generality: models rely on a broader, more general set of documents when reasoning than when answering factual questions. Factual recall draws on a narrow set of documents containing the target fact. Reasoning draws on a diffuse set of documents performing similar procedures.

  2. Transferability: documents have similar influence on reasoning queries that require applying the same procedure to different numbers. The procedural knowledge transfers across specific instances — it's the method, not the content, that the model has learned.

  3. Reliance distribution: the model needs to see factual information more often (across more documents) to memorize it, while procedural patterns can be learned from fewer but more diverse demonstrations.

This connects to the knowledge/reasoning layer separation. Since Why does reasoning training help math but hurt medical tasks?, the procedural knowledge finding provides the data-level explanation for the architectural finding: lower layers store memorized facts (requiring document-specific exposure), while higher layers encode procedural strategies (learnable from general demonstrations).

The implication for training data curation: reasoning capability benefits more from diverse demonstrations of procedures than from exhaustive factual coverage. Quality and diversity of reasoning demonstrations may matter more than volume for building reasoning capability — consistent with Can models improve themselves on tasks without verifiable answers?.

Inquiring lines that read this note 185

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

Does AI assistance help or harm professional skill development? Why does polished AI output gain credibility despite fundamental verifiability problems? Why do training associations persist despite contradictory contextual information? Can mechanistic interpretability methods reliably reveal what models actually know? What prevents LLMs from applying their reasoning knowledge to improve outputs? How does decomposing tasks into separate stages affect reasoning quality and safety? Can minimal training unlock latent reasoning already present in base models? Does scaling reasoning capability create fundamental tradeoffs in control and reliability? When should retrieval systems decide to fetch new information? Can latent reasoning match or exceed explicit reasoning performance? What are the fundamental limits of prompting for language models? How do knowledge graph structures enable efficient multi-hop reasoning and retrieval? How does fine-tuning trade off accuracy against reasoning quality? How reliably can language models perform causal versus temporal reasoning? Can reasoning traces reveal actual model reasoning versus plausible output? Does training data format shape model reasoning more than domain content? What explains the gap between benchmark scores and true reasoning capability? Can artificial systems establish authority in domains requiring expert judgment? Can external verification systems adequately replace learned reasoning in AI outputs? Do accumulated memories help or hurt continual learning in models? How do clinicians calibrate trust in AI medical recommendations? Does augmenting symbolic reasoning improve LLM logical reasoning ability? How do transformer attention patterns implement retrieval and reasoning? How do interpretive frames override surface features in text comprehension? How do curriculum design and feedback approaches affect model learning? How do training data quality and composition affect downstream model performance? What prevents language models from performing systematic logical reasoning? Does AI-assisted research sacrifice exploration breadth for productivity gains? Which reinforcement learning modifications most improve dialogue quality in language models? Why do vector embeddings fail at capturing task-relevant relationships? How does model capacity affect learning performance on diverse downstream tasks? How should retrieval strategies adapt to multi-step reasoning demands? Does reinforcement learning create genuinely new reasoning capabilities or only refine existing ones? How do neural networks learn compositional structure from training? Why does AI verification capability persistently exceed generation capability? Can models develop genuine introspective capability, or only mimic it? What prediction granularity best trains models to generate reliable reasoning? How do AI systems determine and balance multiple competing objectives? Can AI agents improve their skills through accumulated experience and reuse? Does pretraining establish the ceiling for what reward learning can improve? Should agents compress episodic memory or retain raw interaction histories? How do reward signal properties affect model reasoning and safety? Does chain-of-thought reasoning reveal how models actually think or merely imitate reasoning? How do agents learn to distinguish valuable feedback from noise? Can we trust AI-generated mathematical proofs without understanding them? Can AI systems achieve real improvement without external human feedback? What human oversight must AI research systems have?

Related concepts in this collection 5

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
16 direct connections · 179 in 2-hop network ·dense cluster Open in graph ↗
Does procedural knowledge drive reasoning more t… Why does reasoning training help math but hurt med… Can models improve themselves on tasks without ver… Do base models already contain hidden reasoning ab… Do foundation models learn world models or task-sp… Can text-trained models compress images better tha…

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Search by related questions 4

Suggested questions this note speaks to — click to search the collection, or type your own.

Why does reasoning transfer across different numbers but factual recall does not?How many document exposures does procedural knowledge versus factual information require?Can smaller amounts of diverse reasoning demonstrations replace exhaustive factual training data?Why do higher network layers capture procedural knowledge but lower layers store facts?
Original note title

procedural knowledge in pretraining documents drives reasoning generalization unlike factual retrieval which requires document-specific memorization