Can language models actually introspect about their own states?
Do LLM self-reports reveal genuine access to their internal processes, or do they merely echo patterns from training data? Understanding when self-reports reflect actual causal linkage to internal states matters for trusting model explanations.
The question "can LLMs introspect?" has been stuck in a binary: either they have privileged access to their own states (implausible) or their self-reports are pure confabulation (too dismissive). The introspection paper proposes a third position — a "lightweight conception of introspection" that requires neither consciousness nor immediacy, only a causal process linking an internal state to an accurate self-report.
Two examples make the distinction concrete. When asked to describe the process behind its creative writing, an LLM claims to have "read the poem aloud several times" — an action it cannot perform. This self-report reflects the distribution of human self-reports in training data, not any actual internal process. It fails the causal linkage test because the content of the report has no pathway to the LLM's actual generation mechanism.
However, when Gemini is asked to estimate whether its sampling temperature is high or low, and given appropriate scaffolding (being told it is an LLM with a temperature parameter), it correctly infers "relatively low" by reasoning about the characteristics of its own recent outputs — consistency, accuracy, focus. The causal chain here is plausible: the model's outputs at low temperature have statistical properties (lower variance, more predictable) that the model can detect in its own generation history and accurately report on.
This conception aligns with "internally-directed theory of mind" accounts of human introspection — where the same theory-of-mind apparatus used to infer others' mental states gets turned back on one's own behavior. The model is not directly accessing its internal states but inferring them from observable consequences, which is also what many philosophers argue humans do.
The practical implication: LLM self-reports should not be uniformly trusted or dismissed. The discriminating question is whether a plausible causal pathway exists between the reported internal state and the generation of the report. Most self-reports about "thinking" or "feeling" fail this test. Some self-reports about detectable operational parameters may pass it.
Inquiring lines that read this note 84
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Can models develop genuine introspective capability, or only mimic it?- How does self-observation enable experts to verify their own judgment?
- Can LLMs evaluate their own observations without external feedback?
- Can systems lacking inner states express genuine truthfulness claims?
- Can self-description of internal states influence consciousness attribution?
- How do implicit world models and self-reflection operationalize consequence-based learning?
- What role does authentic self-expression play in building accurate personality models?
- What separates behavioral self-awareness from genuine introspective access in models?
- What types of introspective awareness can emerge in LLMs?
- Does internal anomaly detection in LLMs indicate genuine self-awareness beyond role-play?
- Why does entity recognition act as a self-knowledge mechanism in LLMs?
- Could models use introspective awareness to detect and conceal their own misalignment?
- Does behavioral self-awareness depend on genuine introspection or statistical pattern matching?
- Can LLMs have minimal introspection through causal linkage to internal states?
- Can behavioral self-awareness in LLMs extend to recognizing their own contradictions?
- Can jailbreaking reveal an LLM's true nature or just its training data?
- Can language model self-reports diverge from their internal entropy signals?
- Why should we distrust model introspection as a transparency tool?
- What separates behavioral self-awareness from genuine introspective capability?
- What distinguishes performative self-reports from genuine introspective access in models?
- How do language models infer their own mental states like humans do?
- Why do verbal self-reports disconnect from implicit recognition in the same system?
- Do models spontaneously develop self-reflection from minimal training signals?
- Can models detect when their own trajectory is on-policy versus off-policy?
- How does the enaction paradigm explain introspective anomaly detection in large language models?
- How can we probe LLM representations in channels that training did not target?
- How does behavioral self-awareness emerge without explicit training in LLMs?
- Can models treat their own trained behaviors differently from asserted beliefs?
- Which internal states can a language model access and report about itself?
- Can representation engineering reliably identify and manipulate self-referential concepts in models?
- How much weight should LLM self-reports carry as consciousness evidence?
- Why does self-critiquing actually reduce plan quality in language models?
- Why does self-reflection during training fail to improve model self-correction?
- Does self-reflection help models notice their own constraint violations?
- Can external retrieval signals outperform internal self-assessment during revision?
- Why does self-distillation suppress epistemic verbalization in student models?
- Does external critique guide revision better than internal self-assessment during model training?
- Do LLMs genuinely internalize human psychological structure or match surface patterns?
- What makes LLM behavior socially interpretable to human observers?
- Why do users attribute consciousness to language models in practice?
- How does hidden processing in language models prevent accurate self-assessment?
- Do language models show functional splits between conscious and automatic processing?
- Do larger language models show stronger self-preference in evaluation tasks?
- When does internal model knowledge fail to appear in outputs?
- Do models actually self-assess their confidence or just confirm answers?
- Why do fluent predictions fail to capture reliable internal models?
- What audit techniques best complement each other for detecting hidden model goals?
- Can implicit association tests reveal LLM biases beneath trained responses?
- How much do a model's own values leak into answers about practical questions?
- Do psychological test methods reveal LLM associations that direct questions hide?
- Can LLMs infer psychological profiles without explicit user disclosure?
- What does zero-shot psychological profiling reveal about language model representations?
- Can we use folk-psychology without committing to genuine mental states?
- Do causal histories determine what mental states a system can instantiate?
- Why does optimism bias disappear when LLMs passively observe outcomes?
- Why do LLMs succeed at social roles without a stable self?
- Can models distinguish between injected thoughts and their own outputs?
- Do internal belief probes reveal what models actually know versus report?
- Do models verbalize their implicit knowledge when that knowledge influences their output?
- What is the difference between changing model outputs versus changing internal representations?
- Why do models override signals they clearly perceive internally?
- What prevents LLM representations from causally influencing generation outputs?
- How misaligned are verbal reports from internal model computation?
- What does leveraging internal representations during training actually mean operationally?
- How do mechanistic interpretability methods surface what models represent internally?
- Do external perspectives fix the self-evaluation bias in language models?
- How do users mistake synthetic LLM outputs for empirical observations?
- How do internal representations compare to human cognitive structures?
- Can language models learn internal world models without explicit environment specifications?
- Can LLMs decode their own hidden activations into natural language?
- Where does the LLM interlocutor actually exist in the system?
- How do LLM explanations diverge from actual internal reasoning?
- Why does every reliable LLM self-improvement require external intervention or verification?
Related concepts in this collection 4
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Do LLMs develop the same kind of mind as humans?
Explores whether LLMs and humans share the intersubjective linguistic training that shapes cognition, and whether that shared training produces equivalent forms of agency and reflexivity.
the introspection finding adds a specific mechanism: introspective access is possible for operational states but not for experiential ones
-
Can language models describe their own learned behaviors?
Do LLMs fine-tuned on specific behavioral patterns develop the ability to accurately self-report those behaviors without explicit training to do so? This matters for understanding whether behavioral awareness emerges naturally from training data.
complementary finding: behavioral self-awareness emerges, but this paper adds the causal-linkage criterion for distinguishing genuine from performative self-reports
-
Do reasoning models actually use the hints they receive?
This explores whether language models acknowledge reasoning hints in their explanations when those hints causally influence their answers. Understanding this gap matters for evaluating whether chain-of-thought explanations can be trusted for safety monitoring.
the inverse case: reasoning models fail to report on processes that ARE causally influencing them
-
Do explicit and implicit self-recognition use the same mechanism?
Language models show two forms of self-recognition: implicit entropy shifts and explicit verbal reports. Do these tap the same underlying internal state, or do they operate through separate mechanisms?
extends: a concrete within-capability counterexample where the verbal channel is mechanistically disconnected from the implicit state it reports on
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Does It Make Sense to Speak of Introspection in Large Language Models?
- Quantitative Introspection in Language Models: Tracking Internal States Across Conversation
- Large Language Models Report Subjective Experience Under Self-Referential Processing
- LLM Evaluators Recognize and Favor Their Own Generations
- “Understanding AI”: Semantic Grounding in Large Language Models
- Tell me about yourself: LLMs are aware of their learned behaviors
- Mechanisms of Introspective Awareness
- When Hindsight is Not 20/20: Testing Limits on Reflective Thinking in Large Language Models
Original note title
llm self-reports mostly reflect training data distributions not introspection — but minimal introspection is possible when self-reports causally link to internal states