Cognitive Models and Latent Representations

Related topics:

• A polar coordinate system represents syntax in large language models
  Originally formalized with symbolic representations, syntactic trees may also be effectively represented in the activations of large language models (LLMs). Indeed, a “Structural Probe” can find a sub…
• ACE: Abstractions for Communicating Efficiently
  A central but unresolved aspect of problem-solving in AI is the capability to introduce and use abstractions, something humans excel at. Work in cognitive science has demonstrated that humans tend tow…
• Artifacts as Memory Beyond the Agent Boundary
  The situated view of cognition holds that intelligent behavior depends not only on internal memory, but on an agent’s active use of environmental resources. Here, we begin formalizing this intuition w…
• Autotelic Agents with Intrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short Survey
  Building autonomous machines that can explore open-ended environments, discover possible interactions and build repertoires of skills is a general objective of artificial intelligence. Developmental a…
• Bounds of Chain-of-Thought Robustness: Reasoning Steps, Embed Norms, and Beyond
  Existing research indicates that the output of Chain-of-Thought (CoT) is significantly affected by input perturbations. Although many methods aim to mitigate such impact by optimizing prompts, a theor…
• Break It Down: Evidence for Structural Compositionality in Neural Networks
  Though modern neural networks have achieved impressive performance in both vision and language tasks, we know little about the functions that they implement. One possibility is that neural networks im…
• CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learning
  In open-ended environments, autonomous learning agents must set their own goals and build their own curriculum through an intrinsically motivated exploration. They may consider a large diversity of go…
• Chain-of-Thought Reasoning Without Prompting
  In enhancing the reasoning capabilities of large language models (LLMs), prior research primarily focuses on specific prompting techniques such as few-shot or zero-shot chain-of-thought (CoT) promptin…
• Connecting the Dots: LLMs can Infer and Verbalize Latent Structure from Disparate Training Data
  Description automatically generated](file:////Users/adrianchan/Library/Group%20Containers/UBF8T346G9.Office/TemporaryItems/msohtmlclip/clip_image012.png) One way to address safety risks from large la…
• Converging Paradigms: The Synergy of Symbolic and Connectionist AI in LLM-Empowered Autonomous Agents
  Traditionally considered distinct paradigms, connectionist AI focuses on neural networks, while symbolic AI emphasizes symbolic representation and logic. Recent advancements in large language models (…
• Decoupling Knowledge and Reasoning in LLMs: An Exploration Using Cognitive Dual-System Theory
  While large language models (LLMs) leverage both knowledge and reasoning during inference, the capacity to distinguish between them plays a pivotal role in model analysis, interpretability, and develo…
• Discovering Latent Concepts Learned in BERT
  A large number of studies that analyze deep neural network models and their ability to encode various linguistic and non-linguistic concepts provide an interpretation of the inner mechanics of these m…
• Do LLMs Encode Functional Importance of Reasoning Tokens?
  Large language models solve complex tasks by generating long reasoning chains, achieving higher accuracy at the cost of increased computational cost and reduced ability to isolate functionally relevan…
• Do Large Language Models Latently Perform Multi-Hop Reasoning?
  We study whether Large Language Models (LLMs) latently perform multi-hop reasoning with complex prompts such as “The mother of the singer of ‘Superstition’ is”. We look for evidence of a latent reason…
• Do Large Language Models Perform Latent Multi-Hop Reasoning without Exploiting Shortcuts?
  We evaluate how well Large Language Models (LLMs) latently recall and compose facts to answer multi-hop queries like “In the year Scarlett Johansson was born, the Summer Olympics were hosted in the co…
• Efficient Reasoning with Hidden Thinking
  Chain-of-Thought (CoT) reasoning has become a powerful framework for improving complex problem-solving capabilities in Multimodal Large Language Models (MLLMs). However, the verbose nature of textual …
• Eliciting Latent Knowledge from Quirky Language Models
  Eliciting Latent Knowledge (ELK) aims to find patterns in a neural network’s activations that robustly track the true state of the world, even in cases where the model’s output is untrusted and hard t…
• Emergence of Abstractions: Concept Encoding and Decoding Mechanism for In-Context Learning in Transformers
  Humans distill complex experiences into fundamental abstractions that enable rapid learning and adaptation. Similarly, autoregressive transformers exhibit adaptive learning through in-context learning…
• From Tokens to Thoughts: How LLMs and Humans Trade Compression for Meaning
  Humans organize knowledge into compact categories through semantic compression by mapping diverse instances to abstract representations while preserving meaning (e.g., robin and blue jay are both bird…
• Grounding Large Language Models in Interactive Environments with Online Reinforcement Learning
  Large Language Models’ (LLM) abilities to capture abstract knowledge about world’s physics to solve decision-making problems. Yet, the alignment between LLMs’ knowledge and the environment can be wron…
• Human-like Category Learning by Injecting Ecological Priors from Large Language Models into Neural Networks
  large language models can generate cognitive tasks, specifically category learning tasks, that match the statistics of real-world tasks, deriving rational agents adapted to these tasks using the frame…
• In-context learning agents are asymmetric belief updaters
  We study the in-context learning dynamics of large language models (LLMs) using three instrumental learning tasks adapted from cognitive psychology. We find that LLMs update their beliefs in an asymme…
• Inference-Time Intervention: Eliciting Truthful Answers from a Language Model
  We introduce Inference-Time Intervention (ITI), a technique designed to enhance the “truthfulness” of large language models (LLMs). ITI operates by shifting model activations during inference, followi…
• LLM Reasoning Is Latent, Not the Chain of Thought
  This position paper argues that large language model (LLM) reasoning should be studied as latent-state trajectory formation rather than as faithful surface chainof- thought (CoT). This matters because…
• LLMs are Frequency Pattern Learners in Natural Language Inference
  While fine-tuning LLMs on NLI corpora improves their inferential performance, the underlying mechanisms driving this improvement remain largely opaque. In this work, we conduct a series of experiments…
• Language Modeling by Language Models
  Can we leverage LLMs to model the process of discovering novel language model (LM) architectures? Inspired by real research, we propose a multi-agent LLM approach that simulates the conventional stage…
• Large Language Models Reflect the Ideology of their Creators
  In this paper, we uncover notable diversity in the ideological stance exhibited across different LLMs and languages in which they are accessed. We do this by prompting a diverse panel of popular LLMs …
• Large language models can segment narrative events similarly to humans
  Humans perceive discrete events such as "restaurant visits" and "train rides" in their continuous experience. One important prerequisite for studying human event perception is the ability of researche…
• Latent Collaboration in Multi-Agent Systems
  Multi-agent systems (MAS) extend large language models (LLMs) from independent single-model reasoning to coordinative system-level intelligence. While existing LLM agents depend on text-based mediatio…
• Latent Skill Discovery for Chain-of-Thought Reasoning
  Recent advances in Large Language Models (LLMs) have led to an emergent ability of chain-of-thought (CoT) prompting, a prompt reasoning strategy that adds intermediate rationale steps between question…
• LatentQA: Teaching LLMs to Decode Activations Into Natural Language
  A LATENTQA system accepts as input an activation along with any natural language question about the activation and returns a natural language answer as output. For example, the system might accept LLM…
• Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs
  Existing methods for adapting large language models (LLMs) to new tasks are not suited to multi-task adaptation because they modify all the model weights–causing destructive interference between tasks…
• Mastering Diverse Domains through World Models
  Developing a general algorithm that learns to solve tasks across a wide range of applications has been a fundamental challenge in artificial intelligence. Although current reinforcement learning algor…
• Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads
  Large Language Models (LLMs) employ autoregressive decoding that requires sequential computation, with each step reliant on the previous one’s output. This creates a bottleneck as each step necessitat…
• Metacognitive Capabilities of LLMs: An Exploration in Mathematical Problem Solving
  Metacognitive knowledge refers to humans’ intuitive knowledge of their own thinking and reasoning processes. Today’s best LLMs clearly possess some reasoning processes. The paper gives evidence that t…
• Mixture of Thoughts: Learning to Aggregate What Experts Think, Not Just What They Say
  Open-source Large Language Models (LLMs) increasingly specialize by domain (e.g., math, code, general reasoning), motivating systems that leverage complementary strengths across models. Prior multi-LL…
• Nested Attention: Semantic-aware Attention Values for Concept Personalization
  Personalizing text-to-image models to generate images of specific subjects across diverse scenes and styles is a rapidly advancing field. Current approaches often face challenges in maintaining a bala…
• Not All Parameters Are Created Equal: Smart Isolation Boosts Fine-Tuning Performance
  Supervised fine-tuning (SFT) is a pivotal approach to adapting large language models (LLMs) for downstream tasks; however, performance often suffers from the “seesaw phenomenon”, where indiscriminate …
• On the Relationship between Sentence Analogy Identification and Sentence Structure Encoding in Large Language Models
  The ability of Large Language Models (LLMs) to encode syntactic and semantic structures of language is well examined in NLP. Additionally, analogy identification, in the form of word analogies are ext…
• Open Problems in Mechanistic Interpretability
  Recent progress in artificial intelligence (AI) has resulted in rapidly improved AI capabilities. These capabilities are not designed by humans. Instead, they are learned by deep neural networks (Hint…
• Pushdown Layers: Encoding Recursive Structure in Transformer Language Models
  Recursion is a prominent feature of human language, and fundamentally challenging for self-attention due to the lack of an explicit recursive-state tracking mechanism. Consequently, Transformer langua…
• Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis
  Much of the excitement in modern AI is driven by the observation that scaling up existing systems leads to better performance. But does better performance necessarily imply better internal representat…
• Reasoning Beyond Chain-of-Thought: A Latent Computational Mode in Large Language Models
  Chain-of-Thought (CoT) prompting has improved the reasoning performance of large language models (LLMs), but it remains unclear why it works and whether it is the unique mechanism for triggering reaso…
• Reasoning Circuits in Language Models: A Mechanistic Interpretation of Syllogistic Inference
  Recent studies on reasoning in language models (LMs) have sparked a debate on whether they can learn systematic inferential principles or merely exploit superficial patterns in the training data. To u…
• Reasoning to Learn from Latent Thoughts
  Human-written text is the culmination of an underlying thought process—when we write, there is often an internal dialogue that clarifies or even determines the written word. However, modern language m…
• RecExplainer: Aligning Large Language Models for Recommendation Model Interpretability
  we propose a new model interpretation approach for recommender systems, by using LLMs as surrogate models and learn to mimic and comprehend target recommender models. Specifically, we introduce three …
• Reinforcement Learning Finetunes Small Subnetworks in Large Language Models
  Reinforcement learning (RL) yields substantial improvements in large language models’ (LLMs) downstream task performance and alignment with human values. Surprisingly, such large gains result from upd…
• Scalable Language Models with Posterior Inference of Latent Thought Vectors
  We propose a novel family of language models, Latent-Thought Language Models (LTMs), which incorporate explicit latent thought vectors that follow an explicit prior model in latent space. These latent…
• Scaling can lead to compositional generalization
  Can neural networks systematically capture discrete, compositional task structure despite their continuous, distributed nature? The impressive capabilities of large scale neural networks suggest that …
• Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
  We study a novel language model architecture that is capable of scaling test-time computation by implicitly reasoning in latent space. Our model works by iterating a recurrent block, thereby unrolling…
• Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Space
  Human cognition typically involves thinking through abstract, fluid concepts rather than strictly using discrete linguistic tokens. Current reasoning models, however, are constrained to reasoning with…
• SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMs
  Chain-of-Thought (CoT) reasoning enables Large Language Models (LLMs) to solve complex reasoning tasks by generating intermediate reasoning steps. However, most existing approaches focus on hard token…
• Style Vectors for Steering Generative Large Language Models
  This research explores strategies for steering the output of large language models (LLMs) towards specific styles, such as sentiment, emotion, or writing style, by adding style vectors to the activati…
• The Molecular Structure of Thought: Mapping the Topology of Long Chain-of-Thought Reasoning
  Large language models (LLMs) often fail to learn effective long chain-of-thought (Long CoT) reasoning from human or non-Long-CoT LLMs imitation. To understand this, we propose that effective and learn…
• The Unreasonable Ineffectiveness of the Deeper Layers
  We empirically study a simple layer-pruning strategy for popular families of open weight pretrained LLMs, finding minimal degradation of performance on different question-answering benchmarks until af…
• Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens
  Large language models (LLMs) have demonstrated impressive reasoning capabilities by scaling test-time compute via long Chain-of-Thought (CoT). However, recent findings suggest that raw token counts ar…
• Thinking LLMs: General Instruction Following with Thought Generation
  in the standard alignment framework they lack the basic ability of explicit thinking before answering. Thinking is important for complex questions that require reasoning and planning – but can be appl…
• Thought Anchors: Which LLM Reasoning Steps Matter?
  We argue that analyzing reasoning traces at the sentence level is a promising approach to understanding reasoning processes. We present three complementary attribution methods: (1) a black-box method …
• Thought Communication in Multiagent Collaboration
  Natural language has long enabled human cooperation, but its lossy, ambiguous, and indirect nature limits the potential of collective intelligence. While machines are not subject to these constraints,…
• Topology of Reasoning: Understanding Large Reasoning Models through Reasoning Graph Properties
  Recent large-scale reasoning models have achieved state-of-the-art performance on challenging mathematical benchmarks, yet the internal mechanisms underlying their success remain poorly understood. In…
• Toward understanding and preventing misalignment generalization
  Large language models like ChatGPT don’t just learn facts—they pick up on patterns of behavior. That means they can start to act like different “personas,” or types of people, based on the content the…
• Training Large Language Models to Reason in a Continuous Latent Space
  To explore the potential of LLM reasoning in an unrestricted latent space instead of using natural language, we introduce a new paradigm Coconut (Chain of Continuous Thought). We utilize the last hidd…
• Training Nonlinear Transformers for Chain-of-Thought Inference: A Theoretical Generalization Analysis
  Chain-of-Thought (CoT) is an efficient prompting method that enables the reasoning ability of large language models by augmenting the query using multiple examples with multiple intermediate steps. De…
• Turning large language models into cognitive models
  ask whether large language models can be turned into cognitive models. We find that – after finetuning them on data from psychological experiments – these models offer accurate representations of huma…
• Understanding Hidden Computations in Chain-of-Thought Reasoning
  Chain-of-Thought (CoT) prompting has significantly enhanced the reasoning abilities of large language models. However, recent studies have shown that models can still perform complex reasoning tasks e…
• What Has a Foundation Model Found? Using Inductive Bias to Probe for World Models
  Foundation models are premised on the idea that sequence prediction can uncover deeper domain understanding, much like how Kepler’s predictions of planetary motion later led to the discovery of Newton…