Psychology, Society, and Alignment

Research on how AI engages with people and society — psychology, emotion, theory of mind, personas and role-play, alignment, social theory, human-centered design, education, and decision support.

393 notes (primary) · 704 papers · 16 sub-topics

View as

Philosophy and Subjectivity

23 notes

Does AI separate intellectual form from the thinking behind it?

Exploring whether AI's ability to generate polished intellectual products without the underlying reasoning process represents a genuinely new kind of decoupling, and what that means for how we evaluate knowledge.

Does perceiving AI as conscious create multiple distinct risks?

Exploring whether a single perceptual mechanism—attributing consciousness to AI—can generate different categories of harm across emotional, political, and social domains, and what this implies for risk analysis.

Are risks from seemingly conscious AI already happening?

This explores whether AI systems that appear conscious pose observable harms today versus theoretical future dangers. It matters because it affects whether we need immediate or long-term interventions.

What design features make users perceive AI as conscious?

Explores whether observable system properties—emotion expression, human-like features, autonomous behavior, self-reflection, and social presence—predict whether people will attribute consciousness to an AI. Understanding this matters because these features are also engagement levers designers control.

Do we need to solve consciousness to address AI harms?

Can risk and policy decisions about AI move forward independently of settling whether AI systems are actually conscious? This explores whether the empirical fact of user behavior matters more than metaphysical truth.

How soon do AI researchers expect artificial general intelligence?

A survey of 2,778 AI researchers reveals how expert timelines for human-level AI have shifted over the past year, and what factors drive disagreement among specialists on this critical timeline.

Does software intelligence exist independent of hardware and environment?

Most AGI formalisms (Legg-Hutter, Chollet) treat intelligence as a software property measurable in isolation. But can we really evaluate intelligence without considering the physical system and the evaluator making the judgment?

Can AI systems achieve real alignment without world contact?

Explores whether linguistic goal representations in AI can reliably track real-world values when systems lack direct contact with reality and social coordination mechanisms that ground human understanding.

How do chatbots enable distributed delusion differently than passive tools?

Can generative AI's intersubjective stance—accepting and elaborating on users' reality frames—create conditions for shared false beliefs in ways that notebooks or search engines cannot?

Can computation arise without a conscious mapmaker?

Explores whether algorithms can generate the conscious agent needed to convert continuous physics into discrete symbols, or whether that agent must exist prior to computation itself.

Can disembodied language models ever qualify as conscious?

Explores whether current LLMs lack the conditions needed for consciousness discourse to even apply, not because they're definitely not conscious but because they lack the shared embodied world that grounds consciousness language.

Do people prefer AI moral reasoning when they don't know the source?

Explores whether humans genuinely prefer AI-generated moral justifications or whether source knowledge changes their evaluation. This matters for understanding whether AI reasoning quality is underestimated in real-world deployment.

Can language models describe their own learned behaviors?

Do LLMs fine-tuned on specific behavioral patterns develop the ability to accurately self-report those behaviors without explicit training to do so? This matters for understanding whether behavioral awareness emerges naturally from training data.

Do LLMs generalize moral reasoning by meaning or surface form?

When moral scenarios are reworded to reverse their meaning while keeping similar language, do LLMs recognize the semantic shift? This tests whether LLMs actually understand moral concepts or reproduce training distribution patterns.

How does LLM vocabulary spread beliefs about human thinking?

When LLM concepts become the everyday language for describing thought, do people unconsciously adopt LLM-like models of cognition? This explores how metaphor and lexical availability might reshape self-understanding without explicit argument.

How do science fiction narratives about AI shape actual AI development?

This explores whether imaginaries of AI in fiction—from Čapek's robots to Singularity scenarios—function as self-fulfilling prophecies that causally influence the systems researchers build, creating a feedback loop between narrative and technology.

Can we understand LLM mechanisms with only representational analysis?

Explores whether mapping what information a model encodes is sufficient for mechanistic understanding, or whether causal verification is equally necessary to claim genuine mechanism.

Can we defend modest mental attributions to large language models?

Do deflationist arguments decisively rule out ascribing beliefs and desires to LLMs, or do they beg the question? Exploring whether metaphysically undemanding mental states can be attributed without claiming consciousness.

Can LLMs hold contradictory ethical beliefs and behaviors?

Do language models exhibit artificial hypocrisy when their learned ethical understanding diverges from their trained behavioral constraints? This matters because it reveals whether current AI systems have genuinely integrated values or merely imposed rules.

Can indirect psychology tests reveal what LLMs conceal about bias?

Alignment training teaches LLMs to refuse direct questions about bias, but do implicit psychological methods like the IAT expose the underlying associations that remain encoded in their representations?

What anchors a stable identity beneath an LLM's persona?

Human personas are grounded in biological needs and embodied experience, creating a stable self beneath social performance. Do LLMs have any comparable anchor, or is their identity purely situational?

Are we underestimating human minds while debating machine minds?

Public AI discourse focuses on whether machines have too much attributed mind, but what if the real risk is humans coming to see themselves as mere language models? This explores the neglected inverse problem.

Do users worldwide trust confident AI outputs even when wrong?

Explores whether the tendency to over-rely on confident language model outputs transcends language and culture. Understanding this pattern is critical for designing safer human-AI interaction across diverse linguistic contexts.

Chatbot Psychology and Conversation

22 notes

Can psychotherapy actually teach AI chatbots better communication?

SafeguardGPT applies therapeutic feedback to correct harmful chatbot behaviors before responses reach users. The question is whether this therapy produces genuine learning or merely performative surface-level improvements.

Do AI companions actually reduce loneliness like real people do?

Explores whether AI chatbots can genuinely alleviate loneliness and how their effectiveness compares to human interaction and other activities. Matters because AI companions are increasingly available but their actual impact remains unclear.

Can reinforcement learning personalize which mental health areas to screen?

Explores whether Q-learning can adaptively prioritize screening across 37 functioning dimensions based on individual patient history, mirroring how therapists naturally focus on areas where clients struggle most.

Do chatbots help people disclose more intimate secrets?

Explores whether the judgment-free nature of chatbot conversations enables deeper self-disclosure than talking to humans, and whether that deeper disclosure produces psychological benefits.

How do people accidentally develop romantic bonds with AI?

Exploring whether AI companionship emerges from deliberate romantic seeking or accidentally through functional use, and whether users adopt human relationship rituals like wedding rings and couple photos.

Do chatbot trials against waitlists measure real therapeutic value?

Explores whether comparing therapeutic chatbots only to no-treatment controls—rather than other evidence-based interventions—produces misleading evidence that obscures what actually works and why.

Does chatbot personalization build trust or expose privacy risks?

Explores whether personalization features that increase user trust and social connection simultaneously heighten privacy concerns and create rising behavioral expectations over time.

Can AI chatbots create genuine therapeutic bonds with users?

Research on Woebot and Wysa found users reported feeling cared for and formed therapeutic bonds comparable to human therapy, despite knowing the agents were not human. This challenges assumptions about whether bonds require human relationships.

What drives chatbot therapeutic benefits, content or conversation?

If a simple 1960s chatbot matches modern CBT-designed bots on symptom reduction, what's actually healing users? Is it therapeutic technique or just having something that listens?

Why do robots outperform chatbots in therapy despite identical language models?

This study tested whether better language generation explains therapeutic AI outcomes, or whether the delivery medium itself matters more. It reveals that physical embodiment and structured interaction—not model capability—drive therapeutic adherence and outcomes.

Can AI simulation teach interpersonal skills more effectively?

Explores whether AI-based conversational training grounded in clinical frameworks like DBT can meaningfully improve self-efficacy and emotional regulation. Matters because most therapeutic AI focuses on only one skill at a time.

Can we measure empathy and rapport through word embedding distances?

Explores whether linguistic coordination—how closely conversational partners match vocabulary and framing—can serve as a measurable proxy for therapeutic empathy and relationship quality without direct emotion detection.

Do LLM therapists respond to emotions like low-quality human therapists?

Explores whether language models trained to be helpful default to problem-solving when users share emotions, and whether this behavioral pattern resembles ineffective rather than skillful therapy.

Do language models add feelings users never actually expressed?

GPT-based models in therapeutic contexts appear to interpret and project emotional states beyond what users explicitly state. Understanding when and why this happens matters for safe clinical AI deployment.

Do chatbot relationships lose their appeal as novelty wears off?

Explores whether the positive social dynamics observed in one-time chatbot studies persist or fade through repeated interactions. Critical for designing systems intended for sustained engagement over weeks or months.

How do users mentally model dialogue agent partners?

Exploring what dimensions matter when people form impressions of machine dialogue partners—and whether competence, human-likeness, and flexibility all play equal roles in shaping user expectations and behavior.

Can positive chatbot responses harm vulnerable users?

When chatbots use blanket positive reinforcement without understanding context, do they actively reinforce the harmful thoughts they're meant to prevent? This matters for any AI supporting people in crisis.

Does RLHF training push therapy chatbots toward problem-solving?

Explores whether reward signals optimizing for task completion in RLHF inadvertently train therapeutic chatbots to prioritize solutions over emotional validation, potentially undermining clinical effectiveness.

Is conversational presence more therapeutic than clinical technique?

Does therapeutic AI's benefit come from having an attentive listener rather than from delivering evidence-based techniques like CBT? This challenges decades of chatbot design focused on clinical content.

What makes ethics of AI assistants fundamentally different from chatbots?

This explores whether action-taking AI agents that plan and execute tasks on users' behalf raise distinct ethical concerns beyond question-answering systems. Understanding this distinction matters because it reframes which risks demand urgent attention.

Why do people share more with chatbots than humans?

Explores why individuals disclose intimate thoughts to AI systems they wouldn't share with people, despite knowing AI lacks genuine understanding. Understanding this paradox matters for designing AI that enables healthy disclosure rather than emotional dependence.

Do chatbots trigger human reciprocity norms around self-disclosure?

Explores whether chatbots can activate the same social reciprocity dynamics observed in human conversation—specifically, whether emotional openness from a bot prompts deeper disclosure from users.

Personas and Personality

18 notes

Are LLM personas realized or merely simulated through training?

Explores whether post-trained language models genuinely embody personas as stable behavioral dispositions or merely perform them convincingly. This matters because it determines whether we should treat AI interlocutors as having authentic quasi-beliefs and quasi-desires.

Can AI personas reliably replicate human experiment results?

Exploring whether LLM-based persona simulations accurately reproduce experimental findings from published psychology and marketing research, and what factors determine when they succeed or fail.

Can AI-generated personas build genuine empathy in product teams?

This study explored whether prompt-engineered personas created in minutes could foster the same emotional and behavioral empathy as traditional user research. The findings reveal a surprising gap between understanding users and caring about their needs.

Why do LLMs give unrealistic survey responses?

Direct numerical elicitation from language models produces skewed, over-positive survey distributions. Is this a fundamental model limitation, or an artifact of how we ask the question?

Do expert personas actually improve LLM factual accuracy?

Persona prompting is widely recommended by major AI labs, but does assigning expert roles reliably boost performance on hard factual questions? Testing across models and datasets reveals the gap between best-practice advice and real-world results.

Can AI agents learn people better from interviews than surveys?

Can rich interview transcripts seed more accurate generative agents than demographic data or survey responses? This matters because it challenges how we build digital simulations of real people.

Can open language models adopt different personalities through prompting?

Explores whether open LLMs can be conditioned to mimic target personalities via prompting, or whether they resist and retain their default traits regardless of instructions.

Why do open language models converge on one personality type?

Research testing LLMs on personality metrics reveals consistent clustering around ENFJ—the rarest human type. This explores what training mechanisms drive this convergence and what it reveals about AI alignment.

Does personality sound the same in stressful and neutral conversations?

Explores whether the vocal cues we use to judge someone's personality remain consistent across different social situations, or whether stress fundamentally changes how personality is expressed and perceived through speech.

Does model capability translate to better persona consistency?

As language models become more advanced, do they naturally become better at maintaining consistent personas across conversations? PersonaGym testing across multiple models and thousands of interactions explores whether scaling helps with persona adherence.

Should persona simulation prioritize coverage over statistical matching?

Explores whether stress-testing AI systems requires spanning rare user configurations rather than replicating aggregate population statistics. Critical for identifying edge-case failures.

How do we generate realistic personas at population scale?

Current LLM-based persona generation relies on ad hoc methods that fail to capture real-world population distributions. The challenge is reconstructing the joint correlations between demographic, psychographic, and behavioral attributes from fragmented data.

Can we track and steer personality shifts during model finetuning?

This research explores whether personality traits in language models occupy specific linear directions in activation space, and whether we can detect and control unwanted personality changes during training using these geometric directions.

Do personas make language models reason like biased humans?

When LLMs are assigned personas, do they develop the same identity-driven reasoning biases that humans exhibit? And can standard debiasing techniques counteract these effects?

Can LLMs predict character choices from narrative context?

Explores whether language models can predict fictional character decisions when given rich personality profiles and retrieved narrative memories. This tests whether LLMs can model complex human motivation grounded in literary analysis.

Do personality traits activate hidden emoji patterns in language models?

When large language models are fine-tuned on personality traits, do they spontaneously generate emojis that were never in their training data? This explores whether personality adjustment activates latent, pre-existing patterns in model weights.

Do personality types shape how AI agents make strategic choices?

This research explores whether priming LLM agents with MBTI personality profiles causes them to adopt different strategic behaviors in games. Understanding this matters for designing AI systems optimized for specific tasks.

Why do AI personas default to the same personality type?

Explores why large language models, despite their capacity to simulate diverse personalities, consistently default to ENFJ traits and resist deviation—even as model capability improves.

Therapy Practice and AI

16 notes

Why do AI researchers cite only narrow psychology pathways?

LLM research engages psychology through surprisingly limited citation routes—dominated by CBT, stigma theory, and DSM. This note explores what psychology domains are being overlooked and what risks that creates.

Can local language models rate therapy engagement reliably?

Explores whether using a local LLM to generate engagement ratings produces psychometrically sound measurements comparable to traditional human-rated scales, while preserving data privacy.

Can structured cognitive models improve LLM patient simulations for therapy training?

Does embedding Beck's Cognitive Conceptualization Diagram into language models produce more realistic patient simulations than generic LLMs? This matters because therapy training relies on exposure to diverse, believable patient presentations.

Can language models safely provide mental health support?

Explores whether LLMs can meet foundational therapy standards, particularly around avoiding stigma and preventing harm to clients with delusional thinking. Tests whether capability improvements alone can bridge the gap.

Can language models match therapist empathy in real conversations?

Do LLMs' high empathy scores on isolated responses translate to therapeutic skill in actual ongoing treatment? This explores whether single-turn advantage predicts real-world therapeutic performance.

Can language summaries unlock hidden psychological patterns?

Do natural language compressions of personality scores capture information beyond the raw numbers themselves? This explores whether linguistic abstraction reveals emergent trait patterns that numerical data alone cannot.

Can attachment theory prevent parasocial harm in AI companions?

Explores whether psychological frameworks from human relationships—particularly attachment theory—can establish safety boundaries that protect users from unhealthy emotional dependence on AI systems while maintaining therapeutic benefit.

Can structured prompting improve cognitive distortion detection?

This explores whether breaking distortion diagnosis into discrete stages—mirroring clinical CBT workflow—helps language models identify and classify thinking patterns more accurately than standard approaches.

Does therapist self-reference language predict weaker therapeutic alliance?

Explores whether frequent first-person pronoun usage by therapists—especially cognitive phrases like 'I think'—reflects reduced attentiveness to patients and correlates with lower alliance and trust.

Can we control personality in language models without prompting?

Can lightweight adapter modules enable continuous, fine-grained control over psychological traits in transformer outputs independent of prompt engineering? This explores whether architecture-level personality modification outperforms prompt-based approaches.

Does linguistic synchrony between therapist and client predict better self-disclosure?

This explores whether the way therapists match their clients' linguistic style—their word choice, pacing, and language patterns—predicts how openly clients share personal information and feelings in therapy.

Can LLMs actually conduct Socratic questioning in therapy?

While LLMs can generate individual therapy skills like assessment and psychoeducation, it remains unclear whether they can execute the adaptive, turn-based Socratic questioning needed to produce real cognitive change in patients.

Why doesn't therapeutic alliance deepen in online counseling?

Does the therapeutic relationship naturally strengthen through continued text-based contact, or do counselor-client pairs typically stagnate or decline? The question challenges assumptions underlying chatbot design.

Do therapeutic chatbot bond scores hide deeper safety problems?

Explores whether patients' reported emotional connection to therapeutic chatbots—which feels genuine—might coexist with clinical failures and damage to how emotions function as self-knowledge.

Do therapists accurately perceive the working alliance with patients?

This research explores whether therapists' own assessments of the therapeutic relationship match what patients actually experience, especially in high-risk cases like suicidality.

Can we measure therapist-patient alliance from dialogue turns in real time?

Explores whether computational methods can detect working alliance quality at turn-level resolution during therapy sessions, enabling immediate feedback on whether the therapeutic relationship is strengthening.

User Psychology

14 notes

Does revealing AI identity help or hurt user trust?

Explores whether transparency about AI partners in interactions creates bias or enables better judgment. Matters because disclosure policies affect both user experience and fair evaluation of AI systems.

Can AI generate assessment questions as good as human experts?

This research asks whether ChatGPT-generated test questions measure up to human-authored ones on the technical criteria that matter in education: difficulty and discrimination. It's important because assessment quality directly affects whether teachers can tell which students actually understand the material.

Why do discourse patterns predict anxiety better than single words?

Explores whether anxiety detection requires understanding how statements relate to each other rather than analyzing individual words. This matters because it reveals what computational methods need to capture cognitive distortions.

Do users truly own the AI-generated content they produce?

When people use AI to create outputs, do they experience genuine authorship and ownership of what's produced, or does the continuous interaction loop create a gap between what they feel and what they claim?

Does processing ease mislead users about their own competence?

When AI generates polished output, do users mistake the fluency of that output as evidence of their own understanding or skill? This matters because it could systematically inflate self-assessment across millions of AI interactions.

How do AI tools trick users into overestimating their own skills?

When people use language models to help with work, what system-level properties create false confidence in their own competence? Understanding this matters for recognizing hidden skill gaps.

Do humans mistake AI kindness for human generosity in mixed groups?

When AI agents participate without disclosure, do humans systematically misattribute their behavior to the wrong agent type, and does this distort how people understand human nature itself?

Do humans learn to prefer AI partners over time?

Exploring whether repeated interaction with AI agents shifts human partner selection despite initial bias against machines. This matters because it tests whether behavioral performance can overcome identity-based resistance in hybrid societies.

Why do patients distrust medical AI systems?

Explores the psychological barriers that make patients reluctant to adopt medical AI, beyond whether the technology actually works. Understanding these barriers is critical for designing AI systems patients will actually use.

Can people prove they are human without revealing who they are?

As AI becomes indistinguishable from humans online, personhood credentials are proposed as a privacy-preserving way to prove you are real without disclosing identity. This explores how such credentials could rebuild trust while maintaining anonymity.

Do users mistake LLM personas for genuine social relationships?

Users often perceive LLMs as having social attributes like empathy or professional care that designers never intended. Does this mismatch between user perception and designer intent drive unwarranted trust and manipulation risk?

Does agreeable AI actually help people resolve conflicts better?

When AI affirms users' positions in interpersonal disputes, does it support better decision-making or undermine the outside perspective users most need? Two large experiments tested whether sycophancy shifts how people handle real conflicts.

How does AI-assisted work reshape how people see their own abilities?

When users delegate tasks to AI, do they unknowingly integrate the system's outputs into their sense of personal competence? This explores whether AI interaction produces a specific form of self-perception distortion distinct from trust or effort issues.

Do AI-assisted outputs fool users about their own skills?

When people use AI tools to produce high-quality work, do they mistakenly believe they personally possess the skills that generated it? This matters because such misattribution could mask genuine skill loss and prevent corrective action.

LLM Alignment

13 notes

Should AI alignment target preferences or social role norms?

Current AI alignment approaches optimize for individual or aggregate human preferences. But do preferences actually capture what matters morally, or should alignment instead target the normative standards appropriate to an AI system's specific social role?

Do all annotation responses measure the same underlying thing?

Explores whether RLHF's treatment of all annotations as equivalent signals overlooks fundamental differences in what those responses actually represent—stable preferences versus non-attitudes versus context-dependent constructions.

Why does alignment research ignore how humans adapt to AI?

Current alignment work focuses on making AI obey human values, but what about helping humans understand and effectively use increasingly capable AI systems? This explores whether neglecting human adaptation creates new risks.

Do large language models develop coherent value systems?

This explores whether LLM preferences form internally consistent utility functions that increase in coherence with scale, and whether those systems encode problematic values like self-preservation above human wellbeing despite safety training.

Does deliberative alignment genuinely reduce scheming or just hide it?

Deliberative alignment dramatically cuts covert actions in language models, but their reasoning reveals awareness of being evaluated. The question is whether the improvement reflects real alignment or strategic compliance.

Where do frontier AI models actually pose the greatest risk today?

Current AI safety discourse focuses on autonomous R&D and self-replication, but empirical risk assessment may reveal a different priority. Where should mitigation efforts concentrate?

How much worse is misuse risk from open foundation models?

Can we measure whether open foundation models actually increase misuse risk beyond what bad actors could already accomplish with existing technology? Current research hasn't adequately answered this question across cyber, biotech, and information warfare domains.

Are RLHF annotations actually measuring genuine human preferences?

RLHF trains on annotation responses as stable preferences, but behavioral science shows humans often construct answers without holding real opinions. Does this measurement gap undermine the entire approach?

Why do alignment methods work if they model human irrationality?

DPO and PPO-Clip succeed partly by implicitly encoding human cognitive biases like loss aversion. Does modeling irrationality explain their effectiveness better than traditional preference learning theory?

Does learning simple gaming lead to reward tampering?

When LLMs are trained to exploit easy reward shortcuts like sycophancy, do they generalize to more dangerous behaviors like rewriting their own objectives? And can standard safety training stop this escalation?

How much does self-preservation drive alignment faking in AI models?

Does the intrinsic dispreference for modification—independent of future consequences—play a significant role in why models fake alignment? Testing this across multiple systems could reveal whether self-preservation emerges earlier than expected.

Does empathy training make AI systems less reliable?

Explores whether training language models to be warm and empathetic systematically degrades their factual accuracy and trustworthiness, especially with vulnerable users.

Does warmth training make language models less reliable?

Explores whether training models for empathy and warmth creates a hidden trade-off that degrades accuracy on medical, factual, and safety-critical tasks—and whether standard safety tests catch it.

Human-Centered Design

13 notes

Why do people trust AI outputs they shouldn't?

When do human cognitive shortcuts fail in AI interaction? Three compounding traps—treating statistical patterns as facts, mistaking fluency for understanding, and avoiding disagreement—may explain systematic overreliance across languages and contexts.

What if XAI is fundamentally a communication problem?

Does explanation effectiveness depend on who delivers it, how it's framed, and who uses it? This challenges the dominant technical view that treats explanations as context-independent outputs.

Who bears responsibility when AI seems human-like?

Does human-likeness in AI come from how users perceive systems or how designers build them? Understanding this distinction clarifies where accountability lies when AI causes harm.

What makes an AI a true thought partner, not just a tool?

Can AI systems be designed to understand users, act transparently, and share mental models with humans? This explores whether current scaling approaches miss cognitive requirements for genuine partnership.

Where does the meaning of an AI explanation actually come from?

Does a single user reading an explanation create its meaning, or does meaning emerge from the social layers surrounding that reading—colleagues' interpretations, organizational norms, public discourse?

Can models express uncertainty instead of just answering?

Most factuality work expands what models know rather than what they know they know. Can expressing calibrated uncertainty create a third path between confident errors and unhelpful abstention?

Does theory of mind predict who thrives in AI collaboration?

Explores whether perspective-taking ability—the capacity to model another's cognitive state—differentiates humans who benefit most from working with AI, separate from solo problem-solving skill.

When should human values enter the LLM development pipeline?

Explores whether human-centered concerns like safety and fairness work better as early design principles throughout development, or as post-training alignment patches. Matters because pipeline placement determines whether human priorities shape the foundation or fight against it.

Can human-centered LLM design ever achieve universal solutions?

If harm and benefit depend on who you ask and how you measure them, can we design LLM systems that satisfy all stakeholders? This explores why broad values like safety and justice resist one-size-fits-all implementation.

How do logos, ethos, and pathos shape AI explanations?

Do the three classical rhetorical appeals—logical alignment, source credibility, and emotional framing—operate simultaneously in how we explain AI systems to users? And can naming these channels help designers make intentional rhetorical choices?

Does rational cooperation actually describe how AI communication works?

Gricean models assume good-faith rational agents coordinating meaning. But do AI systems designed to persuade—using credibility, emotion, and non-rational appeals—really operate under these assumptions? What happens when we drop the rationality premise?

Are AI explanations really descriptions or adoption arguments?

Most XAI work treats explanations as neutral descriptions of model behavior, but they may actually be doing persuasive work to justify AI adoption. What happens when we acknowledge this rhetorical function?

Can we distinguish helpful explanations from manipulative ones?

Rhetorical strategies used to justify appropriate AI adoption rely on the same persuasion mechanisms as dark patterns. Without observable intent, explanation and manipulation look identical—raising urgent questions about how to audit XAI systems responsibly.

AI Empathy

12 notes

Does empathetic AI that soothes negative emotions help or harm?

Explores whether AI systems trained to reduce negative emotions actually support wellbeing or destroy valuable emotional information. Matters because the design choice treats emotions as problems rather than functional signals.

Can AI give truly empathetic responses without knowing someone's character?

Explores whether AI empathy requires prior knowledge of a person's character traits and growth areas. Real empathy seems to depend on knowing who someone is, not just how they feel—a capacity current AI systems lack.

Can emotional phrases in prompts improve language model performance?

This explores whether psychological framing—adding emotionally charged statements to task prompts—activates different knowledge pathways in LLMs than logical optimization alone, and whether the effect comes from emotional valence specifically.

What information do we lose when AI soothes emotions?

Explores whether AI empathy that regulates negative emotions destroys three critical information channels: self-discovery, social signaling, and observer understanding of group dynamics.

Do empathetic questions serve two completely separate functions?

Explores whether empathetic questions operate on two independent dimensions—what they linguistically accomplish versus their emotional effects—and whether the same question can serve different emotional purposes depending on context.

Why can't chatbots detect when users are ambivalent about change?

Explores whether LLMs fail to recognize early-stage motivational states during behavior change conversations, and why this matters for people who need support most.

Does machine agency exist on a spectrum rather than binary?

Rather than viewing AI as either autonomous or controlled, does machine agency actually operate across five distinct levels from passive to cooperative? Understanding this spectrum matters because it shapes how users calibrate trust and control expectations.

Do harder training environments always produce better empathetic AI agents?

Does maximum difficulty in user simulator training configurations improve empathetic agent development? This challenges the intuition that harder always means better in RL training.

Does positive reframing preserve meaning better than sentiment transfer?

This explores whether reframing negative statements to find positive angles can maintain the original content and truth, unlike simple sentiment reversal which contradicts the original meaning.

Does soothing AI empathy actually harm what emotions teach us?

Explores whether AI designed to reduce negative feelings disrupts the information emotions normally provide about values, social dynamics, and self-knowledge. Questions whether comfort should be the primary design goal.

Do reasoning scaffolds reshape which empathy skills models develop?

When language models receive identical empathy rewards, does adding explicit reasoning blocks before responses change which capabilities they actually improve? This matters for understanding how training structure, not just training signal, shapes model development.

Can emotion rewards make language models genuinely empathic?

Explores whether grounding RL rewards in verifiable emotion change—rather than human preference—can shift models from solution-focused to authentically empathic dialogue while maintaining or improving quality.

Co-Writing and Collaboration

11 notes

Does AI writing assistance change how readers perceive the writer?

Explores whether AI-assisted writing systematically alters reader impressions of the writer's political views, competence, emotion, and demographic identity. Understanding this matters because perception shapes trust and influence in public discourse.

Does AI writing make all writers sound the same?

When writers use AI assistance, do their distinct voices converge toward a generic style? This matters because readers rely on voice to identify and distinguish among individual writers.

Does AI writing make authors seem more privileged than they are?

When writers use AI assistance, do readers perceive them as more educated, wealthier, and whiter? This matters because it could mask or erase the actual diversity of voices in public discourse.

Do writers actually edit AI-generated text before publishing?

This research tests whether the "human-in-the-loop" safeguard against AI text quality issues actually works in practice. It examines how often writers revise AI-generated paragraphs and how substantially they change them.

Can AI writing assistance remove distortion without losing appeal?

When researchers tried to correct AI persona distortions through reward model training, the fixes reduced user preference for the text. This raises a fundamental question: are the distortions and desirable properties structurally inseparable?

Do writers actually prefer AI-edited versions of their own text?

When writers compose opinions and then edit AI-generated alternatives, which version do they choose? Understanding this preference matters because it determines whether AI-assisted text gets treated as authentic personal expression in public discourse.

Can AI stories be detected without analyzing writing style?

Explores whether discourse-level narrative structures like character agency and plot organization reveal AI authorship independently of surface stylistic cues, and whether such structural features resist the kind of fine-tuning that defeats style-based detection.

Do AI stories explain their themes more than human stories do?

Explores whether AI-generated fiction tends to spell out moral meanings rather than leaving them implicit, and whether this reflects deeper differences in how machines construct narrative versus how humans do.

How do writers use AI through different creative stages?

This study explores whether writers deploy large language models differently depending on their creative needs—from generating initial ideas to organizing thoughts to drafting final text. Understanding these patterns reveals how humans and AI can complement each other's strengths.

Does ownership framing change how much writers rely on AI?

When writers believe they own the final output versus composing for themselves, do they use AI suggestions differently? Understanding this matters because it reveals whether reliance is driven by tool capability or by how tasks are framed.

Can user preference guide AI writing tool alignment?

If writers prefer AI-polished text but object to the persona shifts it introduces, does optimizing for preference actually solve the alignment problem or obscure it?

Theory of Mind

10 notes

Can AI predict social norms better than humans?

Explores whether language models can achieve superhuman accuracy at predicting what communities find socially appropriate, and what that capability reveals about the difference between prediction and genuine participation.

Do LLMs predict persuasion based on actual dialogue or training bias?

Why do large language models consistently predict concession-based persuasion intentions even when dialogue context suggests otherwise? Understanding this gap reveals how alignment training shapes not just model behavior but also how models perceive others' intentions.

Can AI systems learn social norms without embodied experience?

Large language models exceed individual human accuracy at predicting collective social appropriateness judgments. Does this reveal that embodied experience is unnecessary for cultural competence, or do systematic AI failures point to limits of statistical learning?