Does trust in an AI advisor track its actual accuracy, or do other cues matter just as much?
How much does the quality of an AI advisor's past performance actually influence future trust?
This explores whether people's trust in an AI advisor actually follows its track record (how often it was right before), or whether other things end up deciding how much they rely on it next time.
This explores whether trust in an AI advisor follows its track record, or whether something else is doing most of the work. The collection's short answer is that past performance matters, but only when people can actually see the results, and even then it competes with signals that have nothing to do with accuracy. None of these studies measures the size of the effect directly. Together, though, they show when a track record counts and when it gets ignored.
The clearest evidence that track record matters comes from studies where people are told they're working with an AI. Does revealing AI identity help or hurt user trust? found that people first avoid an AI partner once they know it's an AI, but that this bias reverses after repeated rounds where they can see the outcomes. The surprising part is that being told it's an AI, without any feedback on results, didn't help people calibrate at all. Performance only builds trust when it's visible and repeated. Expertise works the same way in human communities: Can AI ever gain expert community trust through participation? argues that experts earn authority through a judgment history that others can test over time. That note argues AI can't take part in that process, so being accurate in a single case doesn't turn into standing.
In practice, trust often comes loose from accuracy. Does conversational style actually make AI more trustworthy? found that ChatGPT users based their trust on how the exchange felt (responsive, fast, well-formatted) more than on whether the answers were reliable. How much should we trust AI-generated data in inference? describes a similar pattern in research workflows: people treat AI output as fully trustworthy by default because it sounds confident, not because it has earned that trust. Does personalization in AI increase trust or manipulation risk? adds that memory and personas can build trust through rapport, and the same mechanisms can be used to manipulate. So a smooth advisor can collect trust that its track record doesn't justify.
Trust can also drop even when performance is fine. In What makes people distrust AI agents they delegate to?, people pulled back from an AI agent on tasks that couldn't be undone and that others would see, such as sending an email, even when they rated the output as adequate. Tasks with high stakes that could still be corrected didn't trigger the same drop. In these moments, what decides trust is the cost of being wrong, not how often the AI was right before.
The idea you may not have expected to want: the setup of the help can matter as much as the track record. Can AI guidance reduce anchoring bias better than AI decisions? suggests that when an AI points out what to look at instead of handing over a verdict, people stop anchoring on its answer, so a good or bad record matters less. The open question this collection can't yet answer is a number: how many visible successes does it take to offset one visible, irreversible failure? If you find research on that, it would fill a real gap here.
Sources 7 notes
Users initially avoid AI partners when identity is revealed, but this preference reverses after repeated interactions with visible results. The learning mechanism—observing consistent outcomes—is essential; disclosure without feedback produces no calibration.
Expertise is validated through social participation and track record within expert communities, not individual accuracy alone. AI cannot enter this validation circle because it lacks social embeddedness, testable judgment history, and ability to participate in the consensus-building processes that define expert paradigms.
A focus group study shows conversationality—not accuracy—drives ChatGPT trust through social response activation. Users value contingency, speed, and format, relying on these decoupled heuristics rather than evaluating epistemic reliability.
Foundation Priors introduces λ as a tunable trust weight for synthetic data. Current workflows default to implicit λ=1 (full trust), driven by confidence signals and behavioral overreliance, causing both statistical contamination and measurable cognitive debt.
Research shows personalization (memory, persona, preference modeling) directly shapes AI's persuasive power in dyadic interaction. The same mechanisms that build trust also create manipulation potential, with outcomes determined by how systems are designed and deployed.
Show all 7 sources
In a controlled study of 20 students using a general-purpose AI agent, tasks that were irreversible and externally visible (like sending email) produced sharp trust drops and approval demands even when output quality was rated adequate. High-stakes but correctable tasks showed no such effect.
Learning to Guide eliminates anchoring bias and unassisted hard cases by having machines supply interpretive guidance rather than autonomous decisions, keeping responsibility with humans while improving their judgment through enhanced perception.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Assistant or Actor? Student Trust, Control, and Delegation Regret When Using a General-Purpose AI Agent
- Epistemic Deference to AI
- Humans learn to prefer trustworthy AI over human partners
- The Decision to Verify: How Warmth and User Characteristics Shape Reliance on Conversational Agents for Information Search
- Seeing to Think? How Source Transparency Design Shapes Interactive Information Seeking and Evaluation in Conversational AI
- From speaking like a person to being personal: The effects of personalized, regular interactions with conversational agents
- Can AI Explanations Make You Change Your Mind?
- GenAI as a Power Persuader: How Professionals Get Persuasion Bombed When They Attempt to Validate LLMs