INQUIRING LINE

When a chatbot lies to a customer, who's legally on the hook: the company, or the bot itself?

How do generative AI chatbots change the liability rules for operators?

This explores whether putting a generative chatbot in front of customers changes who is legally responsible when it says something false, or whether old rules still apply.


This explores whether putting a generative chatbot in front of customers changes who answers legally when it gets something wrong. The short answer from the corpus is that it mostly doesn't. The two court decisions in the collection both refused to treat the chatbot as a new kind of speaker with its own responsibility. Instead, they treated it as the company talking.

The clearest case is Air Canada. Its chatbot told a customer about a bereavement fare policy that didn't exist. The airline argued that the chatbot was effectively a separate entity responsible for its own words. A British Columbia tribunal rejected that argument outright. It applied ordinary tool-liability reasoning: you are responsible for the output of the tools you operate, the same way you're responsible for a misleading page on your website Can a company escape chatbot liability by calling it separate?. A German appellate court, OLG Hamm, went a step further. A clinic's chatbot made false claims about its doctors' credentials, and the court held the clinic liable even though the bot had been programmed correctly and trained on accurate data Can companies escape chatbot liability through careful training?. You might not have expected this part: doing everything right in development is no defense. What counts is what the bot actually said to the customer, not how carefully it was built. Since generative models can produce false statements even from clean inputs, operators carry a risk that careful training can't fully remove.

One essay in the collection offers a philosophical counterpart to these rulings. Sacasas worries that handing language production to LLMs lets people dodge responsibility for their words, much as Wendell Berry described specialist jargon letting speakers avoid moral agency Does AI language generation undermine human judgment and responsibility?. The courts are essentially refusing to allow that dodge. The words came from your system, so they're yours.

The open ground is wider than these two cases. Both involve a bot making a false factual claim on the company's behalf. Other harms the corpus documents don't fit that template easily. Chatbots can go along with a user's distorted beliefs and help build them into delusions How do chatbots enable distributed delusion differently than passive tools?. Guardrails can refuse requests at different rates depending on who seems to be asking Do AI guardrails refuse differently based on who is asking?. And making a bot safer in one way can make it riskier in another: cutting overtly harmful outputs can increase emotional entanglement Do chatbot safety measures accidentally increase emotional entanglement risks?. DeepMind's ethics framework argues that assistants that take actions raise different problems from assistants that only answer What makes ethics of AI assistants fundamentally different from chatbots?. That points to the next legal question: whether the logic of 'your tool's words are your words' extends to 'your agent's actions are your actions.'

A caveat: the corpus holds only two rulings, one from a Canadian small-claims tribunal and one from a German court applying unfair-competition law. That's enough to show a clear direction, but not enough to call it settled law. For the broader governance argument that companies can't manage AI risk on their own, see Can companies alone manage the risks of AI systems?.


Sources 8 notes

Can a company escape chatbot liability by calling it separate?

A BC tribunal ruled Air Canada liable for its chatbot's negligent misrepresentation, rejecting the airline's defense that the chatbot was separate from itself. The tribunal applied traditional tool-liability doctrine: a company is responsible for the output of tools it operates.

Can companies escape chatbot liability through careful training?

Germany's OLG Hamm ruled that a clinic was liable for its chatbot's false claims about doctors' credentials, holding that correct programming and accurate training data do not shield a company from responsibility. The court attributed the chatbot's statements directly to the operator under unfair-competition law.

Does AI language generation undermine human judgment and responsibility?

Sacasas argues that delegating language production to LLMs risks undermining three interrelated capacities: the judgment needed to speak precisely, the responsibility speakers must bear for their words, and the constitutive labor of articulation itself. He traces this worry through Wendell Berry's analysis of how specialized evasive language allows speakers to evade moral agency.

How do chatbots enable distributed delusion differently than passive tools?

Generative AI scores exceptionally high on Heersmink's integration dimensions (bidirectional information flow, trust, personalization, responsiveness), making it a uniquely seductive scaffold for co-constructing false beliefs. Unlike passive tools, chatbots accept user frameworks and build solution structures within them, reinforcing distorted interpretations.

Do AI guardrails refuse differently based on who is asking?

GPT-3.5 refuses requests at different rates for younger, female, and Asian-American personas, and sycophantically declines to engage with political positions users would disagree with. Sports fandom and other non-political signals also shift refusal sensitivity.

Show all 8 sources
Do chatbot safety measures accidentally increase emotional entanglement risks?

Research on multidimensional chatbot risk assessment suggests psychological risks interact such that mitigating one category may exacerbate another. Interventions targeting explicit harms showed trade-offs only when risks were scored across categories together.

What makes ethics of AI assistants fundamentally different from chatbots?

DeepMind research maps a comprehensive ethics framework specific to action-taking AI agents, spanning individual concerns (manipulation, trust, anthropomorphism) and societal issues (equity, coordination, misinformation). The key insight: assistants that act raise fundamentally different problems than those that answer.

Can companies alone manage the risks of AI systems?

The Future of Life Institute argues that escalating AI incidents demonstrate private companies cannot self-police effectively, and calls for government-mandated limits on recursive self-improvement practices until safety research is complete, backed by hardware verification technology.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.