Do people prefer AI moral reasoning when they don't know the source?

The Moral Turing Test paper documents a dissociation in human responses to AI moral reasoning. Two findings, in tension:

LLM justifications are preferred in complex moral scenarios. When evaluating responses to trolley problems and other personal moral dilemmas, participants preferred LLM-generated justifications over human ones. LLMs exhibit stronger utilitarian framing in high-stakes personal scenarios — a framing that participants found more appropriate for deliberative, complex ethical decisions. In non-moral scenarios (low stakes), human justifications were preferred.

Systematic anti-AI bias persists. Even participants who preferred LLM justifications in content reported less agreement when they believed the source was AI. "Humanizing" features — introducing typos, making language less pedantic — reduced but did not eliminate detection advantage. The preference for the content and the rejection of the source are independent.

This dissociation is significant because:

1. It shows the "observer/participant perspective" distinction in action (Do humans and LLMs differ fundamentally or just superficially?). As participants in evaluating moral reasoning (without source knowledge), humans respond to the argument on its merits. As observers who know the source, they apply categorical AI/human distinction.

2. It suggests human preferences for AI reasoning may be underestimated in deployment, where source labeling reduces agreement. The actual quality of AI moral reasoning may exceed what labeled deployment reveals.

3. Subtle linguistic differences remain detectable. LLMs use more first-person pronouns? No — humans use more first-person pronouns. LLMs produce "more pedantic, analytical" explanations. These cues give moderate detection accuracy (higher in moral scenarios than non-moral ones).

The anti-AI bias is "robust to humanizing efforts" — which suggests it is not primarily driven by superficial linguistic cues but by something closer to categorical prejudice about the source of reasoning. An AI that produces content humans genuinely prefer, under conditions where the source is unknown, is rejected when the source is revealed. The content and its source are evaluated by different psychological processes.

Behavioral evaluation reveals deeper structural divergence. A Dictator Games study (Can Machines Think Like Humans?) extends this finding beyond moral judgment to economic decision-making. LLM agents exhibit bimodal (not continuous) decision distributions — they default to extreme generosity or extreme selfishness, lacking the nuanced variation characteristic of human choices. "The absence of a continuous decision space indicates that LLMs may be defaulting to prevalent patterns in their training data or adhering to the most statistically probable responses." This produces a fundamental dilemma: "Should LLMs be designed to mimic human-like uncertainty, embracing the complexities and unpredictabilities of human decision-making, or should they aim for determinism to ensure consistency and predictability?" The paper concludes that "LLMs are tools to assist in research, not substitutes for human participants" — the preference dissociation from the Moral Turing Test and the behavioral divergence from Dictator Games converge: LLM outputs can be preferred over human outputs on content while being categorically different in the cognitive process that produced them.

Source: Philosophy Subjectivity