Did ChatGPT displace only low-quality Stack Overflow posts?

The second finding concerns what was displaced. Votes are the excerpt's only quality signal, described as "simple forms of social feedback provided by other users to rate posts." The introduction reports "no change in the votes posts receive on Stack Overflow since the release of ChatGPT," and concludes that ChatGPT "is displacing a wide variety of Stack Overflow posts, including high-quality content." The discussion uses softer wording, "no large change in social feedback," and reads it as "suggesting that average post quality has not changed."

The inference rests on votes standing in for quality. The abstract states the result plainly: posts made after ChatGPT "get similar voting scores than before," which the paper says suggests ChatGPT "is not merely displacing duplicate or low-quality content." To argue that duplicates cannot explain the drop, the paper puts duplicates at "only 3% of posts," citing Correa and Sureka (2013), and notes that it does not observe significant changes in voting outcomes.

Against the nearest notes, this sits next to Can crowdsourced votes reliably rank language models?. There, crowd votes earn credibility because they agree with expert raters. This excerpt has no such validation for Stack Overflow votes, and it says so: quality can be assessed "only partially" through up- and downvotes. The first note in this pair covers how much posting fell. This one asks what was lost, and its answer depends on a proxy the excerpt does not check against expert judgment.

The excerpt raises the obvious alternative: "Users may be posting more challenging questions, ones that LLMs cannot (yet) address." It leaves this to future work, asking whether later activity "is more complex or sophisticated on average," and does not test it. So the flat votes support a narrower claim than "only weak content left." By this measure the decline is not confined to weak posts, but the excerpt cannot say whether the remaining or lost questions were harder. That matters for the commons, because the open pool's value to future models depends on what it still contains, and the excerpt leaves that unanswered.