ChatGPT learned partly from Stack Overflow's answers, so what happened to the site once ChatGPT arrived?

This explores how much Stack Overflow content went into ChatGPT's training, and what the collection can say about it. The direct answer: nothing here measures that. None of these notes gives a count of tokens, posts or a share of ChatGPT's training mix that came from Stack Overflow, and OpenAI hasn't published that breakdown in anything collected here. What the collection does have is a sharper version of the question. It doesn't ask how much ChatGPT took from Stack Overflow. It asks what happened to Stack Overflow afterward.

The clearest evidence comes from a study that compared Stack Overflow with four similar Q&A sites that ChatGPT affected less. Weekly posts on Stack Overflow fell 16% after ChatGPT launched, and the gap grew to about 25% within six months Did ChatGPT cause Stack Overflow posting to decline?. The detail that answers your question indirectly is this: the drop was biggest for programming languages where ChatGPT had more training data. The authors couldn't count the data directly, but the pattern shows up anyway. Wherever the model had absorbed the most community knowledge, people stopped asking the community the most.

You might expect ChatGPT to have replaced only the easy, repetitive questions. It didn't seem to. Vote scores on Stack Overflow didn't change much after launch, which suggests good posts disappeared along with the duplicates Did ChatGPT displace only low-quality Stack Overflow posts?. The caveat is that votes are only a rough stand-in for quality, and no experts checked the posts. Together the two studies point to a loop: a public forum helps teach a model, the model draws questions away from the forum, and the forum produces less of the material that future models would learn from.

The same pattern appears beyond coding. Graphite's analysis of the web found that articles written mainly by AI rose to about half of new content within two years of ChatGPT and then leveled off Has AI-generated content stopped growing on the web?. On the individual level, a 55-student experiment found that programmers using ChatGPT scored higher on coding tasks than those using web search, but remembered fewer of the concepts and felt they owned less of their code Does ChatGPT help students code better but remember less?. Writing up an answer on Stack Overflow is one way programmers build that understanding.

The takeaway you might not have expected: the amount of Stack Overflow data in ChatGPT is unknown, but its effects can be measured. More training data in a topic went with a bigger drop in human questions on that topic. That raises a question the collection leaves open: as fewer people post, where will future models learn from?