Do Kaggle medals still predict performance after AI arrived?

The audit finds that Kaggle's medals kept their informativeness through the AI transition. Across 444,698 participations in the platform's 2010–2026 archive, a medal's "power to predict performance sits almost entirely in its first year, in both formats and eras," and "fresh medals kept most of their value through the AI transition." The outcome is one minus a team's final percentile on the hidden-test leaderboard, and every medal is won on predictions scored against withheld answers, so the credential certifies measured performance rather than the artifacts behind it. The authors read this as "none matches the fear that credentials are now worthless."

The mechanism they give is perishability plus a display problem. Lifetime tiers (Expert, Master, Grandmaster) count medals regardless of age, and the paper says the tiers "discard up to a sixth of the information in the medals," or 13–16% of the available information in sample. A recency-weighted index fit on pre-AI outcomes explains AI-era performance about 13% better than the tiers and selects entrants who perform better on average, though the tiers still identify extreme top performers better. In 26% of AI-era participations by tiered entrants, every medal behind the tier is more than a year old, which is how the stale signals reach the display.

Against the nearest notes, this applies a scarcity argument to credentials. What makes accountable judgment scarce when AI cognition is cheap? holds that once first-pass cognition is cheap, the scarce asset is accountable judgment; the paper's line that generative AI "did not make a top-rank credential cheap to achieve" makes the same point about a verified record. Kaggle also bears on Do university AI policies actually protect what credentials mean?: the platform "does not prohibit AI assistance in either format," so permission does not settle validity. What keeps the medals meaningful is scoring against withheld answers, the kind of evidence standard that the policy audit finds stated less clearly than the permission boundaries. The medals are also the objective kind of record that Can self-ratings replace objective performance scores for AI competence? says self-reports cannot stand in for.

The excerpt does not establish how the medals are used, because it observes no buyer or employer response. The estimates are "associational," measuring "how well credentials predict performance, not why," and the authors call the demand response "the natural next step." They also state that whether AI changes human skill is "unidentifiable here by construction," and they make no such claim. The sample is one platform. The authors expect the three display lessons to carry to platforms with public lifetime credentials, but that is an extrapolation the excerpt does not test. The excerpt also ends during the authors' list of limitations for the AI-era sample, so those limits are not stated here. The implication at the strength the evidence allows is narrow: for a verified credential, the question is how it is aggregated and displayed, and the claim that verification kept Kaggle medals informative needs a second platform before it generalizes.