How much did retiring a competition format hurt medal credibility?

The audit attributes about half of the collapse in one format's medal informativeness to institutional stranding. The platform "had been phasing out that format before AI, and its medal stock aged out on schedule." The authors call this "institutional stranding": "once the platform had phased out that format, the stock aged out under the pre-existing decay pattern." The credentials "stay on display at face value," so the medals remain visible after the format that issued them is gone.

The mechanism turns on what the two formats certify. Upload-competitions score submitted predictions and "cannot tell whether the process behind them was ever validated to be generalizable," while code-competitions run entrants' programs on hidden data, so a process that does not generalize is caught. The paper says the formats differ "in whether skipping the validation has consequences." Before the AI transition the two were statistically indistinguishable in informativeness (0.067 against 0.065), so the authors do not claim the surviving format gives the better signal. The shift to execution-based evaluation was already underway years before generative AI. A related boundary: old medals look more informative in the AI era only when read in isolation, because "a stale medal proxies for its holder's other signals."

This is a different route to the gap in Can a higher evaluation score hide poor task performance?. There a score drifts from task performance because optimization exploits the evaluator; here the score stays fixed while the format that gave it meaning is gone. Both show a recorded score keeping its face value after it stops tracking what it stands for. The paper frames the damage as a design matter: "how a platform weights aging credentials, whether it retires the format that issued those credentials, and how it displays what remains."

The excerpt gives the "about half" share but not how it was separated from the AI effect, so the method for that split is not stated here. It also gives only that the platform's reasons for retiring upload "predate generative AI," without naming them. The paper says a retiring platform "faces a stranding problem that the records themselves do not reveal," which points the design question toward how the display treats what remains; the excerpt does not test whether any display change would be used. An open question the excerpt cannot answer is whether stranding recurs on other platforms that retire an evaluation format. The paper extends its design lessons to platforms with public lifetime credentials, but this excerpt contains no second retirement case.