Self-rated AI skills barely predict how people score on objective tests, yet hiring still runs on those self-reports.

This explores whether people's own ratings of their AI skills tell you anything about how well they actually perform with AI. The short answer from the corpus: almost nothing. A pooled analysis of three studies found a correlation of just .055 between self-reported and measured AI competence, and the confidence intervals include zero Can self-ratings replace objective performance scores for AI competence?. In practical terms, knowing how skilled someone says they are tells you close to nothing about how they'll score on an objective test. One caveat: these studies measure performance on competence tests, not long-term job output. The corpus doesn't yet have direct evidence linking self-reports to on-the-job results.

The gap matters because the hiring market runs on self-reports anyway. In a conjoint experiment with 1,725 recruiters, listing AI skills raised interview invitations by 8 to 15 percentage points across occupations. A certificate added only a modest boost over simply claiming the skill Do AI skills help candidates get more job interviews?. So recruiters reward a signal that barely tracks the thing it's supposed to signal.

Why are self-ratings so poor? The most interesting explanation in the corpus is that AI tools actively distort people's sense of their own ability. Researchers call this the "LLM Fallacy": when AI output is smooth and the line between your contribution and the machine's is blurry, you start counting the output as evidence of your own skill Do AI-assisted outputs fool users about their own skills?. Part of the mechanism is fluency. People use "that went easily and the result looks polished" as a cue that they must be capable, even though the polish came from the model Does processing ease mislead users about their own competence?. One analysis names four forces that compound each other: unclear credit for who did what, the fluency illusion, handing off the thinking itself, and pipelines too opaque to inspect How do AI tools trick users into overestimating their own skills?. Notably, this isn't the same as trusting a hallucination or over-relying on automation. You can misjudge your own competence even when the AI's answer is correct. That's why making the AI more accurate won't fix it. What helps is making the human and machine contributions visible How does AI-assisted work reshape how people see their own abilities?.

The noise goes both ways. Across four experiments with 4,439 people, AI users expected colleagues to see them as less competent and less diligent, and they were less willing to disclose that they used AI Do people fear judgment when they use AI at work?. So some people overclaim on a résumé, where AI skill is rewarded, and underreport at work, where AI use carries a stigma. Neither direction gives you an honest measure.

If self-reports don't work, what might? One promising lead is watching what people actually do. Patterns in coding-agent conversations, such as how people prompt, check, and redirect the agent, explained coding outcomes beyond what prior achievement predicted. The catch is that these traits weren't stable or transferable enough to count as teachable skills yet Can conversation patterns predict coding outcomes better than prior skill?. AI systems have a similar problem: agents that win benchmark contests still fail at long, realistic professional workflows, because the field measured contests rather than work Why do agent benchmarks not predict real economic value?. For humans and AI alike, the lesson is the same: competence that is claimed or measured in tidy conditions doesn't reliably carry over to real work.