Signaling in the Age of AI: Evidence from Cover Letters

Abstract We study the impact of generative AI on labor market signaling using the introduction of an AI-powered cover letter writing tool on a large online labor platform. Our data track both access to the tool and usage at the application level. Difference-in-differences estimates show that access to the tool increased textual alignment between cover letters and job posts and raised callback rates. Time spent editing AI-generated cover letter drafts is positively correlated with hiring success. After the tool’s introduction, the correlation between cover letters’ textual alignment and callbacks fell by 51%, consistent with what theory predicts if the AI technology reduces the signal content of cover letters. In response, employers shifted toward alternative signals, including workers’ prior work histories.

Introduction. Job seekers and college applicants increasingly turn to generative artificial intelligence (AI) tools like ChatGPT to edit r ́esum ́es, craft cover letters, and prepare for interviews.1 Despite the transformative potential of this technology for processes central to economic opportunities and social mobility, credible empirical evidence on how AI affects labor market signaling remains scarce.2 The main obstacle is observability: outside of controlled lab settings, just as employers struggle to identify AI- assisted applications, so too do researchers. Current detection tools are error-prone and unreliable (Weber-Wulff et al. (2023)), making it difficult to assess AI assistance’s real-world impact on hiring.

We overcome this challenge by studying the introduction of a generative AI cover letter writing tool on Freelancer.com, one of the world’s largest online labor platforms. Freelancer connects international workers and employers to collaborate on short-term, skilled, and mostly remote jobs.

On April 19, 2023, Freelancer introduced the “AI Bid Writer,” a tool that automatically generates cover letters tailored to employers’ job descriptions that workers can use or edit. The tool was available to a large subset of workers depending on their membership plans. Employers, as is typical in most real-world settings, could not observe which workers had access to the tool or whether a given cover letter was AI-assisted. We use eight months of data from two leading skill categories, PHP and Internet Marketing, covering 5 million cover letters submitted to over 100,000 jobs.3 We observe access to the tool and usage of the tool at the application level. We link access and usage information to the submitted cover letters and to callback and hiring outcomes to analyze the effect of generative AI at both the individual level and the market level. We also observe timestamps of workers’ clicks on the AI tool and their application submissions, allowing us to measure the time they spent editing the AI-generated texts and examine the effect of human-AI interaction.

The tool’s rollout and the granularity of the data enable us to answer fundamental questions raised by the widespread use of AI in job applications: (i) Does generative AI writing assistance improve an applicant’s chances of receiving a callback? (ii) Does AI complement or substitute existing writing ability? (iii) As AI adoption grows, do employers rely less on cover letters as signals, and do other signals become more important? (iv) Does AI adoption affect overall hiring rates? (v) Do workers revise the AI drafts and do their revisions matter for hiring outcomes?

Related work. Galdin and Silbert (2025) offer complementary evidence on the effects of generative AI on signaling in the same labor market. They use an LLM-based measure of cover letter quality and find, consistent with our results, that its correlation with application outcomes declined after AI tools became available. While we provide difference-in-differences evidence on the platform policy and evidence on human–AI interaction, they estimate an equilibrium model and perform counterfactual simulations. Another closely related paper is Cowgill et al. (2024), who examine the impact of generative AI on job applications and startup pitches through a survey experiment. They find that generative AI reduces employers’ and investors’ screening accuracy. Our study explores similar questions in an online labor market, examining the effect of AI on actual hiring decisions. An additional important difference between the settings is that Cowgill et al. (2024) informed employers whether applicants used generative AI, whereas, in many real-world contexts—including ours—employers typically cannot discern whether an applicant received assistance from AI. As a result, our findings are not directly comparable.

In a similar labor market, Wiles et al. (2023) study the effect of a non-generative algorithmic writing assistance on employment through an experiment. The intervention they study focuses mostly on correcting spelling and grammar errors. They show that the tool improved treated workers’ employment outcomes without affecting the employers’ satisfaction. While we also investigate the role of AI—although of a different nature—in workers’ employment outcomes, we are additionally able to speak to the market-level outcomes associated with the introduction of generative AI.

Another related paper is Wiles and Horton (2025), who study generative AI assistance to employers for crafting job posts in a similar context and find that while the tool increased the number of job posts, it did not increase the number of matches. Our work is complementary to theirs as both sides of the labor market—firms and workers—increasingly adopt generative AI tools in matching.

Also related is work that examines the functioning of online labor markets and their broader economic implications. Horton (2010), Chen and Horton (2016), and Stanton and Thomas (2020) document how these markets operate, the behavioral responses of workers to wage changes, and the significance of platform-mediated employment. Stanton and Thomas (2025) provide evidence on the distributional consequences of gig platforms, showing how access to online markets affects both workers and firms, and highlighting heterogeneity in who ultimately benefits from platform participation.

Method. In this section, we describe the platform and the generative AI–powered cover-letter writing tool that constitute the focus of our analysis.

2.1 Freelancer.com Our analysis employs data from Freelancer.com, which ranks among the world’s largest online task-based labor markets both in terms of user base and the volume of posted jobs. Platforms like Freelancer.com, Upwork, and Fiverr serve as digital marketplaces that connect workers and employers worldwide, facilitating the exchange of remotely delivered tasks such as software development, sales and marketing support, and creative work—commonly referred to as “freelancing.” Since its inception in 2009, the platform has attracted over 83 million users from 247 countries, regions, and territories, with over $1 billion in gross payments transacted.

On Freelancer.com, the matching process unfolds in five main steps. First, firms or individual employers seeking to outsource a task post a project— also referred to as a ‘job’ in this paper—on the platform. The employer describes the job and specifies a minimum budget, which is the lowest amount a worker can bid. Most jobs are fixed-budget rather than hourly. We interpret the minimum budget as a measure of the job’s size, and in all subsequent analyses we normalize workers’ wage bids by this minimum budget. Second, workers with skills matching the job requirements submit bids on these jobs. Bidding for jobs on Freelancer.com is free for workers; however, the number of bids a worker can submit each month is limited. Workers can increase their monthly bidding allowance by subscribing to one of the platform’s membership plans (see https://www.freelancer.com/membership/). Each bid typically comprises two components: the proposed payment amount and a written job proposal, which serves as a cover letter outlining the worker’s suitability for the task. Third, based on the set of submitted bids and the characteristics of the competing workers, the platform employs a recommendation algorithm (as a function of the volume and strength of a bidder’s past reviews, referred to henceforth as the “bidder review score”) to rank the bids for employers. Fourth, employers review the bids, displayed in order of ranking, and select the most appealing one, initiating the work process. Alternatively, employers may opt to leave the job unfilled if none of the bids meet their requirements. During this process, employers can and often reach out to a subset of bidders for a private conversation, which we refer to as a callback.

Finally, after the task is completed, both the worker and the employer rate each other’s performance by assigning a star rating, which contributes to their respective reputations on the platform. Payments are handled through the platform and are typically divided into milestone payments.

2.2 The AI Bid Writer On April 19, 2023, Freelancer.com launched the AI Bid Writer. With a single click, the tool automatically generates a cover letter tailored to the job description, thereby reducing the effort and time required for freelancers to bid. While the auto-generated cover letters are ready for immediate submission, freelancers are free to edit the text before submission. The release of the AI Bid Writer was unannounced beforehand, ensuring that its adoption was unaffected by pre-release anticipation. The tool is available exclusively to users subscribed to the Plus or higher membership tiers. The Plus plan costs $8.95 per month and permits up to 100 bids, compared with 6 bids for free members and 50 bids for Basic members, who pay $4.99 per month. Using the AI Bid Writer incurs no additional cost4. In our data, we observe workers’ membership plans—and thus whether they had access to the AI Bid Writer—as well as whether they clicked on the tool’s button for each bid submitted.

Discussion. The absence of aggregate effects on hiring outcomes may appear surprising given the substantial reallocation of informational content documented in Sections 6.1 and 6.2. However, this pattern is consistent with the interpretation that while AI disrupted one channel of signaling—cover letters—employers successfully shifted toward others, particularly past reputation scores, which partially substituted for the diminished informativeness of written applications. In this sense, the market may have adjusted to maintain equilibrium matching rates, despite changes in the informational environment.

Figure 14: Share of jobs Interviewing, Awarding, and Completing Figure Notes: The above figure plots the percentage of jobs such that i) the employer engaged in at least one interview ii) the employer awarded the job iii) the employee successfully completed the job. A total of 106,714 jobs are represented. Due to data confidentiality, the y-axis is normalized. The maximum value is shown as ‘n’, and other ticks represent fractions of n.

We view this result as preliminary and emphasize two important caveats. First, our analysis focuses on the immediate aftermath of the AI tool’s release, when cover letters remained one of several signals available to employers. Over time, as AI tools become more sophisticated and begin to erode additional dimensions of signaling, employers may find themselves with fewer credible indicators on which to rely. Second, our setting involves short-term freelance jobs where past reviews and platform-calculated rankings are readily available and highly salient. In other labor markets—such as those for first-time job seekers or college admissions—alternative signals may be weaker or absent, leaving fewer substitutes when written applications lose their informational value. Taken together, our findings suggest that the immediate equilibrium impact of AI on hiring rates is limited, but this should not be interpreted as evidence that the technology is innocuous for market-level matching. Rather, it may reflect employers’ ability to substitute toward other signals in the short run, which can generate distributional consequences: workers with more experience on the platform tend to hold higher bid scores. Whether such substitutability will persist as AI adoption spreads more broadly across application components remains an open question.

Conclusion. This study provides empirical evidence on the labor market consequences of AI-assisted job applications. While AI tools allow freelancers to produce more polished and tailored applications with less effort, our findings suggest that they fundamentally reshape how employers interpret cover letters. The widespread adoption of AI-assisted writing diminishes the informational value of cover letters, weakening their role as a hiring signal. In response, employers place greater weight on alternative signals, such as past work experience, that are less susceptible to AI. We find no evidence that this shift in signaling has affected overall hiring rates. Finally, we show that although most workers submit AI-generated drafts with minimal revision, within a worker, more time spent editing is associated with higher hiring success. Online labor platforms provide a unique data opportunity to study the growing use of AI in job applications. At the same time, important differences from conventional labor markets mean that further research is needed to assess whether these findings generalize more broadly. As AI technologies continue to advance, ongoing research will be essential to understand their implications for labor market signaling and screening.

Limitations. We view this result as preliminary and emphasize two important caveats. First, our analysis focuses on the immediate aftermath of the AI tool’s release, when cover letters remained one of several signals available to employers. Over time, as AI tools become more sophisticated and begin to erode additional dimensions of signaling, employers may find themselves with fewer credible indicators on which to rely. Second, our setting involves short-term freelance jobs where past reviews and platform-calculated rankings are readily available and highly salient. In other labor markets—such as those for first-time job seekers or college admissions—alternative signals may be weaker or absent, leaving fewer substitutes when written applications lose their informational value. Taken together, our findings suggest that the immediate equilibrium impact of AI on hiring rates is limited, but this should not be interpreted as evidence that the technology is innocuous for market-level matching. Rather, it may reflect employers’ ability to substitute toward other signals in the short run, which can generate distributional consequences: workers with more experience on the platform tend to hold higher bid scores. Whether such substitutability will persist as AI adoption spreads more broadly across application components remains an open question.