"It was 80% me, 20% AI": Seeking Authenticity in Co-Writing with Large Language Models

Given the rising proliferation and diversity of AI writing assistance tools, especially those powered by large language models (LLMs), both writers and readers may have concerns about the impact of these tools on the authenticity of writing work. We examine whether and how writers want to preserve their authentic voice when co-writing with AI tools and whether personalization of AI writing support could help achieve this goal. We conducted semi-structured interviews with 19 professional writers, during which they co-wrote with both personalized and non-personalized AI writing-support tools. We supplemented writers’ perspectives with opinions from 30 avid readers about the written work co-produced with AI collected through an online survey. Our findings illuminate conceptions of authenticity in human-AI co-creation, which focus more on the process and experience of constructing creators’ authentic selves. While writers reacted positively to personalized AI writing tools, they believed the form of personalization needs to target writers’ growth and go beyond the phase of text production. Overall, readers’ responses showed less concern about human-AI co-writing. Readers could not distinguish AI-assisted work, personalized or not, from writers’ solo-written work and showed positive attitudes toward writers experimenting with new technology for creative writing.

Introduction. From text suggestion [40] and summarization [10] to style transformation [58], metaphor generation [36], and information synthesis [20], burgeoning applications of artificial intelligence (AI) for text production seem to be rapidly reshaping writing experiences and practices, especially with the recent high-profile releases of large language models (LLMs). Consequently, there are also concerns that vast transformations of the writer economy are likely underway [15, 37, 47, 53].

Within such a climate, seeking and preserving authenticity—as a cornerstone for all forms of creation—in writing content co-created with AI is likely to become an increasingly complicated yet critical matter for writers.

Indeed, existing literature has pointed to the importance of understanding authenticity for several reasons: From writers’ perspectives, authenticity often determines the value of their work, which co-writing with AI might potentially threaten [18]. Moreover, writing serves as the medium for writers to connect with their audiences, and authentic expression contributes to the soundness of such bonds [7, 38, 52]. A deeper understanding of authenticity also facilitates discussions around ownership of work [13] and relevant practices such as declaring authorship, regulating copyright, detecting plagiarism, and commissioning writers’ work. Recent work on AI use for writing has begun to explore relevant constructs, such as ownership, authorship, and agency [11, 13, 39, 49], but a more comprehensive understanding of Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org.

© 2024 Association for Computing Machinery. Manuscript submitted to ACM arXiv:2411.13032v1 [cs.HC] 20 Nov 2024 CSCW ’25, October 18 — 22, 2025, Bergen, Norway Hwang et al. views surrounding authenticity in human-AI co-creation remains elusive. Though public discussions reveal growing concerns from writers about the impact of AI on their work and profession [41], it remains unclear whether and how they would like to preserve elements of authenticity in writing.

In this work, we take a closer look at authenticity in writing from both writers’ and readers’ perspectives. We focus on what writers seek for authenticity as new practices of co-writing with AI emerge and whether personalization could support their goals. Furthermore, as personalized AI tools become readily available, we seek to understand the possible impact of personalization on writers’ ability to express and preserve their authentic voices in writing. Specifically, we ask:

• RQ1: How do writers and readers conceptualize authenticity in the context of human-AI co-writing?

• RQ2: Based on their conceptions of authenticity, do writers want to preserve their authentic voices in writing, and if so, how?

• RQ3: Can personalized AI writing assistance support authenticity and help preserve writers’ authentic voices (if desired) in writing, and if so, how?

Related work. The humanistic literature has long acknowledged authenticity as the core of human creative work [21, 42, 51]. Though there is a longstanding history of proposing theoretical frameworks for authenticity, researchers have not yet formed a consensus on the definition of authenticity. Still, they have often proposed three key themes to help define and conceptualize authenticity: Category, Source, and Value [2, 4, 8, 21, 23, 42, 43, 51, 62, 64]. First, the Category theme concerns whether a piece of work matches one’s existing beliefs about its associated category. The scope of a category can vary, ranging from a particular style or school of work (e.g., Bauhaus-style architecture) to a certain era (e.g., a Renaissance painting). Second, the idea of authentic Source concerns whether one can trace a piece of work to a specific source (i.e., a person, a place, an event, or any type of origin). This explains the importance of crediting writers and labeling the origins of work. In particular, when work from certain individuals is truly one-of-a-kind—such as the highly recognizable work of Picasso—audiences can easily identify them as the source of content. In such cases, the concepts of Category and Source become more blended and interchangeable. Finally, the Value theme is about whether there is consistency between a creator’s internal states and their external expression. This focuses on whether a writer’s perspectives, opinions, and values are consistent with what they expressed in their work.

In our study, we explore whether writers’ perceptions of authenticity in writing align with these concepts of authenticity for broader creative work and whether specific concerns apply to writing. Based on writer-centered definitions of authenticity we uncover, we further examine whether co-writing with AI is considered authentic and whether emerging AI technologies change writers’ views about the essence of authenticity.

2.2.1 Writers’ growing concerns regarding AI writing assistance. The increasing popularity of using AI for creative tasks has motivated recent work to investigate the possible impact of AI on several aspects of creators’ work, including credit, authorship, ownership, control, and agency [11, 13, 25, 35], many of which are closely related to authenticity. Recent studies, workshops, panels, and other forms of discussion [41] have thus far revealed mixed opinions from research communities, creators, and the general public toward these topics. Here, we summarize a few emergent themes:

Method. We examined writer-centered definitions of authenticity and the impact of AI writing assistance on authentic writing primarily through semi-structured, qualitative interviews with professional writers. To answer RQ3 and to enable participants to respond to our inquiries with situated experiences, we adopted two versions of AI-powered writing assistance tools: one with personalization through in-context learning, and one without personalization. We investigated how participants wrote with these tools in real time and delved into their co-writing experiences through interviews following each writing session. We complemented perspectives from these direct users of such emerging technology (i.e., writers) through a follow-up online study with indirect stakeholders (i.e., readers) [16]. Through these two parts of the study, we synthesize a more comprehensive view of authenticity in writing. The full study protocol (as illustrated in Figure 1) was reviewed and approved by the Institutional Review Board (IRB) of the authors’ affiliation.

Discussion. Our findings first reveal that writers conceptualize authenticity through the source of content, internal experiences and identities that ground their work, and the actual writing outcomes. Although participants’ reflections did resonate with some of the definitions of authenticity as established in the existing literature (i.e., category, source, and value), they placed further emphasis on viewing authenticity through their internal experiences in addition to through their explicit expressions in writing. Moreover, regardless of how participants defined and understood authenticity, many of them indicated that authentic writing is the essence of good writing, and saw the likely impact of AI on authenticity in CSCW ’25, October 18 — 22, 2025, Bergen, Norway Hwang et al.

Dependent variable Test goal Model tested Part 1: Writers’ Behavioral Data Frequency of requesting AI assistance; Acceptance rate of AI suggestions To compare behavioral patterns of each writer in the personalized vs. non-personalized condition lmer(DV ∼condition + session order + (1|writerID), data) Part 2: Readers’ Self-report Data Familiarity with generative AI; Interest in reading AI-assisted writing; Authenticity of AI-assisted writing To understand readers’ existing perception toward AI-assisted writing by examining the distribution of each variable wilcox.test(DV, μ= 3, conf.int=TRUE) Likeability, enjoyment, and creativity of writing; Likelihood of human writing To compare each reader’s perception after reading the three writing passages (writers’ solo work, work co-written with personalized AI, and work co-written with non-personalized AI) lmer(like ∼condition + reading order + (1|authorID) + (1|readerID), data) Degree of preserving writers’ authentic voices; Credits and authorship of work To examine how each reader compare the work co-written with personalized AI vs. nonpersonalized AI to a writer’s solo work respectively lmer(like ∼condition + reading order + (1|authorID) + (1|readerID), data) Perception of the writing; Perception of the human writer; Appreciation and evaluation of writing To understand readers’ perception toward AIassisted writing after reading AI-co-written work by examining the distribution of each variable wilcox.test(DV, μ= 3, conf.int=TRUE) Table 5. Statistical models used for quantitative data analyses writing. We provide summaries and quotes of each participant’s conceptions of authenticity in Table 6 and further elaborate on the three key themes of writer-centered definitions of authenticity from Sections §4.2.1 to §4.2.3.

4.2.1 Defining authenticity through the source of content (Source authenticity). Several writers described authenticity as a “who” question, focusing on who wrote the text or was the source of the content. This concept mirrors Source, a long-standing theoretical component of authenticity in the existing literature, and is directly related to both writers’ and readers’ considerations for authorship. Participants whose conceptualizations aligned with Source authenticity emphasized the entity who took actions and contributed to a piece of work. For example, such actions might include producing a piece of text or trying to understand the audience’s interest.

Writers who held this view also saw AI writing assistance as a direct threat to authenticity. With AI participating in the writing process, writers are no longer the sole source of content generation, raising questions about the authenticity several internal states during their writing processes as key constructs of authenticity, many of which have been less covered by prior literature.