What Influences Readers' and Writers' Perceived Necessity of AI Disclosure?

The growing capability of artificial intelligence (AI) leads to its increasing adoption in writing, spurring discussions around whether writers should disclose their AI use in writing. What influences the perceived necessity of disclosure? We look into this question from three dimensions: perspective (reader or writer of the text), purpose (the goal of reading or writing), and procedural factors (how AI was used in the writing process in terms of replaceability, effortfulness, intentionality, and directness). In a vignette study (N= 727), we find that readers consider disclosure to be more necessary than writers, and disclosure is regarded as more necessary when AI’s contribution in writing is irreplaceable, directly incorporated, and when the writer does not intentionally steer AI generation. To our surprise, the writers’ intentionality of AI use produces contrasting effects on readers’ and writers’ perceived necessity of disclosure. Moreover, the effort of writing shows no significant effect on the perceived necessity. This study contributes to the conversation on transparent AI use by revealing readers’ and writers’ grassroots judgments, providing a unique angle to reflect on existing regulations, and offering insights into how AI disclosure guidance and tools could be designed to better align with readers’ and writers’ perceptions.

Introduction. Many people are turning to AI to assist with their writing process [11, 70, 129], with goals ranging from grammar check to automated content generation, as well as practices that span using AI-generated text as a reference to adopting it verbatim [63, 109, 128]. Today, AI-generated text is often indistinguishable from human-written text, especially when AI-generated text is partially and incrementally blended into a larger piece of writing [32, 130]. While this marks an exciting advancement of AI technologies, it makes it extremely challenging for readers to tell who created the content that they are reading [50]. Blurred creatorship makes it difficult for readers to contextualize the text, discern its intent, and undermine readers’ ability to judge its accountability and trustworthiness of writing content [118]. Furthermore, numerous studies show that AI may hallucinate (i.e., generate false information that sounds plausible) and undisclosed AI-generated text may be weaponized to exacerbate the spread of misinformation [51, 88, 132]. Given that AI generation detectors fall short in accuracy and reliability, especially as AI models and writers’ practices (e.g., prompting and editing strategies) continue to evolve [57, 59, 130], transparent AI use relies largely on writers’ AI disclosure—the practice of disclosing AI contribution in content creation [96, 124]. Disclosing sources of contributions is well established in human writing; clearly acknowledging human contributors in a collaborative work has long been an ethical and moral norm [48, 81, 103]. However, disclosing AI use in writing has yet to become a common practice [24]. As we are still in the early phase of adapting to an ecosystem with a significant presence of AI involvement, only a small portion of platforms, publishers, and organizations have started to explicitly specify the acceptable use cases of AI and standards of AI disclosure (e.g., [7, 84]). In most cases, however, the expectation of AI disclosure remains ambiguous. Under such circumstances, some writers choose not to disclose their AI use, or even intentionally hide it, due to concerns such as their work being devalued [1, 134]. This writers’ tendency toward non-disclosure diverges from readers’ expectations. For instance, most news readers want journalists to disclose how AI is used in newsrooms [119]. While there appears to be a clear gap between readers’ expectations and writers’ practices, it is not yet clear whether there is an underlying gap between readers’ and writers’ perceptions of the necessity of AI disclosure. With the ultimate goal of aligning disclosure behaviors with ideal practices, we need to first understand the expectations of AI disclosure from the standpoints of readers and writers. Currently, AI disclosure guidelines (which represent the “necessity” of AI disclosure) are usually formulated through a top-down approach by people who implement and enforce regulations (e.g., policymakers, publishers, or community leaders and moderators of content-sharing platforms), who are not the ones directly affected by the writings and AI disclosure [82]. In this study, we take a bottom-up approach by exploring readers’ and writers’ “perceived necessity” of AI disclosure, revealing the grassroots perceptions of stakeholders who directly interact with the writings and are affected by AI disclosure. We believe that our readers- and writers-centered inquiry into AI use transparency is valuable for the FAccT community as it enriches the discussion around AI transparency. As Corbett and Denton [21] pointed out, transparency research in FAccT (1) has mostly focused on algorithmic transparency provided by regulators or technical experts, overlooking AI users’ approach to transparent AI use (e.g., writers in our study context); (2) should place greater emphasis on bridging the gulf between providers of transparency (e.g., writers) and recipients of transparency (e.g., readers); (3) should provide transparency in a way grounded in the needs of end-users (e.g., readers and writers, as opposed to regulators or technical experts). While some recent work is related to (1) and (3)—for example, He et al. [42] designed an attribution toolkit and Hoque et al. [44] proposed an interactive system to facilitate writers’ disclosure—these efforts focus on the providers of transparency, leaving the gap between transparency providers (writers) and transparency recipients (readers) underexplored. We take a step toward exploring this missed opportunity in the space of AI disclosure.

Related work. 2.1 Human-AI collaborative creation and AI-assisted writing The rise of AI capabilities and applications enables human-AI collaborative creation (co-creation) of texts [34, 47, 63], music [30, 79, 85], images [25, 62, 87], and videos [18, 46, 74]. AI-assisted writing is a type of content co-creation that becomes increasingly common across academic [1, 3], professional [41, 68–70, 77], and personal contexts [43, 73, 114]. Literature in writing theory commonly views writing as a process involving recursive cognitive activity rather than merely a product or an output [40, 83]. Today, writers can use AI to support in all stages of their writing process [29], from planning and ideation [33, 105], to information gathering [42, 77], drafting [34, 89], and reviewing [63, 78]. AI involvement may alter both the form and the content of the written text [102], through editing writers’ drafts to correct spelling and grammar errors [15, 27] and refining logic and flow to improve writing clarity [126, 133]. Through the lens of writing ecology [20, 94], the continuous integration of AI in each granular step of the writing process may reconfigure the writer’s metacognitive engagement with their own work.

Writers’ diverse practices in AI-assisted writing result in a wide range of interaction modes. Prior works introduced a number of frameworks and taxonomies to describe how writers used AI in their writing procedures (in other words, how AI was involved or contributed). Khosrowi et al. [54] proposed the collective-centered creation (CCC) framework that described fine-grained creatorship in human-AI content co-creation according to five dimensions: relevance/(non-)redundancy and control, originality, time/effort, leadership and independence, and directness of each contributor’s contribution in collaboration. Wan et al. [120] proposed the CoCo matrix, which categorized writers’ AI usage based on writers’ and AI’s cognitive contributions. They identified entropy and information gain as two dimensions and mapped writers’ AI usage into four quadrants: high entropy and high information gain (e.g., writers plan, AI executes the plan), high entropy and low information gain (e.g., AI expands writers’ drafts), low entropy and high information gain (e.g., AI plans, writers execute), low entropy and low information gain (e.g., AI provides feedback on writers’ writings). Xu et al. [127] categorized AI involvement in cocreation processes based on originality of contribution and level of contribution, where level of contribution can be further categorized based on sense of control and amount of effort in collaboration. He et al. [42] described AI contribution based on the type of contribution, the amount of contribution, and the initiative.

Method. Based on prior works in transparency and disclosure in AI-assisted writing [26, 42, 54, 134], we synthesize three dimensions that may influence the perceptions around AI disclosure: the perspective (i.e., reader or writer of the text), the purpose of the written text (i.e., the goal of reading or writing the text), and procedural factors (i.e., how AI was used). Concretely, procedural factors include replaceability (the extent to which AI’s contribution can be replaced), effortfulness (how much effort the writer put into writing), intentionality (how intentionally the writer steered AI generation), and directness (how much AI-generated content is directly incorporated in the final writing), capturing varied AI involvement in writing. Our research questions are: how do the perspective (RQ1), purpose of the written text (RQ2), and procedural factors (RQ3) affect the perceived necessity of AI disclosure? We conduct a vignette study with 727 participants to investigate these questions. Concretely, we operationalize the combinations of perspective, purpose, and procedural factors into vignettes representing a wide range of hypothetical reading and writing situations, and ask study participants whether they perceive AI disclosure as necessary in each situation. We find that perspective and procedural factors individually and collectively shaped readers’ and writers’ perceived necessity of AI disclosure. Readers were more likely to think disclosure is necessary than writers Previous studies identified many factors that potentially influence people’s views on AI use in writing and their disclosure behaviors [26, 42, 54, 134]. Building on those studies, we synthesize a list of factors that may affect the perceived necessity of disclosing AI use, grouped into three dimensions: perspective (Section 3.1), purpose (Section 3.2), and procedural factors (Section 3.3). We describe these factors and pose hypotheses below.

3.1 Perspective We investigate the perceived necessity of disclosure from the perspectives of two stakeholders who directly engage with AI-assisted writing and AI disclosure: readers and writers:

• Readers are individuals who consume or interpret written texts produced by writers. • Writers are individuals who create or produce written texts intended to be shared with readers.

Readers and writers may have different motivations for providing or seeking AI disclosure, and disclosure may have different influences on readers and writers. For example, writers may consider disclosing AI use in writing as a moral behavior, which aligns with their values and standards of honesty and transparency and helps build trust [71, 96]. Meanwhile, writers and their works can potentially be negatively affected by disclosure, as people may perceive AI co-created writings to as less valuable or think writers who need AI assistance are less competent [14, 52, 67]. On the other hand, many readers expect AI involvement to be disclosed as they believe in the utility of disclosed information. For example, readers can assess how much they should trust the content or even decide whether the content is worth reading [28, 35, 118]. Thus, we hypothesize that the perspective has a main effect on the perceived necessity of AI disclosure (H1); compared to writers, readers deem disclosure to be more necessary (H1a).

3.2 Purpose The purpose of the written text reflects readers’ and writers’ goals of reading and writing. To select purposes to be studied in this work, we review HCI literature on AI-assisted writing and categorize the writing scenarios/tasks investigated in prior works based on the purpose of writing [1, 14, 42, 44, 47, 52, 63, 67, 95, 98, 104, 134]. We only consider the purposes of written texts intended for sharing with others (instead of diaries or notes kept for personal use), as AI disclosure is most relevant to such texts. We select three purposes representative of goals in common reading and writing scenarios, while recognizing that they are neither exhaustive nor mutually exclusive:

Discussion. 6.1 Reflection on study findings Our study result shows a substantial gap in the perceived necessity of AI disclosure between perspectives, indicating that writers are much less likely to perceive disclosure as necessary compared to readers. This gap may be explained by the self-serving bias, which suggests that people tend to attribute desirable outcomes to their own capabilities or behaviors [80, 108]. In the context of this study, self-serving bias may cause writers to overly ascribe the outcome of AI-assisted writing to themselves, thereby underestimating AI’s contribution and consequently perceiving a reduced need for disclosure. Furthermore, this reader-writer gap may be tied to the pervasiveness of AI in the writing ecology [20]. As AI becomes seamlessly integrated into writing interfaces and platforms today, writers may perceive it as a form of infrastructure that naturally exists within the social environment (or in other words, an “ambient environmental factor”) [94, 112]. This diminished visibility of AI from the writers’ view may contribute to their lower perceived necessity of disclosure.

As expected, high AI involvement in writing generally increases the perceived necessity of AI disclosure. However, effortfulness (i.e., how much effort a writer puts into writing) does not have a notable impact on the perceived necessity of disclosing AI use. This appears to contradict the effort heuristic, a well-known effect in cognitive psychology, which demonstrates that people assign value to a work based on the effort it took to produce it [58, 135]. Following the effort heuristic, one would expect that the greater effort invested by a writer should increase the perceived value of the writer’s contribution, which in turn reduces the value assigned to AI contribution and lessens the perceived necessity of disclosure. We also initially hypothesized that the purpose of the written text may affect readers’ and writers’ judgment about whether disclosure is necessary, because readers and writers appear to care more about AI use in some writing scenarios than others (e.g., AI use seems to be more acceptable in creative writing [47] than in writing journalistic content [118]). While our finding of the insignificance of purpose was initially surprising, we later found that it loosely aligns with the findings from prior work. He et al. [42] found that writing context (writing in academic, professional, or technical context) is not a significant predictor of how writers credit authorship to AI, which may correlate with their perceived necessity of disclosure. On the other hand, our supplementary qualitative analysis shows that for some participants, purposes of reading and writing do have an impact on their perceptions of AI disclosure necessity (see Appendix D.2.1). Future work can further investigate this counterintuitive result on the effect of purpose.

6.2 Considerations on the future design of AI disclosure guidance and tools The readers’ and writers’ differing perceptions of AI disclosure necessity underscore the importance of guidance and interventions that raise writers’ recognition of the need to disclose. We note that many writers are aware of the benefits of disclosure (e.g., for meeting ethical standards and building trust, as evidenced by writers’ open-ended responses reported in Appendix D.2), suggesting that they do not hold an entrenched negative attitude towards AI disclosure and that motivating them to disclose AI use is feasible.

Conclusion. AI-assisted writing has become a common part of contemporary writing practices. While AI disclosure is regulated in some cases, it is unclear what leads readers and writers to perceive disclosure as necessary. In this work, we present a vignette study that investigates how the perceived necessity of AI disclosure is influenced by perspectives, purposes, and procedural factors. We found a significant gap between perspectives: readers are more likely to consider disclosure to be necessary than writers. High AI involvement in writing (specifically, low replaceability of AI contribution, low intentionality of writers’ AI use, and highly direct adoption of AI contribution in writing) increases the perceived necessity of AI disclosure. Perceived necessity of AI disclosure is additionally influenced by the interaction between perspective and intentionality of AI use: low intentionality increases readers’ perceived necessity but decreases writers’ perceived necessity. Our study takes a step towards understanding readers’ and writers’ perceptions around AI disclosure and suggests promising approaches for scaffolding AI disclosure and behavioral interventions.

Limitations. There are several limitations of this work. Some of them come from the inherent methodological limitation of the vignette experiment. The vignettes require participants to imagine that they are readers and writers; future work can study the perceived necessity of disclosure in a more realistic setting where participants are actual readers and writers. Relatedly, some participants may not be able to relate to the vignettes that they are assigned: people who seldom write outside of school or work contexts might find “writing for entertainment” hardly relatable, distorting their perceptions of the disclosure necessity. We try to minimize this problem by choosing reading and writing purposes that most people are familiar with. For participants who are assigned writers’ perspective, their perceptions may be affected by social desirability bias [39], especially because disclosure is associated with moral considerations.