Personalization of Large Language Models: A Survey

Personalization of Large Language Models (LLMs) has recently become increasingly important with a wide range of applications. Despite the importance and recent progress, most existing works on personalized LLMs have focused either entirely on (a) personalized text generation or (b) leveraging LLMs for personalization-related downstream applications, such as recommendation systems. In this work, we bridge the gap between these two separate main directions for the first time by introducing a taxonomy for personalized LLM usage and summarizing the key differences and challenges. We provide a formalization of the foundations of personalized LLMs that consolidates and expands notions of personalization of LLMs, defining and discussing novel facets of personalization, usage, and desiderata of personalized LLMs. We then unify the literature across these diverse fields and usage scenarios by proposing systematic taxonomies for the granularity of personalization, personalization techniques, datasets, evaluation methods, and applications of personalized LLMs. Finally, we highlight challenges and important open problems that remain to be addressed. By unifying and surveying recent research using the proposed taxonomies, we aim to provide a clear guide to the existing literature and different facets of personalization in LLMs, empowering both researchers and practitioners.

1. A unifying view and taxonomy for the usage of personalized LLMs (Section 2). We provide a unifying view and taxonomy of the usage of personalized LLMs based on whether they focus on evaluating the generated text directly, or whether the text is used indirectly for another downstream application. This serves as a fundamental basis for understanding and unifying the two separate areas focused on the personalization of LLMs. Further, we analyze the limitations of each, including the features, evaluation, and datasets, among other factors.

2. A formalization of personalized LLMs (Section 3). We provide a formalization of personalized LLMs by establishing foundational concepts that consolidate existing notions of personalization, defining and discussing novel facets of personalization, and outlining desiderata for their application across diverse usage scenarios.

3. An analysis and taxonomy of the personalization granularity of LLMs (Section 4). We propose three different levels of personalization granularity for LLMs, including (i) user-level personalization, (ii) persona-level personalization, and (iii) global preference personalization. We formalize these levels, and then discuss and characterize the trade-offs between the different granularities of LLM personalization. Notably, user-level personalization is the finest granularity; however, it requires a sufficient amount of user-level data.

While direct ➊personalized text generation and ➋downstream task personalization might appear distinct, they share many underlying components and mechanisms. Both settings often involve retrieving and utilizing user-specific data, constructing personalized prompts or embeddings, and leveraging these to enhance model outputs. The key distinction lies in the dataset they use and the evaluation methods: direct text generation focuses on aligning the generated text with user-written ground-truth, while downstream task personalization evaluates the improvement in specific tasks. Despite these differences, the two approaches can complement each other. For instance, advancements in direct personalized text generation can provide richer, more nuanced intermediate text or embeddings that may enhance downstream tasks. Conversely, improvements in downstream task personalization models can inform better methods for retrieving and leveraging user-specific data in direct generation tasks.

3.2 Formulation of Personalization

Definition 5 (Personalization). Personalization refers to the process of tailoring a system’s output to meet the individual preferences, needs, and characteristics of an individual user or a group of users. In the context of LLMs, personalization involves adjusting the model’s responses based on user-specific data, historical interactions, and contextual information to enhance user satisfaction and relevance of the entire system’s generated content.

Definition 6 (User Preferences). User Preferences refer to the specific likes, dislikes, interests, and priorities of an individual user or a group of users. These preferences guide the personalization process by 7 informing the system about the desired characteristics and features of the output. In the context of LLMs, user preferences can be derived from explicit feedback, historical interactions, and contextual signals to tailor responses and improve the relevance and satisfaction of the generated content.

Definition 7 (Personalized Large Language Model). A Personalized Large Language Model (Personalized LLM) Mp is an LLM that has been adapted to align with the individual preferences, needs, and characteristics of a specific user or group of users. This adaptation involves utilizing user-specific data, historical interactions, and contextual information to modify the model’s responses, making them more relevant and satisfying for the user. Personalized LLMs aim to enhance the user experience by providing tailored content that meets the unique expectations and requirements of the user.

Definition 8 (User Documents). User Documents Du refer to the collection of texts and writings generated by a user u. This includes reviews, comments, social media posts, and other forms of written content that provide insights into the user’s preferences, opinions, and sentiments.

Definition 9 (User Attributes). User Attributes Au = {a1, a2, . . . , ak} are the static characteristics and demographic information associated with a user u ∈ U. These attributes include age, gender, location, occupation, and other metadata that remain relatively constant over time.

Definition 10 (User Interactions). User Interactions Iu = {i1, i2, . . . , im} capture the dynamic behaviors and activities of a user u ∈ U within a system. This includes actions such as clicks, views, purchases, and other engagement data that reflect the user’s preferences and interests.

Pair-wise human preferences refer to explicit user feedback indicating their preferred responses from a set of candidate outputs. This data format typically involves human annotations selecting the most desired option, making it essential for training models to align closely with individual user needs and preferences. Unlike static attributes or interaction history, pair-wise preferences offer highly specific and direct feedback, serving as explicit instructions on how users expect the model to behave or respond in given scenarios. For example, users might specify whether they want a response to be easily understood by a layperson or tailored for an expert. In this way, users can explicitly state what they want, reducing ambiguity and implicitly, which can be useful leading to higher user satisfaction and more effective personalization. However, designing an appropriate alignment strategy remains a significant challenge for personalization applications. Most current works focus on aligning models with general, aggregate human preferences, rather than diverse, individual perspectives (Jang et al., 2023). Developing methods to capture and use these individual direct preferences effectively is essential for advancing personalized systems.

3.6.1 Taxonomy of Personalization Granularity of LLMs

We propose three different levels of personalization granularity for LLMs, each addressing different scopes of personalization. These levels help in understanding the depth and breadth of personalization that can be achieved with LLMs. The three levels are:

§4.1 User-level Personalization: Focuses on the unique preferences and behaviors of a single user. Personalization at this level utilizes detailed information about the user, including their historical interactions, preferences, and behaviors, often identified through a user ID.

§4.2 Persona-level Personalization: Targets groups of users who share similar characteristics or preferences, known as personas. Personalization here is based on the collective attributes of these groups, such as expertise, informativeness, and style preferences.

§4.3 Global Preference Personalization: Encompasses general preferences and norms that are widely accepted by the general public, such as cultural standards and social norms.

The granularity of personalization in LLMs involves trade-offs between precision, scalability, and richness of personalized experiences. User-level personalization offers high precision and engagement but faces challenges with data sparsity and scalability. Persona-level personalization is efficient and representative but less granular and requires domain knowledge for defining personas. Global preference personalization provides broad applicability and simplicity but lacks specificity and can introduce noise from aggregated data

• Personalization via RAG (Sec 5.1): This category of methods incorporates user information as an external knowledge base, encoded through vectors. When new inference data arrives, the relevant information is retrieved using embedding space similarity search for downstream personalization tasks.

• Personalization via Prompting (Sec 5.2): This category of methods incorporates user information as the context within the prompts for LLMs. By providing this contextual information, LLMs can either directly perform downstream personalization tasks through text generation or act as intermediate modules to extract more relevant information, thereby enhancing the system’s performance. 18

• Personalization via Representation Learning (Sec 5.3): This category of methods encodes user information into the embedding spaces of neural network modules. The user information can be represented through the entire parameters of the LLM, a subset of the model’s parameters, a small number of additional parameters, or an explicit embedding vector specific to each user.

• Personalization via RLHF (Sec 5.4): This category of methods uses user information as the reward signal to align LLMs with personalized preferences through reinforcement learning.