PersLLM: A Personified Training Approach for Large Language Models

Inspired by these concerns, we propose PersLLM, a comprehensive approach to LLM personification including personified data construction and model tuning. To achieve a sufficient data usage, we use a widely adopted conversational data format annotated by advanced LLMs in retrieval-augmented generation (RAG) (Chen et al., 2024). The processes for retrieving raw data, formulating inputs, and generating responses are outlined, with strategies such as material classification, anti-induction, and Chain-of-Thought (CoT) (Wei et al., 2022b) prompting employed. To avoid the rigid behavior patterns of LLMs, we integrate personified data with general instruction tuning data to fine-tune the model and then apply Direct Preference Optimization (DPO) (Rafailov et al., 2024) to highlight personality and temporal differences.

To evaluate PersLLM, we examine it from both theoretical and practical perspectives. Theoretically, psychologists define personality as a dynamic organization of psychophysical systems that generates consistent patterns of behavior, thought, and feeling (Carver, 2011). Based on this, we establish three key criteria for LLM personification: distinction, consistency, and dynamic development, which we assess through quantitative experiments and psychological scale measurements. Practically, we evaluate humanized knowledge and opinion interactions by examining the precision of the generation and the performance of the human-agent interaction. Experimental results show that PersLLM achieves more precise alignment with specific personalities and superior interaction performance in real-world scenarios.