Large Language Models Can Infer Psychological Dispositions of Social Media Users

we test whether GPT-3.5 and GPT-4 can derive the Big Five personality traits from users’ Facebook status updates in a zero-shot learning scenario. Our results show an average correlation of r = .29 (range = [.22, .33]) between LLM-inferred and self-reported trait scores – a level of accuracy that is similar to that of supervised machine learning models specifically trained to infer personality. Our findings also highlight heterogeneity in the accuracy of personality inferences across different age groups and gender categories: predictions were found to be more accurate for women and younger individuals on several traits, suggesting a potential bias stemming from the underlying training data or differences in online self-expression. The ability of LLMs to infer psychological dispositions from user generated ssssstext has the potential to democratize access to cheap and scalable psychometric assessments for both researchers and practitioners

given a large enough dataset including both self-reported personality traits and people’s digital footprints - such as Facebook Likes, music playlists, or browsing histories – machine learning models are able to statistically relate both inputs in a way that allows them to predict personality traits after observing a person’s digital footprints [14, 15]. This is also true for various forms of text data, including social media posts [16, 17], personal blogs [18], or short text responses collected in the context of job applications [19].