Friday 07 March 2025
Researchers have been exploring the potential of large language models (LLMs) in inferring personality traits, but a new study has taken a nuanced approach by examining the effectiveness of these models in predicting Big Five personality traits from user conversations.
The study, which analyzed the performance of two LLMs, GPT-4o and GPT-4o mini, found that while both models were able to predict some personality traits with moderate accuracy, their ability to capture subtle differences between individuals was limited. The results suggest that integrating structured psychometric frameworks, such as the Big Five Inventory (BFI), into LLM-based assessments can significantly improve predictive accuracy.
The researchers used a dataset of user conversations and evaluated the performance of the two LLMs in predicting five personality traits: extraversion, agreeableness, conscientiousness, neuroticism, and openness. They found that both models were able to accurately predict some traits, such as extraversion and openness, but struggled with others, like neuroticism.
The study also explored how the performance of the LLMs varied depending on the presence or absence of depressive symptoms in the user conversations. The results showed that GPT-4o mini was more accurate in predicting personality traits when users were experiencing depressive symptoms, while GPT-4o performed better when users were not experiencing symptoms.
The findings have implications for the potential use of LLMs in mental health assessments and interventions. While the models may not be able to accurately predict all personality traits, they could potentially be used as a tool to identify individuals who are more likely to experience certain psychological states or behaviors.
One potential limitation of the study is that it relied on a dataset of user conversations, which may not be representative of all users. Additionally, the researchers used a fixed set of personality traits and did not explore the possibility of using other frameworks or theories to inform their analysis.
Despite these limitations, the study provides valuable insights into the potential uses and limitations of LLMs in predicting personality traits. As the technology continues to evolve, it will be important to continue evaluating its effectiveness and exploring new applications for mental health assessments and interventions.
The results of this study suggest that while LLMs may not be perfect predictors of personality traits, they could potentially be used as a tool to identify individuals who are more likely to experience certain psychological states or behaviors. As the technology continues to evolve, it will be important to continue evaluating its effectiveness and exploring new applications for mental health assessments and interventions.
Cite this article: “Predicting Personality Traits with Large Language Models”, The Science Archive, 2025.
Language Models, Personality Traits, Big Five Inventory, Bfi, Gpt-4O, Gpt-4O Mini, Mental Health Assessments, Neuroticism, Depressive Symptoms, User Conversations.
