Thursday 27 March 2025
A team of researchers has made a significant breakthrough in extracting social connections from historical interviews, using large language models (LLMs) without any prior training on the specific task.
The study focused on Finnish Karelian refugee interviews, which were conducted post-WWII and provide valuable insight into the lives of families who relocated from Finnish Eastern Karelia. The researchers aimed to identify hobbies and social organizations mentioned in the interviews, separately for each family member.
Traditionally, tasks like this require extensive human annotation and training data. However, the team used LLMs to extract information without any prior knowledge about the specific task or domain. This approach allows for zero-shot learning, where the model can learn from general language patterns and apply them to a new task.
The researchers evaluated several alternative approaches, including fine-tuning a Finnish BERT model using data generated by GPT-4, one of the most advanced LLMs available. They found that the best open-source model, Llama-3-70B-Instruct, performed almost as well as the commercial GPT-4 model.
The team’s approach demonstrated several advantages over traditional methods. For instance, they were able to extract information from a large dataset of 89,339 interviews with remarkable accuracy, achieving an F-score of 87.7%. This is comparable to human performance and significantly outperforms previous results in the field.
Moreover, the study showed that fine-tuning a lightweight encoder model using NER-like training data created by LLMs can yield competitive results. This approach has significant implications for tasks where large amounts of data need to be processed.
The research has far-reaching applications beyond language processing. By analyzing social connections and relationships in historical interviews, researchers can gain valuable insights into the lives of individuals and families affected by war, migration, and other significant events.
In a broader sense, this study highlights the potential for LLMs to revolutionize various fields, including history, sociology, and psychology. As these models continue to advance, they may enable new forms of analysis and discovery that were previously impossible or impractical.
The findings have also sparked interest in exploring further applications for LLMs in information extraction, text classification, and sentiment analysis. With their ability to learn from general language patterns, these models could potentially be applied to a wide range of tasks, opening up new avenues for research and innovation.
Cite this article: “Unlocking Social Connections from Historical Interviews with Large Language Models”, The Science Archive, 2025.
Large Language Models, Historical Interviews, Social Connections, Finnish Karelian Refugee, Zero-Shot Learning, Fine-Tuning, Bert Model, Gpt-4, Llama-3-70B-Instruct, Natural Language Processing
