Thursday 27 March 2025
Researchers have made significant strides in developing large language models (LLMs) that can generate text on a wide range of topics. However, these models often struggle to accurately represent and serve people from diverse geo-cultural backgrounds. A recent study aimed to address this issue by investigating the effectiveness of two strategies for improving cultural awareness in LLMs.
The first strategy, known as retrieval-augmented generation (RAG), involves combining the model’s own text generation capabilities with external knowledge retrieved from a bespoke knowledge base. This approach is designed to help the model learn about different cultures and incorporate that knowledge into its generated text.
The second strategy, search grounding, takes a more straightforward approach. It uses web searches to retrieve relevant information on specific cultural topics and then generates text based on those results.
Researchers tested both strategies using three large language models: Gemini, GPT, and OLMo. They created prompts related to various national cultures, including China, Ethiopia, Greece, Indonesia, Iran, Mexico, South Korea, Spain, the United Kingdom, and the United States. The prompts were designed to elicit culturally informed responses that would showcase each model’s ability to understand and represent different cultures.
The results showed that search grounding significantly improved the performance of the Gemini and GPT models on multiple-choice benchmarks that test propositional knowledge about national cultures. However, this strategy also increased the risk of stereotypical judgments by the language models.
In contrast, selective KB-grounding, which involves retrieving relevant information from a knowledge base and incorporating it into the generated text, performed better in terms of cultural awareness. This approach was particularly effective when combined with open-ended generation tasks, where the model was asked to generate creative responses that showcased its understanding of different cultures.
The study’s findings have important implications for the development of LLMs that can accurately represent and serve people from diverse geo-cultural backgrounds. While search grounding may be effective in certain contexts, it is essential to consider the potential risks of stereotyping and bias when using this approach.
In contrast, selective KB-grounding offers a more nuanced and culturally sensitive way of incorporating external knowledge into an LLM’s text generation capabilities. This approach has the potential to improve the overall cultural awareness and diversity of LLMs, making them more effective tools for communication and collaboration in a globalized world.
The study’s results also highlight the importance of considering the cultural context and nuances when evaluating the performance of LLMs.
Cite this article: “Improving Cultural Awareness in Large Language Models”, The Science Archive, 2025.
Large Language Models, Cultural Awareness, Retrieval-Augmented Generation, Search Grounding, Kb-Grounding, Propositional Knowledge, National Cultures, Stereotypical Judgments, Open-Ended Generation, Cultural Context
