Wednesday 19 March 2025
The podcasting landscape has seen a surge in popularity over the past decade, with millions of shows available at our fingertips. But despite this proliferation of content, fact-checking and verifying claims made on these audio platforms remains a significant challenge. A new tool aims to change that by providing an open-source platform for annotating and fact-checking podcasts.
The tool, developed by researchers from the University of Stavanger in Norway, allows users to listen to podcasts while simultaneously correcting transcription errors, identifying check-worthy claims, and resolving contextual ambiguities. The system is designed to streamline the process of fact-checking, making it easier for individuals and organizations to verify the accuracy of information presented on podcasts.
One of the key features of the tool is its ability to integrate real-time audio playback with interactive annotation. This allows users to listen to a podcast while correcting transcription errors and identifying check-worthy claims. The system also includes a dedicated interface for fact-checking, where users can generate search queries, evaluate retrieved documents, and assign verdicts to claims.
The researchers behind the tool have created a large-scale dataset of annotated podcasts, which they are releasing alongside the platform. This dataset includes over 531 episodes from 38 different podcasts, with annotations for claim detection, fine-grained claim categorization, and claim verification. The dataset is specifically designed for end-to-end fact-checking, making it an invaluable resource for researchers and developers working on this problem.
The tool’s creators have also developed a novel approach to fact-checking, which involves identifying the motivations behind why someone might want to fact-check a particular claim. They identify five primary reasons: to learn more about a topic, due to public interest, because of potential harm caused by misinformation, or because the claim is surprising or made by a prominent individual.
The researchers have also tested their tool against two popular language models, XLM-Roberta-Large and GPT-4, to evaluate its performance in claim detection and stance detection tasks. The results show that both models perform well, but with some differences: XLM-Roberta-Large excels at identifying true claims, while GPT-4 is better at detecting false ones.
The release of this tool and dataset marks an important step forward in the development of fact-checking technology for podcasts. By providing a platform for annotating and verifying claims made on these audio platforms, the researchers aim to make it easier for individuals and organizations to verify the accuracy of information presented on podcasts.
Cite this article: “Fact-Checking Podcasts: A New Open-Source Tool Aims to Verify Accuracy”, The Science Archive, 2025.
Podcasting, Fact-Checking, Annotation, Verification, Transcription Errors, Claim Detection, Language Models, Dataset, Misinformation, Accuracy
Reference: Vinay Setty, Adam James Becker, “Annotation Tool and Dataset for Fact-Checking Podcasts” (2025).
