Friday 28 February 2025
Deepfake analysis, a crucial tool for detecting manipulated images and videos, has been plagued by limitations in its ability to accurately identify and classify deepfakes. Researchers have traditionally relied on discriminative models, which are effective but lack the versatility to adapt to new scenarios. Now, a team of scientists has proposed an innovative approach that leverages instruction-tuning on multi-modal large language models (MLLMs) to create an interactive deepfake analysis system.
The new system, dubbed DFA-GPT, is designed to perform four key tasks: detecting whether an image or video is forged, classifying the specific type of forgery technique used, describing the artifacts present in the image that indicate manipulation, and engaging in free conversation with users about the forgery. To achieve this, the researchers developed a novel data construction process called DFA-Instruct, which generates instruction-following data.
DFA-Bench, a comprehensive benchmark for evaluating MLLMs’ deepfake analysis capabilities, was also created to test the performance of DFA-GPT. The results show that current advanced MLLMs lack adequate understanding of face forgery, but DFA-GPT outperforms them in all four tasks. The system’s ability to adapt to new scenarios and provide detailed explanations of its findings makes it a powerful tool for identifying and mitigating deepfakes.
The development of DFA-GPT has significant implications for various fields, including information forensics, security, and law enforcement. By providing a more accurate and versatile method for detecting deepfakes, this technology can help prevent the misuse of manipulated media and protect individuals from identity theft and other forms of fraud.
Cite this article: “Interactive Deepfake Analysis System Developed Using Large Language Models”, The Science Archive, 2025.
Deepfake, Analysis, Detection, Classification, Manipulation, Images, Videos, Large Language Models, Mllms, Instruction-Tuning
