Wednesday 22 January 2025
The quest for a foolproof method of detecting deepfakes has been an ongoing challenge for researchers and developers in recent years. With the rise of AI-generated video and audio, it’s become increasingly difficult to distinguish between real and fake content. A new study proposes a novel approach that combines facial landmark features with heart rate data to identify deepfake videos.
The authors of this study present a unified framework that can detect all three types of deepfakes: face-swap, lip-sync, and puppet-master. Unlike previous methods that focused on specific types of deepfakes or relied on manual feature engineering, the proposed approach uses a machine learning-based method to extract features from facial landmarks and heart rate data.
The researchers used a dataset called WLDR (World Leaders Dataset), which contains real and fake videos of ten US politicians and comedy impersonators. The dataset is unique in that it includes all three types of deepfakes, making it an ideal testing ground for the proposed method.
The authors analyzed various combinations of facial landmark features, including 2D and 3D landmarks, eye landmarks, head pose, shape, and action unit features. They found that combining these features with heart rate data resulted in a significant improvement in detection accuracy.
The proposed framework uses an extreme gradient boosting algorithm (XGBoost) to classify the videos as real or fake. The results show that the XGBoost-based model outperforms existing methods, including those based on deep learning techniques such as LSTM-FCN.
One of the notable aspects of this study is its focus on interpretability. Unlike many machine learning models, which can be opaque and difficult to understand, the proposed approach provides insights into the features that contribute most to the detection accuracy. This makes it easier for developers to identify areas where the model may need improvement or fine-tuning.
The authors also highlight the limitations of their study, including the fact that the dataset is relatively small compared to other available datasets. However, they argue that this is a deliberate choice, as the WLDR dataset provides a more challenging testing ground for deepfake detection.
In summary, this study presents a novel approach to detecting deepfakes by combining facial landmark features with heart rate data. The proposed framework uses an XGBoost algorithm and outperforms existing methods in terms of accuracy. Its focus on interpretability makes it an attractive option for developers seeking to create more robust and transparent deepfake detection systems.
Cite this article: “Deepfake Detection with Facial Landmarks and Heart Rate Data”, The Science Archive, 2025.
Deepfakes, Facial Landmarks, Heart Rate Data, Machine Learning, Xgboost, Video Analysis, Deep Learning, Lstm-Fcn, Interpretability, Computer Vision
