An efficient approach for deepfake detection employing micro-expressions with a hierarchical transformer network

Abstract

The widespread use of deepfake technology makes it difficult to tell the difference between modified and real digital content, as deepfakes increasingly use digitally altered faces or bodies to mimic others for malicious purposes or to spread false in- formation. With artificial intelligence technology improving, deepfakes are becoming more realistic and challenging to detect using conventional techniques that focus on surface-level visual cues like pixel irregularities and visual artifacts, which may soon be insufficient due to the rapid advancement of deepfake generation techniques. This study adapts the Hierarchical Transformer Network (HTNet), originally developed for recognizing micro-expressions, to detect deepfake videos. The HTNet, known for its ability to identify subtle involuntary facial movements, is modified to discern the synthetic traces typically undetectable by traditional methods. The model is particularly attentive to critical facial areas such as the eyes and lips, which often display signs of manipulation. Optical flow is utilized to track subtle motion be- tween frames in both real and deepfake content. The performance of the retrained HTNet model is rigorously evaluated using standard micro-expression benchmarks, including SAMM, SMIC, CASME II, and CASME III, and its effectiveness in deep- fake detection is thoroughly assessed using the FaceForensics++ dataset. Tests on deepfake and micro-expression datasets demonstrate the model’s capability in iden- tifying fake videos, showcasing HTNet as an effective advanced model for tackling the growing problem of deepfakes.