Deepfake Image and Video Detection System for Sri Lankan Facial Features Using Machine Learning

Abstract

Deepfakes and other AI-generated manipulated images and videos have become an increasing cyber threat to Sri Lanka as AI-generated multimedia content becomes more accessible to consumers. Global AI-generated multimedia datasets used in global deepfake detection models do not include sufficient representation of Sri Lankan characteristics including: darker/mixed brown skin tone; South Asian facial structure; ethnic diversity (Sinhalese, Tamil, Muslim, Burgher); traditional clothing; and lighting found in a variety of local environments. Therefore, many international deep fake detection systems are either fail to accurately identify manipulated images of Sri Lankan faces, or fail when detecting low resolution video content captured on mobile devices that are commonly used in Sri Lanka. A system designed to detect deep faked images and videos of Sri Lankan faces is presented in this research. The system uses a CNN-based image forensic model in combination with frequency domain-based artifact analysis and landmark consistency checks to evaluate each image submitted by a user. Additionally, the system also analyzes video submissions for deep fakes by extracting frames from the input video and evaluating each frame individually using the trained image model. Finally, the results from each individual frame are aggregated into an overall decision regarding the authenticity of the video submission. A custom dataset was developed for the purposes of training the system’s models, which focuses on a variety of aspects of Sri Lankan skin tones, facial structures, cultural elements, and environmental factors.