Speech emotion recognition for intelligent voice-based fraud detection

Abstract

This study explores fraud detection using Speech Emotion Recognition (SER). The proposed system aims to enhance the efficiency of manual call centre procedures by automatically analysing callers’ emotional cues to determine their authenticity. Leveraging a neural network module, the system predicts whether a caller is genuine or fraudulent and presents results through intuitive visualisations, improving user experience. Although the speech emotion recognition model is initially trained on British-accented English, the approach can be adapted to other languages and accents, making it versatile across different domains. By focusing on voice features rather than converting speech to text, this technique overcomes language barriers and avoids limitations inherent in traditional Natural Language Processing (NLP) based models.