Real-time facial expression recognition with Bengali audio feedback: bridging communication gaps

Abstract

This study investigates the efficacy of deep learning models in facial expression recognition while incorporating Bengali audio feedback. We utilize a meticulously curated dataset comprising diverse facial images depicting individuals expressing various emotions, annotated with corresponding Bengali audio descriptions. Each image is labeled with the emotion it represents, and the dataset includes metadata such as age, gender, and cultural context. We explore the performance of convolutional neural networks (CNNs), recurrent neural networks (RNNs), and hybrid models in recognizing facial expressions from images and associating them with Bengali audio feedback. Additionally, we assess the impact of data augmentation techniques on model performance. Our experiments reveal that hybrid CNN-RNN models achieve the highest accuracy in recognizing facial expressions and generating appropriate Bengali audio feedback. Furthermore, we analyze the robustness of the models across diverse demographic groups within the dataset. This study contributes to the advancement of multimodal deep learning techniques for enhancing communication experiences, particularly in contexts where Bengali audio feedback is essential.