Action Recognition Based Real-time Bangla Sign Language Detection and Sentence Formation

Abstract

Sign language is a system of communication that uses visual motions and signs to communicate with persons who are deaf or mute due to a hearing or speech impairment. A real-time Bangla Sign Language (BdSL) detection system was proposed in this paper, which can generate Bangla sentences from a sequence of images or a video feed which can help those who are not familiar with sign language. Blazepose algorithm was used to identify the sign language body posture sequence. After detecting the body posture the data was gathered as a numpy file. A Long Short-Term Memory (LSTM) network was used to train the numpy files since this network can generate predictions based on sequential data. After 85 epochs of training, the model's training accuracy was 93.85%, and its validation accuracy was 87.14%, which indicates that the model's ability to recognize BdSL sentences in real-time is adequate.