Kotha: An App for Communicating with Deaf & Dumb Using CNN

Abstract

There are people around us who can’t talk or listen, and sometimes both. Few of them are born, and others become like this due to injuries or accidents. This inspired us to create a project that can help hearing-impaired and mute people communicate with others. We used the Convolutional Neural Network(CNN) to train our model and OpenCV to create our gestures. We created 36 gestures with 50x50 resolution, where there are 1200 images for each gesture. These pictures were then flipped using flip_image.py. This script flips every image along the vertical axis. The model was trained multiple times for an accurate result. For the app, we used TensorFlow Lite to convert our model to be usable on mobile devices. TensorFlow Lite allows pre-trained models to be used in Android apps.