Intelligent sound localization and recognition with tactile feedback system to assist hearing-impaired individuals

Abstract

A significant portion of the global population suffers to some extent from hearing loss and requires assistive devices. The everyday lives of hearing-impaired individuals are hampered by their inability to recognize important environmental sounds in their surroundings. This thesis proposes an innovative system that can both recognize and localize critical environmental sounds while providing tactile feedback to enhance user awareness in real time. Machine learning models such as Convolutional Neural Networks (CNNs) are utilized to classify specific sounds such as car horn, dog bark, scream, and calling bell. Furthermore, localization algorithms are integrated into a wearable device to determine the direction of these sounds. Once these sounds are detected and localized, the user is notified through vibration feedback from their smartphone or smartwatch, while the sound type, confidence level, and direction are simultaneously displayed. The proposed system is designed to be a low-cost, compact, and accessible solution for recognizing important environmental sounds and represents a meaningful advancement in the field of assistive devices by significantly enhancing the safety and quality of life of individuals with hearing impairments.