Enhancing indoor navigation for the visually impaired: A neurosymbolic AI approach with visual question answering for object recognition and localization

Abstract

Comprehension of the indoor setting is remarkably important for the visually impaired people to maneuver and interact with the surroundings in an efficient way. In this era of high-tech and innovative technologies, the existing assistive technologies typically lack of malleability in the indoor environment where objects are true, as a result, it impedes the capability of relevant scene comprehension. Additionally, this study unveils a novel neuro-symbolic approach for recognizing items and discernment of the indoor setting through the visual-question-answering (VQA). Moreover, the custom model assesses visual scenes, recognizes the objects as well as construes spatial relationships for providing accurate responses. Here, the system uses the image-driven scene encoding using YOLOv3 incorporating answer set programming (ASP). Furthermore, the system actually augments scene understanding and versatility in unique enclosed settings by integrating perceptual thought-driven technologies. However, the custom model improves the text understanding depending on the BERT-based question encrypting. In addition to that, the model is assessed on a unique dataset which illustrates its effectiveness in indoor surroundings. Therefore, our research appraises the new model’s architecture not only in identifying objects but also in answering the contextual questions presented in the scenario of the indoor which basically depicts potency for amending scenario-based assistance for unsighted individuals.