Automated drug detection and location identification for visually impaired using image processing and voice commands

Abstract

This report proposed a system which will aid visually impaired to detect and locate medicine (drug) using voice commands. It is a challenge to build a system which will assist a person how has eye sight difficulties. So we took that challenge and tried to build a system which will able to locate any medicine location around that person’s surroundings. This system includes optical character recognition (OCR) and depth segmentation of Kinect sensor in order to process extraction of characters from frames in real time video. For extracting the text name of the medicine we tested several OCR engines which includes – Google-Tesseract-OCR, and Aspose-OCR. To process images we used emguCV which is a .NET wrapper of the Intel OpenCV image-processing library. In order to operate voice commands we used Microsoft speech recognition library. Finally, the system enables instructions which will guide a visually impaired person to find his/her medicine for daily usage.