A Benchmark for Detection and Recognition of Bangladeshi Traffic Signs in Real-world Images
Date
2022-05-30
Journal Title
Journal ISSN
Volume Title
Publisher
Department of Computer Science and Engineering(CSE), Islamic University of Technology(IUT), Board Bazar, Gazipur, Bangladesh
Abstract
Traffic sign detection is an indispensable part of autonomous driving and transportation
safety systems. However, the accurate detection and recognition of
traffic signs remain challenging, especially under extreme conditions, such as
various weather and geo-social features. Though a lot of work has been done in
the domain of Traffic Sign Detection and Recognition (TSDR) systems, only a
few of them focus on a dataset that comprises the real-world challenges. Moreover,
in the context of Bangladesh, there is no well-defined public dataset,
let alone one that focuses on real-life challenges. The geo-social features of
Bangladesh add some unique challenges that are not seen in most parts of the
world. This proposal aims to address the lack of quality Bangla traffic sign
detection dataset. To accomplish this task, traffic sign images will be extracted
from videos collected from Bangladeshi roads. A performance benchmark will
be presented by applying state-of-the-art methods to the said dataset. Using
the best-performing method, an autonomous driver notification system will be
developed to alert the drivers on the go.
Description
Supervised by
Dr. Md. Hasanul Kabir,
Professor,
Co-supervisor
Mr. Sabbir Ahmed
Lecturer
Department of Computer Science and Engineering(CSE),
Islamic University of Technology (IUT)
Board Bazar, Gazipur-1704, Bangladesh.
This thesis is submitted in partial fulfillment of the requirements for the degree of Bachelor of Science in Computer Science and Engineering, 2022.
Keywords
Object detection, CNN, BdTSDB, YOLO, Fast R-CNN, Faster R-CNN, EfficientDet, ResNet
Citation
[1] A. Gautam and S. Singh, “Deep learning based object detection combined with internet of things for remote surveillance,” Wireless Personal Communications, vol. 118, no. 4, p. 2121–2140, 2021. [2] B. Wu, F. Iandola, P. H. Jin, and K. Keutzer, “Squeezedet: Unified, small, low power fully convolutional neural networks for real-time object detection for autonomous driving,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, July 2017. [3] R. Gavrilescu, C. Zet, C. Fosalau, M. Skoczylas, and D. Cotovanu, “Faster r-cnn:an approach to real-time object detection,” 2018 International Conference and Exposition on Electrical And Power Engineering (EPE), 2018. [4] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. [5] M. Tan, R. Pang, and Q. V. Le, “Efficientdet: Scalable and efficient object detection,” 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020. [6] R. Timofte, K. Zimmermann, and L. Van Gool, “Multi-view traffic sign detection, recognition, and 3d localisation,” Machine vision and applications, vol. 25, no. 3, pp. 633–647, 2014. [7] A. Mogelmose, M. M. Trivedi, and T. B. Moeslund, “Vision-based traffic sign detection and analysis for intelligent driver assistance systems: Perspectives and survey,” IEEE Transactions on Intelligent Transportation Systems, vol. 13, no. 4, pp. 1484–1497, 2012. [8] Z. Zou, Z. Shi, Y. Guo, and J. Ye, “Object detection in 20 years: A survey,” 2019. [Online]. Available: https://arxiv.org/abs/1905.05055 [9] J. Li, D. Zhang, J. Zhang, J. Zhang, T. Li, Y. Xia, Q. Yan, and L. Xun, “Facial expression recognition with faster r-cnn,” Procedia Computer Science, vol. 37 107, pp. 135–140, 2017, advances in Information and Communication Technology: Proceedings of 7th International Congress of Information and Communication Technology (ICICT2017). [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1877050917303447 [10] R. Rastgoo, K. Kiani, and S. Escalera, “Sign language recognition: A deep survey,” Expert Systems with Applications, vol. 164, p. 113794, 2021. [Online]. Available: https://www.sciencedirect.com/science/article/ pii/S095741742030614X [11] Z. Xu, W. Yang, A. Meng, N. Lu, H. Huang, C. Ying, and L. Huang, “Towards end-to-end license plate detection and recognition: A large dataset and baseline,” in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 255–271. [12] A. Arcos-Garcia, J. A. Alvarez-Garcia, and L. M. Soria-Morillo, “Evaluation of deep neural networks for traffic sign detection systems,” Neurocomputing, vol. 316, pp. 332–344, 2018. [13] E. Khatab, A. Onsy, M. Varley, and A. Abouelfarag, “Vulnerable objects detection for autonomous driving: A review,” Integration, vol. 78, pp. 36– 48, 2021. [14] A. Voulodimos, N. Doulamis, A. Doulamis, and E. Protopapadakis, “Deep learning for computer vision: A brief review,” Computational intelligence and neuroscience, vol. 2018, 2018. [15] K. Xiao, L. Engstrom, A. Ilyas, and A. Madry, “Noise or signal: The role of image backgrounds in object recognition,” arXiv preprint arXiv:2006.09994, 2020. [16] S. B. Wali, M. A. Abdullah, M. A. Hannan, A. Hussain, S. A. Samad, P. J. Ker, and M. B. Mansor, “Vision-based traffic sign detection and recognition systems: Current trends and challenges,” Sensors, vol. 19, no. 9, p. 2093, 2019. 38 [17] T. R. . November and T. Report, “Bangladesh 106th among 183 countries for having most road accidents: Report,” Tech. Rep., Nov 2021. [Online]. Available: https://www.tbsnews.net/bangladesh/bangladesh106th-among-183-countries-having-most-road-accidents-report-335299 [18] K. Maniruzzaman and R. Mitra, “Road accidents in bangladesh,” IATSS research, vol. 29, no. 2, p. 71, 2005. [19] “Road traffic injuries,” https://www.who.int/news-room/fact-sheets/ detail/road-traffic-injuries, 2021. [20] S. Houben, J. Stallkamp, J. Salmen, M. Schlipsing, and C. Igel, “Detection of traffic signs in real-world images: The german traffic sign detection benchmark,” in The 2013 international joint conference on neural networks (IJCNN). Ieee, 2013, pp. 1–8. [21] J. Stallkamp, M. Schlipsing, J. Salmen, and C. Igel, “The german traffic sign recognition benchmark: a multi-class classification competition,” in The 2011 international joint conference on neural networks. IEEE, 2011, pp. 1453–1460. [22] Z. Zhu, D. Liang, S. Zhang, X. Huang, B. Li, and S. Hu, “Traffic-sign detection and classification in the wild,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2110–2118. [23] D. M. Ram´ık, C. Sabourin, R. Moreno, and K. Madani, “A machine learning based intelligent vision system for autonomous object detection and recognition,” Applied intelligence, vol. 40, no. 2, pp. 358–375, 2014. [24] A. Vennelakanti, S. Shreya, R. Rajendran, D. Sarkar, D. Muddegowda, and P. Hanagal, “Traffic sign detection and recognition using a cnn ensemble,” in 2019 IEEE international conference on consumer electronics (ICCE). IEEE, 2019, pp. 1–4. [25] A. Shustanov and P. Yakimov, “Cnn design for real-time traffic sign recognition,” Procedia engineering, vol. 201, pp. 718–725, 2017. 39 [26] R. Girshick, “Fast r-cnn,” in Proceedings of the IEEE international conference on computer vision, 2015, pp. 1440–1448. [27] L. Wu, H. Li, J. He, and X. Chen, “Traffic sign detection method based on faster r-cnn,” in Journal of Physics: Conference Series, vol. 1176, no. 3. IOP Publishing, 2019, p. 032045. [28] K. He, G. Gkioxari, P. Dollar, and R. Girshick, “Mask r-cnn,” in ´ Proceedings of the IEEE international conference on computer vision, 2017, pp. 2961–2969. [29] T.-Y. Lin, P. Dollar, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Fea- ´ ture pyramid networks for object detection,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 2117–2125. [30] X. Changzhen, W. Cong, M. Weixin, and S. Yanmei, “A traffic sign detection algorithm based on deep convolutional neural network,” in 2016 IEEE International Conference on Signal and Image Processing (ICSIP). IEEE, 2016, pp. 676–679. [31] M. Tan and Q. Le, “Efficientnet: Rethinking model scaling for convolutional neural networks,” in International Conference on Machine Learning. PMLR, 2019, pp. 6105–6114. [32] S. P. Rajendran, L. Shine, R. Pradeep, and S. Vijayaraghavan, “Real-time traffic sign recognition using yolov3 based detector,” in 2019 10th International Conference on Computing, Communication and Networking Technologies (ICCCNT). IEEE, 2019, pp. 1–7. [33] A. Ellahyani, M. Ansari, I. Jaafari, and S. Charfi, “Traffic sign detection and recognition using features combination and random forests,” International Journal of Advanced Computer Science and Applications, vol. 7, no. 1, pp. 686–693, 2016. [34] C. G. Serna and Y. Ruichek, “Classification of traffic signs: The european dataset,” IEEE Access, vol. 6, pp. 78 136–78 148, 2018. 40 [35] X. Bangquan and W. X. Xiong, “Real-time embedded traffic sign recognition using efficient convolutional neural network,” IEEE Access, vol. 7, pp. 53 330–53 346, 2019. [36] Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel, “Backpropagation applied to handwritten zip code recognition,” Neural computation, vol. 1, no. 4, pp. 541–551, 1989. [37] A. Jose, H. Thodupunoori, and B. B. Nair, “A novel traffic sign recognition system combining viola–jones framework and deep learning,” in Soft Computing and Signal Processing. Springer, 2019, pp. 507–517. [38] K. Kaplan, C. Kurtul, and H. L. Akin, “Real-time traffic sign detection and classification method for intelligent vehicles,” in 2012 IEEE International Conference on Vehicular Electronics and Safety (ICVES 2012). IEEE, 2012, pp. 448–453. [39] J. Zhang, Z. Xie, J. Sun, X. Zou, and J. Wang, “A cascaded r-cnn with multiscale attention and imbalanced samples for traffic sign detection,” IEEE Access, vol. 8, pp. 29 742–29 754, 2020. [40] J. Cao, C. Song, S. Peng, F. Xiao, and S. Song, “Improved traffic sign detection and recognition algorithm for intelligent vehicles,” Sensors, vol. 19, no. 18, p. 4021, 2019. [41] Y. Jin, Y. Fu, W. Wang, J. Guo, C. Ren, and X. Xiang, “Multi-feature fusion and enhancement single shot detector for traffic sign recognition,” IEEE Access, vol. 8, pp. 38 931–38 940, 2020. [42] H.-Y. Lin, C.-C. Chang, V. L. Tran, and J.-H. Shi, “Improved traffic sign recognition for in-car cameras,” Journal of the Chinese Institute of Engineers, vol. 43, no. 3, pp. 300–307, 2020. [43] A. Avramovic, D. Sluga, D. Tabernik, D. Sko ´ caj, V. Stojni ˇ c, and N. Ilc, ´ “Neural-network-based traffic sign detection and recognition in highdefinition images using region focusing and parallelization,” IEEE Access, vol. 8, pp. 189 855–189 868, 2020. 41 [44] B. B. Fan and H. Yang, “Multi-scale traffic sign detection model with attention,” Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering, vol. 235, no. 2-3, pp. 708–720, 2021. [45] S. M. M. Ahsan, S. Das, S. Kumar, and Z. La Tasriba, “A detailed study on bangladeshi road sign detection and recognition,” in 2019 4th International Conference on Electrical Information and Communication Technology (EICT). IEEE, 2019, pp. 1–6. [46] S. Chakraborty, M. N. Uddin, and K. Deb, “Bangladeshi road sign recognition based on dtbs vector and artificial neural network,” in 2017 International Conference on Electrical, Computer and Communication Engineering (ECCE). IEEE, 2017, pp. 599–603. [47] Y. Fan and W. Zhang, “Traffic sign detection and classification for advanced driver assistant systems,” in 2015 12th international conference on Fuzzy systems and knowledge discovery (FSKD). IEEE, 2015, pp. 1335–1339. [48] T. Bui-Minh, O. Ghita, P. F. Whelan, and T. Hoang, “A robust algorithm for detection and classification of traffic signs in video data,” in 2012 International Conference on Control, Automation and Information Sciences (ICCAIS). IEEE, 2012, pp. 108–113. [49] M. Shahed, M. A. U. Khan, and S. A. Chowdhury, “Detection and recognition of bangladeshi road sign based on maximally stable extremal region,” in 2017 3rd International Conference on Electrical Information and Communication Technology (EICT). IEEE, 2017, pp. 1–6. [50] P. Dhar, M. Z. Abedin, T. Biswas, and A. Datta, “Traffic sign detection—a new approach and recognition using convolution neural network,” in 2017 IEEE Region 10 Humanitarian Technology Conference (R10- HTC). IEEE, 2017, pp. 416–419. [51] Y. Sun, P. Ge, and D. Liu, “Traffic sign detection and recognition based on convolutional neural network,” in 2019 Chinese Automation Congress (CAC). IEEE, 2019, pp. 2851–2854. 42 [52] Bangladesh Road Sign Manual, 2000, vol. 1 and 2. [Online]. Available: https://bsp.brta.gov.bd/roadSignMannul;jsessionid= mRcupj7oFff6rDG9QKvVOBturxwVFmDYUlMnD3dkPICYxuDMbQ5n! 1875640356?lan=en [53] J. Huang, V. Rathod, C. Sun, M. Zhu, A. Korattikara, A. Fathi, I. Fischer, Z. Wojna, Y. Song, S. Guadarrama et al., “Speed/accuracy trade-offs for modern convolutional object detectors,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 7310–7311.