Alphabet recognition in unconstrained Air Writing using Depth Information
Date
2018-08-30
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Department of Computer Science and Engineering (CSE), Islamic University of Technology (IUT), Board Bazar, Gazipur-1704, Bangladesh
Abstract
In this thesis, we present a machine learning approach to recognize on-air writing
of English Capital Alphabets (ECAs) using di erent feature is introduced include
depth information. The hand nger's motion while writing the alphabet in the
air was captured as depth images with the help of a depth camera. The depth
images were then processed to track nger movements and after that smoothing
procedure was applied to generate hand trajectory data. 11 point-wise features
including depth value were calculated from the hand trajectory data which are
also time series. Each air written alphabet is then compared with 26 alphabet
templates using Dynamic Time Warping (DTW). The DTW distance features are
normalized between 0 to 1 and used as features. So, a feature vector of 11x26 =286
normalized features and the appropriate class label was fed to Support Vector
Machine for training and testing. 15 fold cross veri cation classi cation result
provided an average accuracy of 55.4% with 15 users.
We also explored feature removal method based on a gain ratio. We removed the
features that have the worst gain ratio. Iteratively 60 features were removed and
the accuracies were compared. However, the best accuracy of 57.17% was found
by removing eight features.
Description
Supervised by
Dr. Md. Kamrul Hasan
Professor,
Department of Computer Science and Engineering,
Islamic University of Technology. Board Bazar, Gazipur, Bangladesh
Keywords
Air Writing; Gesture Recognition; Depth Information; Time Series; Dynamic Time Warping; Support Vector Machine;
Citation
[1] C. Amma, M. Georgi, and T. Schultz, \Airwriting: A wearable handwriting recognition system," Personal Ubiquitous Comput., vol. 18, no. 1, pp. 191{ 203, Jan. 2014. [2] A. Wexelblat, \An approach to natural gesture in virtual environments," ACM Transactions on Computer-Human Interaction, vol. 2, no. 3, pp. 179{ 200, sep 1995. [3] A. Priya, S. Mishra, S. Raj, S. Mandal, and S. Datta, \Online and o ine character recognition: A survey," in 2016 International Conference on Com- munication and Signal Processing (ICCSP). IEEE, apr 2016. [4] S. Agrawal, I. Constandache, S. Gaonkar, R. Roy Choudhury, K. Caves, and F. DeRuyter, \Using mobile phones to write in air," in Proceedings of the 9th International Conference on Mobile Systems, Applications, and Services, ser. MobiSys '11. New York, NY, USA: ACM, 2011, pp. 15{28. [5] M. K. H. H. A. R. Robiul Islam, Hasan Mahmud, \Alphabet recognition in air writing using depth information," The Ninth International Conference on Advances in Computer-Human Interactions, pp. 299{301, April 2016. [6] A. Jalalian and S. K. Chalup, \Gdtw-p-svms: Variable-length time series analysis using support vector machines," Neurocomputing, vol. 99, pp. 270 { 282, 2013. [7] N. S. Sreekanth and N. K. Narayanan, \Dynamic gesture recognition|a machine vision based approach," pp. 105{115, 2017. [8] I. E. Sutherland, \Sketchpad," in Proceedings of the May 21-23, 1963, spring joint computer conference on - AFIPS '63 (Spring). ACM Press, 1963. [9] W. Buxton, \Lexical and pragmatic considerations of input structures," ACM SIGGRAPH Computer Graphics, vol. 17, no. 1, pp. 31{37, jan 1983. 40 Bibliography 41 [10] P. R. Cohen, M. Johnston, D. McGee, S. Oviatt, J. Pittman, I. Smith, L. Chen, and J. Clow, \QuickSet," in Proceedings of the fth conference on Applied natural language processing -. Association for Computational Linguistics, 1997. [11] D. J. Sturman, D. Zeltzer, and S. Pieper, \Hands-on interaction with virtual environments," in Proceedings of the 2nd annual ACM SIGGRAPH sympo- sium on User interface software and technology - UIST '89. ACM Press, 1989. [12] F. K. H. Quek, \Toward a vision-based hand gesture interface," in Proceedings of the Conference on Virtual Reality Software and Technology, ser. VRST '94. River Edge, NJ, USA: World Scienti c Publishing Co., Inc., 1994, pp. 17{31. [13] R. A. Bolt, \\put-that-there"," ACM SIGGRAPH Computer Graphics, vol. 14, no. 3, pp. 262{270, jul 1980. [14] W. T. Freeman and C. D. Weissman, \Television control by hand gestures," in International Workshop on Automatic Face and Gesture Recognition, 1995, pp. 179{183. [15] euronews Knowledge. Future of texting: writing in the air! Youtube. [Online]. Available: https://www.youtube.com/watch?v=XMU4zh083l4 [16] D. H. Kim, H. I. Choi, and J. H. Kim, \3d space handwriting recognition with ligature model," in Proceedings of the Third International Conference on Ubiquitous Computing Systems, ser. UCS'06. Berlin, Heidelberg: Springer- Verlag, 2006, pp. 41{56. [17] S. Mori, C. Suen, and K. Yamamoto, \Historical review of OCR research and development," Proceedings of the IEEE, vol. 80, no. 7, pp. 1029{1058, jul 1992. [18] I. Adeyanju, O. Ojo, and E. Omidiora, \Recognition of typewritten characters using hidden markov models," vol. 12, pp. 1{9, 01 2016. [19] N. Greco, D. Impedovo, M. Lucchese, A. Salzo, and L. Sarcinella, \Bankcheck processing system: modi cations due to the new european currency," pp. 343{ 348 vol.1, 09 2003. [20] F. L. Siena, B. Byrom, P.Watts, and P. Breedon, \Utilising the intel realsense camera for measuring health outcomes in clinical research," Journal of medical systems, vol. 42, no. 3, p. 53, 2018. Bibliography 42 [21] S. Totilo. Natal recognizes 31 body parts, uses tenth of xbox 360 "computing resources". [Online]. Available: https://kotaku.com/5442775/ natal-recognizes-31-body-parts-uses-tenth-of-xbox-360-computing-resources [22] Kinect - wikipedia. Wikipedia. [Online]. Available: https://en.wikipedia.org/ wiki/Kinect [23] B. Pang, L. Lee, and S. Vaithyanathan, \Thumbs up?: Sentiment classi- cation using machine learning techniques," in Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing - Volume 10, ser. EMNLP '02. Stroudsburg, PA, USA: Association for Computational Linguistics, 2002, pp. 79{86. [24] T. Evgeniou, M. Pontil, and T. Poggio, \Statistical learning theory: A primer," vol. 38, pp. 9{13, 06 2000. [25] O. Bousquet, S. Boucheron, and G. Lugosi, \Introduction to statistical learning theory," in Advanced Lectures on Machine Learning. Springer Berlin Heidelberg, 2004, pp. 169{207. [26] R. Mullin, \Time warps, string edits, and macromolecules: The theory and practice of sequence comparison. edited by d. sanko and j. b. kruskal. addison-wesley publishing company, inc., advanced book program, reading, mass., don mills, ontario, 1983. 300 pp. u. s. $31.95. ISBN 0-201-07809-0," Canadian Journal of Statistics, vol. 13, no. 2, pp. 167{168, jun 1985. [27] C. Qu, \Online kinect handwritten digit recognition based on dynamic time warping and support vector machine," vol. 12, pp. 413{422, 01 2015. [28] D. M. Viswanathan and S. M. Idicula, \Recognition of hand gestures of english alphabets using HOG method," in 2014 International Conference on Data Science & Engineering (ICDSE). IEEE, aug 2014. [29] M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten, \The weka data mining software: An update," SIGKDD Explor. Newsl., vol. 11, no. 1, pp. 10{18, Nov. 2009. [30] N. R. Draper and H. Smith, \Applied regression analysis, john wiley & sons inc," Applied regression analysis. 2nd ed. John Wiley and Sons, NY., 1981. [31] M. Bennasar, Y. Hicks, and R. Setchi, \Feature selection using joint mutual information maximisation," Expert Systems with Applications, vol. 42, no. 22, pp. 8520 { 8532, 2015. Bibliography 43 [32] H. Liu and R. Setiono, \Chi2: feature selection and discretization of numeric attributes," in Proceedings of 7th IEEE International Conference on Tools with Arti cial Intelligence, Nov 1995, pp. 388{391. [33] J. F. Kenney and E. S. Keeping, \Moving averages," pp. 221{223, 1962. [34] A. G. Karegowda, A. Manjunath, and M. Jayaram, \Comparative study of attribute selection using gain ratio and correlation based feature selection," In- ternational Journal of Information Technology and Knowledge Management, vol. 2, no. 2, pp. 271{277, 2010.