Rough Sets Theory with Deep Learning for Tracking in Natural Interaction with Deaf
محورهای موضوعی : Machine learning
Mohammad Ebrahimi
1
,
Hossein Ebrahimpour-Komeleh
2
1 - Electrical and Computer Engineering Kashan University
2 - Faculty of Electrical and Computer Engineering Kashan University
کلید واژه: Natural Interaction with Deaf, Machine Vision, Persian Deaf News Hand Tracking, Sign Language, Rough Sets Theory, Deep Learning.,
چکیده مقاله :
Sign languages commonly serve as an alternative or complementary mode of human communication Tracking is one of the most fundamental problems in computer vision, and use in a long list of applications such as sign languages recognition. Despite great advances in recent years, tracking remains challenging due to many factors including occlusion, scale variation, etc. The mistake detecting of head or left hand instead of right hand in overlapping are, modes like this, and due to the uncertainty of the hand area over the deaf news video frames; we proposed two methods: first, tracking using particle filter and second tracking using the idea of the rough set theory in granular information with deep neural network. We proposed the method for Combination the Rough Set with Deep Neural Network and used for in Hand/Head Tracking in Video Signal DeafNews. We develop a tracking system for Deaf News. We used rough set theory to increase the accuracy of skin segmentation in video signal. Using deep neural network, we extracted inherent relationships available in the frame pixels and generalized the achieved features to tracking. The system proposed is tested on the 33 of Deaf News with 100 different words and 1927 video files for words then recall, MOTA and MOTP values are obtained.
Sign languages commonly serve as an alternative or complementary mode of human communication Tracking is one of the most fundamental problems in computer vision, and use in a long list of applications such as sign languages recognition. Despite great advances in recent years, tracking remains challenging due to many factors including occlusion, scale variation, etc. The mistake detecting of head or left hand instead of right hand in overlapping are, modes like this, and due to the uncertainty of the hand area over the deaf news video frames; we proposed two methods: first, tracking using particle filter and second tracking using the idea of the rough set theory in granular information with deep neural network. We proposed the method for Combination the Rough Set with Deep Neural Network and used for in Hand/Head Tracking in Video Signal DeafNews. We develop a tracking system for Deaf News. We used rough set theory to increase the accuracy of skin segmentation in video signal. Using deep neural network, we extracted inherent relationships available in the frame pixels and generalized the achieved features to tracking. The system proposed is tested on the 33 of Deaf News with 100 different words and 1927 video files for words then recall, MOTA and MOTP values are obtained.
[1] H. Tang, H. Liu, W. Xiao and N. Sebe, "Fast and robust dynamic hand gesture recognition via key frames extraction and feature fusion", Neurocomputing, Vol. 331, 2019, pp. 424-433.
[2] L. Kraljevi´c, M. Russo, M. Paukovi´c and M. Šari´c, “A Dynamic Gesture Recognition Interface for Smart Home Control based on Croatian Sign Language”, Appl. Sci. 2020, 10, 2300.
[3] A. Wadhawan and P. Kumar, "Sign Language Recognition Systems: A Decade Systematic Literature Review", Arch Computat Methods Eng 2019, https://doi.org/10.1007/s11831-019-09384-2.
[4] P. Kim, "MATLAB Deep Learning: With Machine Learning, Neural Networks and Artificial Intelligence", Apress, 2017.
[5] A.E. Hassanien, A. Abraham, J.F. Peters, G. Schaefer, Henry C., "Rough sets and near sets in medical imaging: a review", IEEE Transactions on Information Technology in Biomedicine, Vol. 13(6), 2009, pp. 955-968.
[6] V. Sattari-Naeini, and A. Moaref, "Fuzzy-rough Information Gain Ratio Approach to Filter-wrapper Feature Selection", International Journal of Engineering, Vol. 30(9), 2017, pp. 1326-1333.
[7] D. Li, C. R. Opazo, X. Yu and H. Li, "Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison", Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2020, pp. 1459-1469.
[8] A. H. Mazinan, J. Hassanian, “A Hybrid Object Tracking for Hand Gesture Approach based on MS-MD and its Application,” Journal of Information Systems and Telecommunication (JIST), 2015, Vol. 3, No. 4.
[9] S. Ildarabadi, M. Ebrahimi, H. R. Pourreza, "Improvement Tracking Dynamic Programming using Replication Function for Continuous Sign Language Recognition", International Journal of Engineering Trends and Technology (IJETT), Vol 7(3), 2014, pp. 97-101.
[10] P. Chiranjeevi, S. Sengupta, “Rough-Set-Theoretic Fuzzy Cues-Based Object Tracking Under Improved Particle Filter Framework”, IEEE transactions on fuzzy systems, Vol. 24, 2016, No. 3.
[11] J. B. Zhang, T. R. Li, & H. M. Chen. "Composite rough sets for dynamic data mining", Information Science, Vol. 257, 2014, pp. 81–100.
[12] Y. Xu, A. Sep, Y. Ban, R. Horaud, L. Leal-Taixe and X. Alameda-Pineda, “How to train your deep multi-object tracker", Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), pp. 6787-6796, Jun. 2020.
