Human Body Posture Recognition Approaches:

Mohammed A. Ali; Abir J. Hussain; Ahmed T.  Sadiq

doi:10.14500/aro.10930

Mohammed A. Ali Department of Computer Sciences, University of Technology, Baghdad, Iraq https://orcid.org/0000-0002-9117-1221
Abir J. Hussain School of Computer Sciences and Mathematics, Liverpool John Moores University, Liverpool, England https://orcid.org/0000-0001-8413-0045
Ahmed T. Sadiq Department of Computer Sciences, University of Technology, Baghdad, Iraq https://orcid.org/0000-0002-4217-1321

Keywords: Acceleration based, Computer vision, Health monitoring, Human body posture recognition, Security

Abstract

Human body posture recognition has become the focus of many researchers in recent years. Recognition of body posture is used in various applications, including surveillance, security, and health monitoring. However, these systems that determine the body’s posture through video clips, images, or data from sensors have many challenges when used in the real world. This paper provides an important review of how most essential ‎ hardware technologies are ‎used in posture recognition systems‎. These systems capture and collect datasets through ‎accelerometer sensors or computer vision. In addition, this paper presents a comparison ‎study with state-of-the-art in terms of accuracy. We also present the advantages and ‎limitations of each system and suggest promising future ideas that can increase the ‎efficiency of the existing posture recognition system. Finally, the most common datasets ‎applied in these systems are described in detail. It aims to be a resource to help choose one of the methods in recognizing the posture of the human body and the techniques that suit each method. It analyzes more than 80 papers between 2015 and 2020

Downloads

Download data is not yet available.

Author Biographies

Mohammed A. Ali, Department of Computer Sciences, University of Technology, Baghdad, Iraq

Mohammed A. Ali is a Ph.D. student at the Department of Computer Science, University of Technology. He got the B.Sc. and M.Sc. degree in Computer Sciences. His research interests are in Machine learning, Data Security and Computer vision.

Abir J. Hussain, School of Computer Sciences and Mathematics, Liverpool John Moores University, Liverpool, England

Abir J. Hussain is a Professor at the Department of Computer Science, Liverpool John Moores University. She got the B.Sc., M.Sc. and Ph.D. degree in Computer Sciences. His research interests are in machine learning algorithms and their applications to medical, image and signal processing and data analysis.

Ahmed T. Sadiq, Department of Computer Sciences, University of Technology, Baghdad, Iraq

Ahmed T. Sadiq is a Professor at the Department of Computer Science, University of Technology. He got the B.Sc., M.Sc. and Ph.D. degree in Computer Sciences. His research interests are in Artificial Intelligence, Data Mining, Pattern Recognition and Data Security.

References

Abdulhussein, A.A. and Raheem, F.A., 2020. Hand gesture recognition of static letters American sign language (ASL) using deep learning. Engineering and Technology Journal, 38(6A), pp.926-937.

Abedi, W.M.S., Ibraheem-Nadher, D. and Sadiq, A.T., 2020. Modified deep learning method for body postures recognition. International Journal of Advanced Science and Technology, 29, 3830-3841.

Andriluka, M., Pishchulin, L., Gehler, P. and Schiele, B., 2014. 2D Human Pose Estimation: New Benchmark and State of the Art Analysis. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. pp.3686-3693.

Ben Mabrouk, A. and Zagrouba, E., 2018. Abnormal behavior recognition for intelligent video surveillance systems: A review. Expert Systems with Applications, 91, pp.480-491.

Cao, Z., Simon, T., Wei, S.E. and Sheikh, Y., 2017. Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields. In: Proceedings 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, 2017-Janua, pp.1302-1310.

Chen, Y. and Xue, Y., 2016. A Deep Learning Approach to Human Activity Recognition Based on Single Accelerometer. In: Proceedings 2015 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2015. IEEE, pp.1488-1492.

Chen, Y., Tian, Y. and He, M., 2020. Monocular human pose estimation: A survey of deep learning-based methods. Computer Vision and Image Understanding, 192, p.102897.

Colleges, Gujarat Technological University, 2014. Microsoft COCO. Eccv, (June). Colleges, Gujarat Technological University, pp.740-755.

Dang, Q., Yin, J., Wang, B. and Zheng, W., 2019. Deep learning based 2D human pose estimation: A survey. Tsinghua Science and Technology, 24, pp. 663-676.

Ding, W., Hu, B., Liu, H., Wang, X. and Huang, X., 2020. Human posture recognition based on multiple features and rule learning. International Journal of Machine Learning and Cybernetics, 11(11), pp.2529-2540.

El Amine Elforaici, M., Chaaraoui, I., Bouachir, W., Ouakrim Y. and Mezghani, N., 2018. Posture Recognition Using an rgb-d Camera: Exploring 3d Body Modeling and Deep Learning Approaches. In: 2018 IEEE Life Sciences Conference, LSC. IEEE, pp.69-72.

Ghafoor, K.J., Rawf, K.M., Abdulrahman, A.O. and Taher, S.H., 2021. Kurdish Dialect Recognition using 1D CNN. Aro-the Scientific Journal of Koya University, 9(2), pp.10-14.

Gong, W., Zhang, X., Gonzàlez, J., Sobral, A., Bouwmans, T., Tu, C. and Zahzah, E., 2016. Human pose estimation from monocular images: A comprehensive survey. Sensors, 16(12), p.1966.

Gouiaa, R. and Meunier, J., 2017. Learning cast shadow appearance for human posture recognition. Pattern Recognition Letters, 97, pp.54-60.

Gupta, A., Gupta, K., Gupta, K. and Gupta, K., 2020. A Survey on Human Activity Recognition and Classification. In: 2020 International Conference on Communication and Signal Processing (ICCSP). IEEE, pp.915-919.

Hameed, F.S., Alwan, H.M. and Ateia, Q.A., 2020. Pose estimation of objects using digital image processing for pick-and-place applications of robotic arms. Engineering and Technology Journal, 38(5A), pp.707-718.

Hassan, M.M., Huda, S., Uddin, M.Z., Almogren, A. and Alrubaian, M., 2018. Human activity recognition from body sensor data using deep learning. Journal of Medical Systems, 42(6), p.99.

Hsiao, R.S., Chen, T.X., Bitew, M.A., Kao, C.H. and Li, T.Y., 2018. Sleeping posture recognition using fuzzy c-means algorithm. BioMedical Engineering Online, 17(S2), p.157.

Huang, X., Wang, F., Zhang, J., Hu, Z. and Jin, J., 2019. A posture recognition method based on indoor positioning technology. Sensors, 19(6), p.1464.

Jalal, A., Akhtar, I. and Kim, K., 2020. Human posture estimation and sustainable events classification via Pseudo-2D stick model and K-ary tree hashing. Sustainability (Switzerland), 12(23), pp.1-24.

Joo, H., Simon, T., Li, X., Liu, H., Tan, L., Gui, L., Banerjee, S., Godisart, T., Nabbe, B., Matthews, I., Kanade, T., Nobuhara, S. and Sheikh, Y., 2019. Panoptic studio: A massively multiview system for social interaction capture. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(1), pp.190-204.

Kale, H., Mandke, P., Mahajan, H. and Deshpande, V., 2018. Human Posture Recognition using Artificial Neural Networks. In: Proceedings of the 8th International Advance Computing Conference, IACC 2018. IEEE, pp.272-278.

Kamel, A., Sheng, B., Yang, P., Li, P., Ruimin, S. and Feng, D.D., 2019. Deep convolutional neural networks for human action recognition using depth maps and postures. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 49(9), pp.1806-1819.

Kamiŝalić, A., Fister, I., Turkanović, M. and Karakatič, S., 2018. Sensors and functionalities of non-invasive wrist-wearable devices: A review. Sensors (Switzerland), 18(6), p.1714.

Kanazawa, A., Black, M.J., Jacobs, D.W., Malik, J., 2018. End-to-end recovery of human shape and pose. IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp.7122-7131.

Kareem, A.A., Ali, W.H. and Jasim, M.H., 2020. Design and implementation of a wireless system to locate a user in indoor environments. Engineering and Technology Journal, 38(11A), pp.1640-1651.

Khokhlov, I., Reznik, L., Cappos, J. and Bhaskar, R., 2018. Design of Activity Recognition Systems with Wearable Sensors. In: 2018 IEEE Sensors Applications Symposium (SAS). IEEE, pp.1-6.

Kim, S.T. and Lee, H.J., 2020. Lightweight stacked hourglass network for human pose estimation. Applied Sciences, 10(18), p.6497.

Kołodziej, M., Majkowski, A., Tarnowski, P., Rak, R.J., Gebert, D. and Sawicki, D., 2019. Registration and analysis of acceleration data to recognize physical activity. Journal of Healthcare Engineering, 2019, pp.9497151.

Li, W., Zhang, Z. and Liu, Z., 2010. Action Recognition Based on a Bag of 3D Points. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. IEEE, pp.9-14.

Liang, M. and Hu, Y., 2020. Application of Human Body Posture Recognition Technology in Robot Platform for Nursing Empty-Nesters. In: 2020 6th International Conference on Control, Automation and Robotics, ICCAR 2020. IEEE, pp.91-95.

Liu, G., Lin, L., Zhou, W., Zhang, R., Yin, H., Chen, J. and Guo, H., 2019. A posture recognition method applied to smart product service. In: Procedia CIRP, pp.425-428.

Liu, J., Wang, Y., Liu, Y., Xiang, S. and Pan, C., 2020. 3D PostureNet: A unified framework for skeleton-based posture recognition. Pattern Recognition Letters, 140, pp.143-149.

Liu, M., Liu, H. and Chen, C., 2017. Enhanced skeleton visualization for view invariant human action recognition. Pattern Recognition, 68, pp.346-362.

Liu, Z., Zhu, J., Bu, J. and Chen, C., 2015. A survey of human pose estimation: The body parts parsing based methods. Journal of Visual Communication and Image Representation, 32, pp.10-19.

Lo Presti, L. and La Cascia, M., 2016. 3D skeleton-based human action classification: A survey. Pattern Recognition, 53, pp.130-147.

Marszałek, M., Laptev, I. and Schmid, C., 2009. Marszałek-cvpr09-Action in Context, pp.2929-2936.

Mehta, D., Rhodin, H., Casas, D., Fua, P., Sotnychenko, O., Xu, W. and Theobalt, C., 2018. Monocular 3D Human Pose Estimation in the Wild Using Improved CNN Supervision. In: Proceedings 2017 International Conference on 3D Vision, 3DV 2017, pp.506-516.

Mirza, S.F. and Al-Talabani, A.K., 2021. Efficient kinect sensor-based kurdish sign language recognition using echo system network. ARO The Scientific Journal of Koya University, 9(2), pp.1-9.

Mo, L., Li, F., Zhu, Y. and Huang, A., 2016. Human Physical Activity Recognition Based on Computer Vision with Deep Learning Model. In: 2016 IEEE International Instrumentation and Measurement Technology Conference Proceedings. IEEE, pp.1-6.

Nandy, A., Saha, J. and Chowdhury, C., 2020. Novel features for intensive human activity recognition based on wearable and smartphone sensors. Microsystem Technologies, 26(6), pp.1889-1903.

Neili, S., Gazzah, S., El Yacoubi, M.A. and Ben Amara, N.E., 2017. Human Posture Recognition Approach Based on ConvNets and SVM Classifier. In: Proceedings 3rd International Conference on Advanced Technologies for Signal and Image Processing, ATSIP 2017. IEEE, pp.1-6.

Niebles, J.C., Chen, C.W. and Fei-Fei, L., 2010. Modeling temporal structure of decomposable motion segments for activity classification. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 6312 LNCS (PART 2), pp.392-405.

Ning, G., Zhang, Z. and He, Z., 2018. Knowledge-guided deep fractal neural networks for human pose estimation. IEEE Transactions on Multimedia, 20(5), pp.1246-1259.

Patel, P., Bhatt, B. and Patel, B., 2017. Human Body Posture Recognition a Survey. In: 2017 International Conference on Innovative Mechanisms for Industry Applications (ICIMIA). IEEE, pp.473-477.

Quan, W., Woo, J., Toda, Y. and Kubota, N., 2019. Human posture recognition for estimation of human body condition. Journal of Advanced Computational Intelligence and Intelligent Informatics, 23(3), pp.519-527.

Rahmani, H. and Bennamoun, M., 2017. Learning action recognition model from depth and skeleton videos. In: Proceedings of the IEEE International Conference on Computer Vision. IEEE, pp.5833-5842.

Rahmani, H., Mian, A. and Shah, M., 2018. Learning a deep model for human action recognition from novel viewpoints. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(3), pp.667-681.

Ravi, D., Wong, C., Lo, B. and Yang, G.Z., 2016. Deep Learning for Human Activity Recognition: A Resource Efficient Implementation on Low-power Devices. In: 2016 IEEE 13th International Conference on Wearable and Implantable Body Sensor Networks (BSN). IEEE, pp.71-76.

Rawat, W. and Wang, Z., 2017. Deep convolutional neural networks for image classification: A comprehensive review. Neural Computation, 29(9), pp.2352-2449.

Reddy, K.K. and Shah, M., 2013. Recognizing 50 human action categories of web videos. Machine Vision and Applications, 24(5), pp.971-981.

Ren, W., Ma, O., Ji, H. and Liu, X., 2020. Human posture recognition using a hybrid of fuzzy logic and machine learning approaches. IEEE Access, 8, pp.135628-135639.

Saini, R., Kumar, P., Kaur, B., Roy, P.P., Dogra, D.P. and Santosh, K.C., 2019. Kinect sensor-based interaction monitoring system using the BLSTM neural network in healthcare. International Journal of Machine Learning and Cybernetics, 10(9), pp.2529-2540.

Santhoshkumar, R. and Geetha, M.K., 2019. Deep Learning Approach for Emotion Recognition from Human Body Movements with Feedforward Deep Convolution Neural Networks. In: Procedia Computer Science, pp.158-165.

Sarafianos, N., Boteanu, B., Ionescu, B. and Kakadiaris, I.A., 2016. 3D human pose estimation: A review of the literature and analysis of covariates. Computer Vision and Image Understanding, 152, pp.1-20.

Shahroudy, A., Liu, J., Ng, T.T. and Wang, G., 2016. NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2016-Decem. pp.1010-1019.

Soomro, K., Zamir, A.R. and Shah, M., 2012. UCF101: A Dataset of 101 Human Actions Classes from Videos in the Wild. arXiv, 2012, 12120402.

Sun, W. and Wang, R., 2018. Fully convolutional networks for semantic segmentation of very high resolution remotely sensed images combined with DSM. IEEE Geoscience and Remote Sensing Letters, 15(3), pp.474-478.

Vecchio, A., Mulas, F. and Cola, G., 2017. Posture recognition using the interdistances between wearable devices. IEEE Sensors Letters, 1(4), pp.1-4.

Voicu, R.A., Dobre, C., Bajenaru, L. and Ciobanu, R.L., 2019. Human physical activity recognition using smartphone sensors. Sensors, 19(3), p.458.

Wan, S., Qi, L., Xu, X., Tong, C. and Gu, Z., 2020. Deep learning models for real-time human activity recognition with smartphones. Mobile Networks and Applications, 25(2), pp.743-755.

Wang, J. and Liu, X.H., 2020. Human Posture Recognition Method Based on Skeleton Vector with Depth Sensor. In: IOP Conference Series: Materials Science and Engineering, p.012035.

Wang, J., Nie, X., Xia, Y., Wu, Y. and Zhu, S.C., 2014. Cross-view Action Modeling, Learning, and Recognition. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp.2649-2656.

Wang, P., Li, W., Ogunbona, P., Wan, J. and Escalera, S., 2018. RGB-D-based human motion recognition with deep learning: A survey. Computer Vision and Image Understanding, 171, pp.118-139.

Wang, W.J., Chang, J.W., Haung, S.F. and Wang, R.J., 2016. Human posture recognition based on images captured by the kinect sensor. International Journal of Advanced Robotic Systems, 13(2), p.54.

Wu, D., Sharma, N. and Blumenstein, M., 2017. Recent Advances in Video-based Human Action Recognition Using Deep Learning: AReview. In: 2017 International Joint Conference on Neural Networks (IJCNN). IEEE, pp.2865-2872.

Xia, L., Chen, C. and Aggarwal, J., 2012. View Invariant Human Action Recognition Using Histograms of 3D Joints The University of Texas at Austin. CVPR 2012 HAU3D Workshop, pp.20-27.

Xiao, J., Cui, X. and Li, F., 2020. Human action recognition based on convolutional neural network and spatial pyramid representation. Journal of Visual Communication and Image Representation, 71, p.102722.

Yan, C., Coenen, F. and Zhang, B., 2016. Driving posture recognition by convolutional neural networks. IET Computer Vision, 10(2), pp.103-114.

Zhang, H., Yan, X. and Li, H., 2018. Ergonomic posture recognition using 3D view-invariant features from single ordinary camera. Automation in Construction, 94, pp.1-10.

Zhang, H.B., Lei, Q., Zhong, B.N., Du, J.X. and Peng, J.L., 2016. A survey on human pose estimation. Intelligent Automation and Soft Computing, 22(3), pp.483-489.

Zhang, H.B., Zhang, Y.X., Zhong, B., Lei, Q., Yang, L., Du, J.X., Chen, D.S., 2019. A comprehensive survey of vision-based human action recognition methods. Sensors, 19(5), p.1005.

Zhang, J., Wu, C. and Wang, Y., 2020. Human fall detection based on body posture spatio-temporal evolution. Sensors (Switzerland), 20(3), p.946.

Zhang, S. and Callaghan, V., 2020. Real-time human posture recognition using an adaptive hybrid classifier. International Journal of Machine Learning and Cybernetics, 12(2), pp.489-499.

Zhang, S., McCullagh, P., Zheng, H. and Nugent, C., 2017. Situation awareness inferred from posture transition and location: Derived from smartphone and smart home sensors. IEEE Transactions on Human-Machine Systems, 47(6), pp.814-821.

Zhao, C.H., Zhang, B.L., He, J. and Lian, J., 2012. Recognition of driving postures by contourlet transform and random forests. IET Intelligent Transport Systems, 6(2), pp.161-168.

Zhao, J. and Obonyo, E., 2020. Convolutional long short-term memory model for recognizing construction workers’ postures from wearable inertial measurement units. Advanced Engineering Informatics, 46, p.101177.

Zhao, L. and Chen, W., 2020. Detection and recognition of human body posture in motion based on sensor technology. IEEJ Transactions on Electrical and Electronic Engineering, 15(5), pp.766-770.

Zhao, Z.Q., Zheng, P., Xu, S.T. and Wu, X., 2019. Object detection with deep learning: A review. IEEE Transactions on Neural Networks and Learning Systems, 30(11), pp.3212-3232.

Zhou, E. and Zhang, H., 2020. Human action recognition toward massive-scale sport sceneries based on deep multi-model feature fusion. Signal Processing: Image Communication, 84, p.115802.

Zhou, Y., Sun, X., Zha, Z.J. and Zeng, W., 2018. MiCT: Mixed 3D/2D Convolutional tube for human action recognition. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, pp.449-458.

Human Body Posture Recognition Approaches

A Review

Abstract

Downloads

Author Biographies

References