volume | PIER Journals

Abstract

In order to solve the problem that the recognition accuracy of human motion is not high when a single feature is used, a feature fusion human motion recognition method based on Frequency Modulated Continuous Wave (FMCW) radar is proposed. By preprocessing the FMCW radar echo data, the range and Doppler parameters of human motions are obtained, and the range-time feature map and Doppler-time feature map datasets are constructed. In order to fully extract and accurately identify the human motion features, the two features are fused, and then the two features maps and feature fusion spectrograms are put into the VGG16 network model based on transfer learning for identification and classification. Experimental results show that this method can effectively solve the problem of lack of information and recognition rate of single feature motion recognition, and the recognition accuracy is more than 1{\%} higher than that of the single feature recognition method.

1. Yang, L. M. and Z. H. Li, "Design of gesture recognition system towards human computer interaction," Industrial Control Computer, Vol. 33, No. 3, 18-20, Mar. 2020. Google Scholar

2. Zhang, Y. Y. and X. Guo, "Research and realization of dynamical gesture recognition algorithm based on kinect," Computer Technology and Development, Vol. 27, No. 12, 11-15, Aug. 2017. Google Scholar

3. Liu, Y., R. Y. Xie, Y. Feng, et al., "Survey on resident’s daily activity recognition in smart homes," Computer Engineering and Applications, Vol. 54, No. 7, 35-42, Jan. 2021. Google Scholar

4. Gao, X. W., Z. Shen, G. Y. Xu, et al., "Traffic anomaly detection based on multi-target tracking," Application Research of Computers, Vol. 38, No. 6, 1879-1883, Dec. 2021. Google Scholar

5. Tran, Du, Heng Wang, Lorenzo Torresani, Jamie Ray, Yann LeCun, and Manohar Paluri, "A closer look at spatiotemporal convolutions for action recognition," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 6450-6459, Salt Lake City, USA, Jun. 2018.

6. Xiong, X., Y. Zheng, and S. Zhang, "Fall detection and human behavior recognition system based on long and short time memory networks and variants," Information Communications, No. 2, 65-67, Feb. 2020. Google Scholar

7. Sabokrou, Mohammad, Masoud Pourreza, Mohsen Fayyaz, Rahim Entezari, Mahmood Fathy, Jürgen Gall, and Ehsan Adeli, "AVID: Adversarial visual irregularity detection," 14th Asian Conference on Computer Vision, 488-505, Perth, Australia, 2018.

8. Liu, Tianliang, Qingwei Qiao, Junwei Wang, Xiubin Dai, and Jiebo Luo, "Human action recognition via spatio-temporal dual network flow and visual attention fusion," Journal of Electronics & Information Technology, Vol. 40, No. 10, 2395-2401, Aug. 2018. Google Scholar

9. Jiang, L. B., G. Y. Wei, and L. Che, "Human motion recognition by 77 GHz radar based on dictionary learning," Science Technology and Engineering, Vol. 20, No. 6, 2137-2324, Feb. 2020. Google Scholar

10. Li, Xinyu, Yuan He, and Xiaojun Jing, "A survey of deep learning-based human activity recognition in radar," Remote Sensing, Vol. 11, No. 9, 1068, 2019. Google Scholar

11. Lee, Jonghyeok, Sunghyun Hwang, Sungjin You, Woo-Jin Byun, and Jaehyun Park, "Joint angle, velocity, and range estimation using 2D MUSIC and successive interference cancellation in FMCW MIMO radar system," IEICE Transactions on Communications, Vol. 103, No. 3, 283-290, 2020. Google Scholar

12. Shrestha, Aman, Haobo Li, Julien Le Kernec, and Francesco Fioranelli, "Continuous human activity classification from FMCW radar with Bi-LSTM networks," IEEE Sensors Journal, Vol. 20, No. 22, 13607-13619, 2020. Google Scholar

13. Zhang, L. L., B. Liu, L. L. Qu, et al., "Human activity recognition with FMCW radar based on fusion feature convolutional neural network," Telecommunication Engineering, Vol. 62, No. 2, 147-154, Jul. 2022. Google Scholar

14. Wang, Yong, Jinjun Wu, Zengshan Tian, Mu Zhou, and Shasha Wang, "Gesture recognition with multi-dimensional parameter using FMCW radar," Journal of Electronics & Information Technology, Vol. 41, No. 4, 822-829, 2019. Google Scholar

15. Zhao, Yinan, Zihao Zhang, and Zhaolin Zhang, "Multi-angle data cube action recognition based on millimeter wave radar," 2020 Chinese Control and Decision Conference (CCDC), 749-753, Hefei, China, Aug. 2020.

16. Franceschini, Stefano, Michele Ambrosanio, Vito Pascazio, and Fabio Baselice, "Hand gesture signatures acquisition and processing by means of a novel ultrasound system," Bioengineering, Vol. 10, No. 1, 36, 2023. Google Scholar

17. Simonyan, Karen and Andrew Zisserman, "Very deep convolutional networks for large-scale image recognition," Computer Science, 2014. Google Scholar

18. Hashemi, Sajjad, Hojjat Emami, and Amin Babazadeh Sangar, "A new comparison framework to survey neural networks-based vehicle detection and classification approaches," International Journal of Communication Systems, Vol. 34, No. 14, e4928, 2021. Google Scholar

19. Ali, Mohamed Ashraf, Hossam E. Abd El Munim, Ahmed Hassan Yousef, and Sherif Hammad, "A deep learning approach for vehicle detection," 2018 13th International Conference on Computer Engineering and Systems (ICCES), 98-102, Egypt, Dec. 2018.

20. Qi, C., Y. Zuo, Z. Chen, and K. Chen, "Rice processing accuracy classification method based on improved VGG16 convolution neural network," Transactions of the Chinese Society of Agricultural Machinery, Vol. 52, No. 5, 301-307, Mar. 2021. Google Scholar

21. Zhuang, F. Z., P. Luo, Q. He, et al., "Survey on transfer learning research," Journal of Software, Vol. 26, No. 1, 26-39, Jul. 2015. Google Scholar

22. Liu, W. and W. Q. Ning, "Research and application of face mask wear recognition based on transfer learning," Journal of Jilin Normal University (Natural Science Edition), Vol. 44, No. 1, 96-103, Feb. 2023. Google Scholar

23. Zhou, K. and M. Jiang, "Research progress and prospect of small sample target recognition based on transfer learning," Aeronautical Science and Technology, Vol. 34, No. 2, 1-9, Feb. 2023. Google Scholar

Dr. Xiaoyu Luo

Professor Qiusheng Li