Human Activity Recognition Technology Based on Intelligent Perception
-
摘要: 基于智能感知的人体活动识别(Human Activity Recognition,HAR)技术应用潜力巨大,尤其是在健康监测、智能运动和康复训练等领域。为了分析当前人体活动识别技术水平和未来发展方向,首先,阐述基于可穿戴传感器的智能传感技术;其次,归纳对比不同模态的公开数据集;再次,梳理智能算法研究现状,分析机器学习算法、深度学习算法和多模态算法在人体活动识别中的应用效果;最后,论述新型传感技术、感存算一体化架构及多模态方法等未来研究方向及主要挑战,如数据多样性、算法泛化能力和隐私保护等。Abstract: Human activity recognition(HAR) technology based on intelligent perception has shown great application potential,especially in the fields of health monitoring,intelligent sports and rehabilitation training.In order to analyze the current level and future development direction of human activity recognition technology,firstly,the intelligent sensing technology based on wearable sensor is described.Secondly,the open data sets of different modes are summarized and compared.Thirdly,the research status of existing intelligent algorithms is summarized,and the application effects of machine learning algorithms,deep learning algorithms and multi-modal algorithms in human activity recognition are analyzed.Finally,the future research direction and main challenges of new sensing technology,integrated sensor-memory and computing architecture,and multi-modal approach are discussed,such as data diversity,algorithm generalization ability and privacy protection.
-
-
[1] SERPUSH F, MENHAJ M B, MASOUMI B, et al.Wearable sensor-based human activity recognition in the smart healthcare system[J]. Computational Intelligence and Neuroscience, 2022(2):1391906.
[2] MEKRUKSAVANICH S, JITPATTANAKUL A.Biometric user identification based on human activity recognition using wearable sensors: an experiment using deep learning models[J]. Electronics, 2021, 10(3):308.
[3] LIU R, RAMLI A A, ZHANG H, et al.An overview of human activity recognition using wearable sensors: healthcare and artificial intelligence[C]. 2021 International Conference on Internet of Things-ICIOT, 2021.
[4] YADAV S K, TIWARI K, PANDEY H M, et al.A review of multimodal human activity recognition with special emphasis on classification, applications, challenges and future directions[J]. Knowledge-Based Systems, 2021(223):106970.
[5] ISLAM M M, NOORUDDIN S, KARRAY F, et al.Human activity recognition using tools of convolutional neural networks:a state of the art review, data sets, challenges, and future prospects[J]. Computers in Biology and Medicine, 2022(149):106060.
[6] SUN W B, GUO Z L, YANG Z Q, et al.A review of recent advances in vital signals monitoring of sports and health via flexible wearable sensors[J]. Sensors, 2022, 22(20):7784.
[7] WAN T Q, SHAO B J, MA S J, et al.In-sensor computing:materials, devices, and integration technologies[J]. Advanced Materials, 2023, 35(37):2203830.
[8] 吉司, 森塔戈泰.人体解剖图谱:骨骼, 关节, 韧带, 肌肉[M]. 北京:人民卫生出版社, 1959. [9] CHI H G, HA M H, CHI S, et al.InfoGCN:representation learning for human skeleton-based action recognition[C]. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
[10] SCATAGLINI S, TRUIJEN S.Overview of software and file exchange formats in 3D and 4D body shape scanning[C]. Proceedings of the 7th International Digital Human Modeling Symposium (DHM 2022), 2022.
[11] DANG L M, MIN K, WANG H X, et al.Sensor-based and vision-based human activity recognition:a comprehensive survey[J]. Pattern Recognition, 2020(108):107561.
[12] BANIQUED P D E, STANYER E C, AWAIS M, et al.Brain-computer interface robotics for hand rehabilitation after stroke:a systematic review[J]. Journal of Neuroengineering and Rehabilitation, 2021(18):15.
[13] TAO W, LI C, SONG R C, et al.EEG-based emotion recognition via channel-wise attention and self attention[J]. IEEE Transactions on Affective Computing, 2023, 14(1):382-393.
[14] HE C Y, CHEN Y Y, PHANG C R, et al.Diversity and suitability of the state-of-the-art wearable and wireless EEG systems review[J]. IEEE Journal of Biomedical and Health Informatics, 2023, 27(8):3830-3843.
[15] AHMAD N, GHAZILLA R A R, KHAIRI N M, et al.Reviews on various inertial measurement unit (IMU) sensor applications[J]. International Journal of Signal Processing Systems, 2013, 1(2):256-262.
[16] 秦永元.惯性导航[M]. 2版.北京:科学出版社, 2014. [17] PANKAJ, KUMAR A, KOMARAGIRI R, et al.A review on computation methods used in photoplethysmography signal analysis for heart rate estimation[J]. Archives of Computational Methods in Engineering, 2022(29):921-940.
[18] EL_RAHMAN S A.Biometric human recognition system based on ECG[J]. Multimedia Tools and Applications, 2019(78):17555-17572.
[19] ALMANIFI O R A, KHAIRUDDIN I M, RAZMAN M A M, et al.Human activity recognition based on wrist PPG via the ensemble method[J]. ICT Express, 2022, 8(4):513-517.
[20] RANI G J, HASHMI M F, GUPTA A.Surface electromyography and artificial intelligence for human activity recognition:a systematic review on methods, emerging trends applications, challenges, and future implementation[J]. IEEE Access, 2023(11):105140-105169.
[21] IBRAHIM A F T, GANNAPATHY V R, CHONG L W, et al.Analysis of electromyography(EMG) signal for human arm muscle: a review[C]. Advanced Computer and Communication Engineering Technology:Proceedings of ICOCOE 2015, 2015.
[22] ISLAM M R U, WARIS A, KAMAVUAKO E N, et al.A comparative study of motion detection with FMG and sEMG methods for assistive applications[J]. Journal of Rehabilitation and Assistive Technologies Engineering, 2020(7):2055668320938588.
[23] WANG C J, CAI M, HAO Z M, et al.Stretchable, multifunctional epidermal sensor patch for surface electromyography and strain measurements[J]. Advanced Intelligent Systems, 2021, 3(11):2100031.
[24] CASTILLO C S M, WILSON S, VAIDYANATHAN R, et al.Wearable MMG-plus-one armband:evaluation of normal force on mechanomyography (MMG) to enhance human-machine interfacing[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2021(29):196-205.
[25] JAMES K, ELDEMIRE-SHEARER D, GOULDBOURNE J, et al.Falls and fall prevention in the elderly:the jamaican perspective[J]. The West Indian Medical Journal, 2007, 56(6):534-539.
[26] PRAKASH C, KUMAR R, MITTAL N.Recent developments in human gait research:parameters, approaches, applications, machine learning techniques, datasets and challenges[J]. Artificial Intelligence Review, 2018(49):1-40.
[27] NI J Y, TANG H, HAQUE S T, et al.A survey on multimodal wearable sensor-based human action recognition[J]. arXiv, 2024(4):2404.15349.
[28] CHANG W N, DAI L L, SHENG S L, et al.A hierarchical hand motions recognition method based on IMU and sEMG sensors[C]. 2015 IEEE International Conference on Robotics and Biomimetics (ROBIO), 2015.
[29] JIANG S, GAO Q H, LIU H Y, et al.A novel, co-located EMG-FMG-sensing wearable armband for hand gesture recognition[J]. Sensors and Actuators A:Physical, 2020(301):111738.
[30] WANG H, KANG P Q, GAO Q H, et al.A novel PPG-FMG-ACC wristband for hand gesture recognition[J]. IEEE Journal of Biomedical and Health Informatics, 2022, 26(10):5097-5108.
[31] XIA W, ZHOU Y, YANG X C, et al.Toward portable hybrid surface electromyography/a-mode ultrasound sensing for human-machine interface[J]. IEEE Sensors Journal, 2019, 19(13):5219-5228.
[32] DIN I U, GUIZANI M, HASSAN S, et al.The internet of things: a review of enabled technologies and future challenges[J]. IEEE Access, 2019(7):7606-7640.
[33] ROGGEN D, CALATRONI A, ROSSI M, et al.Collecting complex activity datasets in highly rich networked sensor environments[C]. 2010 Seventh International Conference on Networked Sensing Systems (INSS), 2010.
[34] ZAPPI P, LOMBRISER C, STIEFMEIER T, et al.Activity recognition from on-body sensors:accuracy-power trade-off by dynamic sensor selection[C]. Wireless Sensor Networks:5th European Conference, 2008.
[35] ANGUITA D, GHIO A, ONETO L, et al.A public domain dataset for human activity recognition using smartphones[C]. ESANN 2013 Proceedings, 2013.
[36] REISS A, STRICKER D.Introducing a new benchmarked dataset for activity monitoring[C]. 201216th International Symposium on Wearable Computers, 2012.
[37] ZHANG M, SAWCHUK A A.USC-HAD:a daily activity dataset for ubiquitous activity recognition using wearable sensors[C]. Proceedings of the 2012 ACM Conference on Ubiquitous Computing, 2012.
[38] KWAPISZ J R, WEISS G M, MOORE S A.Activity recognition using cell phone accelerometers[J]. ACM SIGKDD Explorations Newsletter, 2011, 12(2):74-82.
[39] BARSHAN B, YVKSEK M C.Recognizing daily and sports activities in two open source machine learning environments using body-worn sensor units[J]. The Computer Journal, 2014, 57(11):1649-1667.
[40] HUYNH T, FRITZ M, SCHIELE B.Discovery of activity patterns using topic models[C]. Proceedings of the 10th International Conference on Ubiquitous Computing, 2008.
[41] WEISS G M, LOCKHART J W, PULICKAL T T, et al.Actitracker:a smartphone-based activity recognition system for improving health and well-being[C]. 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA), 2016.
[42] SHOAIB M, BOSCH S, INCEL O D, et al.Fusion of smartphone motion sensors for physical activity recognition[J]. Sensors, 2014, 14(6):10146-10176.
[43] BANOS O, GARCIA R, HOLGADO-TERRIZA J A, et al.mHealthDroid:a novel framework for agile development of mobile health applications[C]. Ambient Assisted Living and Daily Activities (IWAAL 2014), 2014.
[44] ROGGEN D, PLOTNIK M, HAUSDORFF J.UCI Machine Learning Repository:daphnet freezing of gait[Z]. 2010.
[45] RAVÌ D, WONG C, LO B, et al.A deep learning approach to on-node sensor data analytics for mobile or wearable devices[J]. IEEE Journal of Biomedical and Health Informatics, 2017, 21(1):56-64.
[46] KAWAGUCHI N, OGAWA N, IWASAKI Y, et al.HASC challenge:gathering large scale human activity corpus for the real-world activity understandings[C]. Proceedings of the 2nd Augmented Human International Conference, 2011.
[47] BULLING A, BLANKE U, SCHIELE B.A tutorial on human activity recognition using body-worn inertial sensors[J]. ACM Computing Surveys (CSUR), 2014, 46(3):1-33.
[48] IWAMA H, OKUMURA M, MAKIHARA Y, et al.The OU-ISIR gait database comprising the large population dataset and performance evaluation of gait recognition[J]. IEEE Transactions on Information Forensics and Security, 2012, 7(5):1511-1521.
[49] ZHANG Y, YANG F, FAN Q, et al.Research on sEMG-based gesture recognition by dual-view deep learning[J]. IEEE Access, 2022(10):32928-32937.
[50] XU Z Y, YU J X, XIANG W T, et al.A novel SE-CNN attention architecture for sEMG-based hand gesture recognition[J]. Computer Modeling in Engineering & Sciences, 2023, 134(1):157-177.
[51] DAI Q F, WONG Y K, KANKANHALI M, et al.Improved network and training scheme for cross-trial surface electromyography (sEMG)—based gesture recognition[J]. Bioengineering, 2023, 10(9):1101.
[52] KHUSHABA R N, KODAGODA S.Electromyogram(EMG) feature reduction using mutual components analysis for multifunction prosthetic fingers control[C]. 201212th International Conference on Control Automation Robotics & Vision (ICARCV), 2012.
[53] KHUSHABA R N, TAKRURI M, MIRO J V, et al.Towards limb position invariant myoelectric pattern recognition using time-dependent spectral features[J]. Neural Networks, 2014(55):42-58.
[54] SAPSANIS C, GEORGOULAS G, TZES A, et al.Improving EMG based classification of basic hand movements using EMD[C]. 201335th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2013.
[55] SAPSANIS C.Recognition of basic hand movements using electromyography[J]. arXiv, 2018(10):1810.10062.
[56] AL-TIMEMY A H, KHUSHABA R N, BUGMANN G, et al.Improving the performance against force variation of EMG controlled multifunctional upper-limb prostheses for transradial amputees[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2016, 24(6):650-661.
[57] THEODORIDIS T, AGAPITOS A, HU H S.A gaussian groundplan projection area model for evolving probabilistic classifiers[C]. Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation, 2011.
[58] LUAN Y, SHI Y H, WU W Y, et al.HAR-sEMG: a dataset for human activity recognition on lower-limb sEMG[J]. Knowledge and Information Systems, 2021(63):2791-2814.
[59] GU F Q, KHOSHELHAM K, VALAEE S, et al.Locomotion activity recognition using stacked denoising autoencoders[J]. IEEE Internet of Things Journal, 2018, 5(3):2085-2093.
[60] KHUSHABA R N, AL-ANI A, AL-TIMEMY A, et al.A fusion of time-domain descriptors for improved myoelectric hand control[C]. 2016 IEEE Symposium Series on Computational Intelligence (SSCI), 2016.
[61] AZEVEDO B F, ROCHA A M A C, PEREIRA A I.Hybrid approaches to optimization and machine learning methods: a systematic literature review[J]. Machine Learning, 2024(113):4055-4097.
[62] NASTESKI V.An overview of the supervised machine learning methods[J]. Horizons, 2017(4):51-62.
[63] ZHANG S B, LI Y X, ZHANG S, et al.Deep learning in human activity recognition with wearable sensors:a review on advances[J]. Sensors, 2022, 22(4):1476.
[64] ZHANG C, YANG Z C, HE X D, et al.Multimodal intelligence:representation learning, information fusion, and applications[J]. IEEE Journal of Selected Topics in Signal Processing, 2020, 14(3):478-493.
[65] WANG B Y.Data feature extraction method of wearable sensor based on convolutional neural network[J]. Journal of Healthcare Engineering, 2022(1):1580134.
[66] JAMEER S, SYED H.A DCNN-LSTM based human activity recognition by mobile and wearable sensor networks[J]. Alexandria Engineering Journal, 2023(80):542-552.
[67] SUH S, REY V F, LUKOWICZ P.TASKED:transformer-based adversarial learning for human activity recognition using wearable sensors via self-knowledge distillation[J]. Knowledge-Based Systems, 2023(260):110143.
[68] LAHAT D, ADALI T, JUTTEN C.Multimodal data fusion: an overview of methods, challenges, and prospects[J]. Proceedings of the IEEE, 2015, 103(9):1449-1477.
[69] YU Y, CHEN X, CAO S, et al.Exploration of Chinese sign language recognition using wearable sensors based on deep belief net[J]. IEEE Journal of Biomedical and Health Informatics, 2020, 24(5):1310-1320.
[70] HASSANI S, DACKERMANN U, MOUSAVI M, et al.A systematic review of data fusion techniques for optimized structural health monitoring[J]. Information Fusion, 2024(103):102136.
[71] LIU W J, DU Z J, DUAN Z Y, et al.Neuroprosthetic contact lens enabled sensorimotor system for point-of-care monitoring and feedback of intraocular pressure[J]. Nature Communications, 2024(15):5635.
[72] TEHRANI F, TEYMOURIAN H, WUERSTLE B, et al.An integrated wearable microneedle array for the continuous monitoring of multiple biomarkers in interstitial fluid[J]. Nature Biomedical Engineering, 2022(6):1214-1224.
[73] MIRANDA E, SUÑÉ J.Memristors for neuromorphic circuits and artificial intelligence applications[J]. Materials, 2020, 13(4):938.
[74] LIU J L, WANG Y T, LIU Y W, et al.Recent progress in wearable near-sensor and in-sensor intelligent perception systems [J]. Sensors, 2024, 24(7):2180.
[75] LIU X R, SUN C, GUO Z C, et al.Near-sensor reservoir computing for gait recognition via a multi-gate electrolyte-gated transistor[J]. Advanced Science, 2023, 10(15):2300471.
[76] JIANG C P, LIU J Q, YANG L, et al.A flexible artificial sensory nerve enabled by nanoparticle-assembled synaptic devices for neuromorphic tactile recognition[J]. Advanced Science, 2022, 9(24):2106124.
[77] SENGUPTA D, MASTELLA M, CHICCA E, et al.Skin-inspired flexible and stretchable electrospun carbon nanofiber sensors for neuromorphic sensing[J]. ACS Applied Electronic Materials, 2022, 4(1):308-315.
[78] GHOSH A, SADANA H R, DEBNATH M, et al.Approximate ADCs for in-memory computing[J]. arXiv, 2024(8):2408.06390.
[79] HUANG X H, LIU C S, TANG Z W, et al.An ultrafast bipolar flash memory for self-activated in-memory computing[J]. Nature Nanotechnology, 2023(18):486-492.
[80] NING H K, YU Z H, ZHANG Q T, et al.An in-memory computing architecture based on a duplex two-dimensional material structure for in situ machine learning[J]. Nature Nanotechnology, 2023(18):493-500.
[81] Peng Y X, Qi J W, Yuan Y X.Modality-specific cross-modal similarity measurement with recurrent attention network[J]. IEEE Transactions on Image Processing, 2018, 27(11):5585-5599.
[82] Chang Z Q, Liu S B, Xiong X X, et al.A survey of recent advances in edge-computing-powered artificial intelligence of things[J]. IEEE Internet of Things Journal, 2021, 8(18):13849-13875.
计量
- 文章访问数: 419
- HTML全文浏览量: 0
- PDF下载量: 16
下载:
