MiTAR: a study on human activity recognition based on NLP with microscopic perspective
Huichao MEN1, Botao WANG1(), Gang WU1,2
1. School of Computer Science and Engineering, Northeastern University, Shenyang 110000, China 2. State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210000, China
Nowadays, human activity recognition is becoming a more and more significant topic, and there is also a wide range of applications for it in real world scenarios. Sensor data is an important data source in engineering and application. At present, some studies have been carried out in the field of human activity recognition based on sensor data in a macroscopic perspective. However, many studies in this perspective face some limitations. One pivotal limitation is uncontrollable data segment length of different kinds of activities. Suitable feature and data form are also influencing factors. This paper carries out the study creatively on a microscopic perspective with an emphasis on the logic and relevance between data segments, attempting to apply the idea of natural language processing and the method of data symbolization to the study of human activity recognition and try to solve the problem above. In this paper, several activity-element definitions and three algorithms are proposed, including the algorithm of dictionary building, the algorithm of corpus building, and activity recognition algorithm improved from a natural language analysis method, TFIDF. Numerous experiments on different aspects of this model are taken. The experiments are carried out on six complex and representative single-level sensor datasets, namely UCI Sports and Daily dataset, Skoda dataset, WISDM Phoneacc dataset, WISDM Watchacc dataset, Healthy Older People dataset and HAPT dataset, which prove that this model can be applied to different datasets and achieve a satisfactory recognition result.
. [J]. Frontiers of Computer Science, 2021, 15(5): 155330.
Huichao MEN, Botao WANG, Gang WU. MiTAR: a study on human activity recognition based on NLP with microscopic perspective. Front. Comput. Sci., 2021, 15(5): 155330.
W Lin, M Yang, J Wu, L Ke, H Xiong. Action recognition with coarse-tofine deep feature integration and asynchronous fusion. In: Proceedings of the National Conference on Artificial Intelligence. 2018, 1–8
2
R Franco, Q Facundo, L Laura, E Cesar. Distribution of action movements dam a descriptor for human action recognition. Frontiers of Computer Science, 2015, 9(6): 956–965 https://doi.org/10.1007/s11704-015-4320-x
3
K Chen, G Ding, J Han. Attribute-based supervised deep learning model for action recognition. Frontiers of Computer Science, 2017, 11(2): 219–229 https://doi.org/10.1007/s11704-016-6066-5
4
J Wang, D Chen, J Yang. Human behavior classification by analyzing periodic motions. Frontiers of Computer Science, 2010, 4(4): 580–587 https://doi.org/10.1007/s11704-009-0070-y
5
A Bracciali, E Larsson. Data-intensive modelling and simulation in life sciences and socio-economical and physical sciences. Data Science and Engineering, 2017, 2(3): 197–198 https://doi.org/10.1007/s41019-017-0049-x
6
W Pan, Z Li, Y Zhang, C Weng. The new hardware development trend and the challenges in data management and analysis. Data Science and Engineering, 2018, 3(3): 263–276 https://doi.org/10.1007/s41019-018-0072-6
7
H Wu, W Pan, X Xiong, S Xu. Human activity recognition based on the combined svm&hmm. In: Proceedings of IEEE International Conference on Information & Automation. 2014, 219–224 https://doi.org/10.1109/ICInfA.2014.6932656
8
D Anguita, A Ghio, L Oneto, X Parra, J L Reyes-Ortiz. Human activity recognition on smartphones using a multiclass hardware-friendly support vector machine. In: Proceedings of International Conference on Ambient Assisted Living & Home Care. 2012, 216–223 https://doi.org/10.1007/978-3-642-35395-6_30
9
R Krishnan, M Subedar, O Tickoo. Bar: Bayesian activity recognition using variational inference. In: Proceedings of the 3rd Workshop on Bayesian Deep Learning. 2018, 1–8
10
V S Dave, B Zhang, P Chen, M A Hasan. Neural-brane: neural Bayesian personalized ranking for attributed network embedding. Data Science and Engineering, 2019, 4(2): 119–131 https://doi.org/10.1007/s41019-019-0092-x
11
M Yuan, E Chen, G Lei. Posture selection based on two-layer AP with application to human action recognition using HMM. In: Proceedings of IEEE International Symposium on Multimedia. 2017, 359–364 https://doi.org/10.1109/ISM.2016.0079
12
N Ranjan, K Mundada, K Phaltane, S Ahmad. A survey on techniques in NLP. International Journal of Computer Applications, 2016, 134: 6–9 https://doi.org/10.5120/ijca2016907355
13
K Altun, B Barshan, O Tunçel. Comparative study on classifying human activities with miniature inertial and magnetic sensors. Pattern Recognition, 2010, 43(10): 3605–3620 https://doi.org/10.1016/j.patcog.2010.04.019
14
P Zappi, T Stiefmeier, E Farella, D Roggen, L Benini, G Troster. Activity recognition from on-body sensors by classifier fusion: sensor scalability and robustness. In: Proceedings of International Conference on Intelligent Sensors. 2007, 281–286 https://doi.org/10.1109/ISSNIP.2007.4496857
15
D Roggen, A Calatroni, M Rossi, T Holleczek, K Förster, G Tröster, P Lukowicz, D Bannach, G Pirkl, A Ferscha. Collecting complex activity data sets in highly rich networked sensor environments. In: Proceedings of the 7th International Conference on Networked Sensing Systems. 2010, 233–240 https://doi.org/10.1109/INSS.2010.5573462
16
R Chavarriaga, H Sagha, A Calatroni, S Digumarti, G Tröster, J D R Millán, D Roggen. The opportunity challenge: a benchmark database for on-body sensor-based activity recognition. Pattern Recognition Letters, 2013, 34(15): 2033–2042 https://doi.org/10.1016/j.patrec.2012.12.014
17
X Xie. Human action recognition in the range of Wi-Fi with CNN and ELM. Master Thesis, Beijing, University of Posts and Telecommunication, 2018
18
R Khanna, M Awad. Efficient Learning Machines: Theories, Concepts, and Applications for Engineers and System Designers. Berkeley California: Apress, 2015
19
P Bharti, D De, S Chellappan, S K Das. Human: complex activity recognition with multi-modal multi-positional body sensing. IEEE Transactions on Mobile Computing, 2018, 18(4): 857–870 https://doi.org/10.1109/TMC.2018.2841905
20
A Stolke, S Omohundro. Hidden markrov model induction by Bayesian model merging. In: Proceedings of the 5th International Conference on Neural Information Processing Systems. 1992, 11–18
21
MK Adil, L Young-Koo, S Y Lee, K Tae-Seong. A triaxial accelerometerbased physical-activity recognition via augmented-signal features and a hierarchical recognizer. IEEE Transactions on Information Technology in Biomedicine, 2010, 14(5): 1166–1172 https://doi.org/10.1109/TITB.2010.2051955
22
K Greff, R K Srivastava, J Koutnik, B R Steunebrink, J Schmidhuber. LSTM: a search space odyssey. IEEE Transactions on Neural Networks & Learning Systems, 2016, 28(10): 2222–2232 https://doi.org/10.1109/TNNLS.2016.2582924
23
H Qi, K Fang, X Wu, L Xu, Q Lang. Human activity recognition method based on molecular attributes. International Journal of Distributed Sensor Networks, 2019, 15(4): 1–13 https://doi.org/10.1177/1550147719842729
24
V Ashish, S Noam, P Niki, U Jakob, J Llion, N G Aidan, K Lukasz, P Illia. Attention is all you need. In: Proceedings of Annual Conference on Neural Information Processing Systems. 2017, 5998–6008
25
J Devlin, M Chang, K Lee, K Toutanova. Bert: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019, 4171–4186
26
B Barshan, M C Yüksek. Recognizing daily and sports activities in two open source machine learning environments using body-worn sensor units. Computer Journal, 2013, 57(11): 1649–1667 https://doi.org/10.1093/comjnl/bxt075
27
R Brena, E Garcia-Ceja. A crowdsourcing approach for personalization in human activities recognition. Intelligent Data Analysis, 2017, 21: 721–738 https://doi.org/10.3233/IDA-170884
28
X Wang, L Wang, L Lopes. Unsupervised categorization of human motion sequences. Intelligent Data Analysis, 2013, 17(6): 1057–1074 https://doi.org/10.3233/IDA-130620
29
X Wang, B Zhang, G Teng, Z Sun, J Wei. Toward robust activity recognition: hierarchical classifier based on gaussian process. Intelligent Data Analysis, 2016, 20(3): 701–717 https://doi.org/10.3233/IDA-160827
E L Khair, A Ibrahim. TF*IDF. Boston: Springer US, 2009
32
S Mika, B Schölkopf, A Smola, K R Müller, G Rätsch. Kernel PCA and De-noising in Feature Spaces. In: Proceedings of the 12th Annual Conference on Neural Information Processing Systems II. 1999, 536–542
33
K Altun, B Barshan. Human activity recognition using inertial/magnetic sensor units. Lecture Notes in Computer Science, 2010, 6219: 38–51 https://doi.org/10.1007/978-3-642-14715-9_5
34
P Zappi, C Lombriser, T Stiefmeier, E Farella, D Roggen, L Benini, G Tröster. Activity Recognition from On-Body Sensors: Accuracy-Power Trade-Off by Dynamic Sensor Selection. Berlin: Springer Berlin Heidelberg, 2008
35
D Roggen, G Troster. Fusion of string-matched templates forcontinuous activity recognition. In: Proceedings of IEEE International Symposium on Wearable Computers. 2007, 1–4
36
M W Gary, Y Kenichi, H Thaier. Smartphone and smartwatch-based biometrics using activities of daily living. IEEE Access, 2019, 7: 133190–133202 https://doi.org/10.1109/ACCESS.2019.2940729
37
A Wickramasinghe, D C Ranasinghe, C Fumeaux, K D Hill, R Visvanathan. Sequence learning with passive rfid sensors for real-time bed-egress recognition in older people. IEEE Journal of Biomedical & Health Informatics, 2017, 21(4): 917–929 https://doi.org/10.1109/JBHI.2016.2576285
38
S T Roberto, V Renuka, H Stephen, V D H Anton, R Damith. Effectiveness of a batteryless and wireless wearable sensor system for identifying bed and chair exits in healthy older people. Sensors, 2016, 16(4): 546–562 https://doi.org/10.3390/s16040546
39
A Wickramasinghe, D C Ranasinghe. Recognisingactivities in real time using body worn passive sensors with sparse data streams: to interpolate or not to interpolate? In: Proceedings of the 12th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services. 2015, 21–30 https://doi.org/10.4108/eai.22-7-2015.2260068
40
S T Roberto, R Damith, Q Shi. Evaluation of wearable sensor tag data segmentation approaches for real time activity classification in elderly. Springer International Publishing, 2014, 131: 384–395 https://doi.org/10.1007/978-3-319-11569-6_30
41
A P Sample, S T Roberto, D C Ranasinghe, Q Shi. Sensor enabled wearable rfid technology for mitigating the risk of falls near beds. In: Proceed ings of IEEE International Conference on RFID. 2013, 191–198
42
J L Reyes-Ortiz , L Oneto, A Sama, X Parra, D Anguita. Transition-aware human activity recognition using smartphones. Neurocomputing, 2016, 171: 754–767 https://doi.org/10.1016/j.neucom.2015.07.085
43
D Trabelsi, S Mohammed, Y Amirat, L Oukhellou. Activity recognition using body mounted sensors: an unsupervised learning based approach. In: Proceedings of International Joint Conference on Neural Networks. 2012, 1–7 https://doi.org/10.1109/IJCNN.2012.6252819
44
A Subasi. Eeg signal classification using wavelet feature extraction and a mixture of expert model. Expert Systems with Applications, 2007, 32(4): 1084–1093 https://doi.org/10.1016/j.eswa.2006.02.005
45
B Li, D Aleksandr, Y Gue, T Liu, M Satoshi, X Du. Scaling word2vec on big corpus. Data Science and Engineering, 2019, 4(2): 157–175 https://doi.org/10.1007/s41019-019-0096-6
