A probabilistic generative model for tracking multi-knowledge concept mastery probability

doi:10.1007/s11704-023-3008-x

Front. Comput. Sci.

2024, Vol. 18

Issue (3) : 183602 https://doi.org/10.1007/s11704-023-3008-x

RESEARCH ARTICLE

A probabilistic generative model for tracking multi-knowledge concept mastery probability

Hengyu LIU¹, Tiancheng ZHANG¹(

), Fan LI¹, Minghe YU², Ge YU¹

¹. School of Computer Science and Engineering, Northeastern University, Shenyang 110169, China
². Software College, Northeastern University, Shenyang 110169, China

Download: PDF(10173 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

Knowledge tracing aims to track students’ knowledge status over time to predict students’ future performance accurately. In a real environment, teachers expect knowledge tracing models to provide the interpretable result of knowledge status. Markov chain-based knowledge tracing (MCKT) models, such as Bayesian Knowledge Tracing, can track knowledge concept mastery probability over time. However, as the number of tracked knowledge concepts increases, the time complexity of MCKT predicting student performance increases exponentially (also called explaining away problem). When the number of tracked knowledge concepts is large, we cannot utilize MCKT to track knowledge concept mastery probability over time. In addition, the existing MCKT models only consider the relationship between students’ knowledge status and problems when modeling students’ responses but ignore the relationship between knowledge concepts in the same problem. To address these challenges, we propose an inTerpretable pRobAbilistiC gEnerative moDel (TRACED), which can track students’ numerous knowledge concepts mastery probabilities over time. To solve explain away problem, we design long and short-term memory (LSTM)-based networks to approximate the posterior distribution, predict students’ future performance, and propose a heuristic algorithm to train LSTMs and probabilistic graphical model jointly. To better model students’ exercise responses, we proposed a logarithmic linear model with three interactive strategies, which models students’ exercise responses by considering the relationship among students’ knowledge status, knowledge concept, and problems. We conduct experiments with four real-world datasets in three knowledge-driven tasks. The experimental results show that TRACED outperforms existing knowledge tracing methods in predicting students’ future performance and can learn the relationship among students, knowledge concepts, and problems from students’ exercise sequences. We also conduct several case studies. The case studies show that TRACED exhibits excellent interpretability and thus has the potential for personalized automatic feedback in the real-world educational environment.

Keywords probabilistic graphical model deep learning knowledge tracing learner modeling

Corresponding Author(s): Tiancheng ZHANG

Just Accepted Date: 20 February 2023 Issue Date: 26 April 2023

Cite this article:

Hengyu LIU,Tiancheng ZHANG,Fan LI, et al. A probabilistic generative model for tracking multi-knowledge concept mastery probability[J]. Front. Comput. Sci., 2024, 18(3): 183602.

URL:

https://academic.hep.com.cn/fcs/EN/10.1007/s11704-023-3008-x
https://academic.hep.com.cn/fcs/EN/Y2024/V18/I3/183602

Fig.1 A toy example of the knowledge tracking task

Fig.2 The framework of the TRACED model

	Notation	Description
Dataset description	N	The total number of students
	M	The total number of problems
	K	The total number of knowledge concepts
	$S i$	Student $i$ ’s exercise record
	$r i$	The response result sequence of student $i$ ’s exercise record
	$e i$	The problem sequence of student $i$ ’s exercise record
	$τ i$	The time sequence of student $i$ ’s exercise record
	$Q j, k$	Problem $j$ contains knowledge concept $k$ or not
Model parameters	$E e, j$	The distributed representation of problem $j$
	$E c, k$	The distributed representation of knowledge concept $k$
	$π k$	The probability that students initially master the knowledge concept $k$
	$θ s, j$	Problem $j$ ’s slipping parameters
	$θ g, j$	Problem $j$ ’s guessing parameters
	$θ l, k$	Knowledge concept $k$ ’s learning parameters
	$θ f, k$	Knowledge concept $k$ ’s forgetting parameters
	$b l, k$	Knowledge concept $k$ ’s learning bias
	$b f, k$	Knowledge concept $k$ ’s forgetting bias
	$w e, j$	Problem $j$ ’s bias
	$w c, k$	Knowledge concept $k$ ’s bias
	$Z ∗ ∗, b ∗$	The parameters in LSTM
Random variable	$u i, k t$	Student $i$ masters knowledge concept $k$ or not at the $t$ -th exercise record
	$s j$	Students made a mistake on problem $j$
	$g j$	Students answer problem $j$ by guessing
	$f k$	Students forget knowledge concept $k$
	$l k$	Students master knowledge concept $k$ through learning
Hyperparameter	$d e$	The dimension of distributed representation
	$d h$	The dimension of hidden state in the LSTM which approximates posterior distribution
	$d p$	The dimension of hidden state in the LSTM which predicts students' future performance
	$Δ τ^$	The time interval for calculating knowledge concepts exercise frequency

Tab.1 Key notations in IKT

Fig.3 Graphical representation of TRACED

Tab.2 Statistics of the datasets

Tab.3 Results for predicting future student performance on the HDU and POJ datasets

Tab.4 Results for predicting future student performance on the algebra06 and algebra08 datasets

Tab.5 Results of predicting relationships between concepts

Tab.6 Results of predicting concepts of problems

Fig.4 The loss values of TRACED. (a) The loss in wake phase; (b) the loss in sleep phase

Fig.5 The visualization of prior and posterior of TRACED on the HDU and POJ datasets. (a) HDU; (b) POJ

Fig.6 Visualization of the learned distributed representations of students, knowledge concept and problem for the HDU dataset, where the learned representations have been reduced to 2 dimensions by means of PCA

1	Q, Liu S, Tong C, Liu H, Zhao E, Chen H, Ma S Wang . Exploiting cognitive structure for adaptive learning. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2019, 627−635
2	R, Wu Q, Liu Y, Liu E, Chen Y, Su Z, Chen G Hu . Cognitive modelling for predicting examinee performance. In: Proceedings of the 24th International Conference on Artificial Intelligence. 2015, 1017−1024
3	F, Ai Y, Chen Y, Guo Y, Zhao Z, Wang G, Fu G Wang . Concept-aware deep knowledge tracing and exercise recommendation in an online learning system. In: Proceedings of the 12th International Conference on Educational Data Mining. 2019
4	C, Piech J, Bassen J, Huang S, Ganguli M, Sahami L Guibas . Deep knowledge tracing. In: Proceedings of the 28th International Conference on Neural Information Processing Systems. 2015, 505−513
5	Y, Chen Q, Liu Z, Huang L, Wu E, Chen R, Wu Y, Su G Hu . Tracking knowledge proficiency of students with educational priors. In: Proceedings of 2017 ACM on Conference on Information and Knowledge Management. 2017, 989−998
6	S, Sun X, Hu C, Bu F, Liu Y, Zhang W Luo . Genetic algorithm for Bayesian knowledge tracing: a practical application. In: Proceedings of the 13th International Conference on Sensing and Imaging. 2022, 282−293
7	T L, Wong D, Zou G, Cheng J K T, Tang Y, Cai F L Wang . Enhancing skill prediction through generalising Bayesian knowledge tracing. International Journal of Mobile Learning and Organisation, 2021, 15( 4): 358–373
8	F, Liu X, Hu C, Bu K Yu . Fuzzy Bayesian knowledge tracing. IEEE Transactions on Fuzzy Systems, 2022, 30( 7): 2412–2425
9	K, Zhang Y Yao . A three learning states Bayesian knowledge tracing model. Knowledge-Based Systems, 2018, 148: 189–201
10	M P, Wellman M Henrion . Explaining ’explaining away’. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1993, 15( 3): 287–292
11	der Linden W J, van R K Hambleton . Handbook of Modern Item Response Theory. New York: Springer, 2013
12	H, Cen K, Koedinger B Junker . Learning factors analysis–A general method for cognitive model evaluation and improvement. In: Proceedings of the 8th International Conference on Intelligent Tutoring Systems. 2006, 164−175
13	Jr P I, Pavlik H, Cen K R Koedinger . Performance factors analysis–A new alternative to knowledge tracing. In: Proceedings of the Artificial Intelligence in Education: Building Learning Systems that Care: From Knowledge Representation to Affective Modelling, Proceedings of the 14th International Conference on Artificial Intelligence in Education. 2009
14	Vie J J, Kashima H. Knowledge tracing machines: factorization machines for knowledge tracing. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence, AAAI 2019, the 31st Innovative Applications of Artificial Intelligence Conference, IAAI 2019, the 9th AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019. 2019, 750−757
15	B, Choffin F, Popineau Y, Bourda J J Vie . DAS3H: modeling student learning and forgetting for optimally scheduling distributed practice of skills. In: Proceedings of the 12th International Conference on Educational Data Mining. 2019
16	A T, Corbett J R Anderson . Knowledge tracing: modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, 1994, 4( 4): 253–278
17	G, Gorgun O Bulut . Considering disengaged responses in Bayesian and deep knowledge tracing. In: Proceedings of the 23rd International Conference on Artificial Intelligence in Education. 2022, 591−594
18	la Torre J de . Dina model and parameter estimation: a didactic. Journal of Educational and Behavioral Statistics, 2009, 34( 1): 115–130
19	H, Liu T, Zhang F, Li Y, Gu G Yu . Tracking knowledge structures and proficiencies of students with learning transfer. IEEE Access, 2021, 9: 55413–55421
20	D P Ausubel . Educational Psychology: A Cognitive View. New York: Holt, Rinehart and Winston, 1968
21	Su Y, Liu Q, Liu Q, Huang Z, Yin Y, Chen E, Ding C, Wei S, Hu G. Exercise-enhanced sequential modeling for student performance prediction. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence and 30th Innovative Applications of Artificial Intelligence Conference and 8th AAAI Symposium on Educational Advances in Artificial Intelligence. 2018, 297
22	Q, Liu Z, Huang Y, Yin E, Chen H, Xiong Y, Su G Hu . EKT: exercise-aware knowledge tracing for student performance prediction. IEEE Transactions on Knowledge and Data Engineering, 2021, 33( 1): 100–115
23	Y, Lu D, Wang Q, Meng P Chen . Towards interpretable deep learning models for knowledge tracing. In: Proceedings of the 21st International Conference on Artificial Intelligence in Education. 2020, 185−190
24	J, Zhang X, Shi I, King D Y Yeung . Dynamic key-value memory networks for knowledge tracing. In: Proceedings of the 26th International Conference on World Wide Web. 2017, 765−774
25	X, Sun X, Zhao Y, Ma X, Yuan F, He J Feng . Muti-behavior features based knowledge tracking using decision tree improved DKVMN. In: Proceedings of the ACM Turing Celebration Conference-China. 2019, 72
26	S, Pandey G Karypis . A self attentive model for knowledge tracing. In: Proceedings of the 12th International Conference on Educational Data Mining. 2019
27	S, Pandey J Srivastava . RKT: relation-aware self-attention for knowledge tracing. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2020, 1205−1214
28	A, Ghosh N, Heffernan A S Lan . Context-aware attentive knowledge tracing. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2020, 2330−2339
29	J, Zhu W, Yu Z, Zheng C, Huang Y, Tang G P C Fung . Learning from interpretable analysis: Attention-based knowledge tracing. In: Proceedings of the 21st International Conference on Artificial Intelligence in Education. 2020, 364−368
30	M, Yu F, Li H, Liu T, Zhang G Yu . ContextKT: a context-based method for knowledge tracing. Applied Sciences, 2022, 12( 17): 8822
31	Heng-yu L I U, Tian-cheng Z, Pei-wen W U, Ge Y U. A review of knowledge tracking. Journal of East China Normal University (Natural Science), 2019, 2019(5): 1
32	M J, Anzanello F S Fogliatto . Learning curve models and applications: literature review and research directions. International Journal of Industrial Ergonomics, 2011, 41(5): 573−583
33	von Foerster H. Understanding Understanding: Essays on Cybernetics and Cognition. Springer Science & Business Media, 2007
34	L R Rabiner . A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 1989, 77( 2): 257–286
35	G E, Hinton P, Dayan B J, Frey R M Neal . The “wake-sleep” algorithm for unsupervised neural networks. Science, 1995, 268( 5214): 1158–1161
36	J, Fogarty R S, Baker S E Hudson . Case studies in the use of ROC curve analysis for sensor-based estimates in human computer interaction. In: Proceedings of Graphics Interface 2005. 2005, 129−136
37	H, Liu Q, Fu L, Du T, Zhang G, Yu S, Han D Zhang . Learning rate perturbation: a generic plugin of learning rate schedule towards flatter local minima. In: Proceedings of the 31st ACM International Conference on Information & Knowledge Management. 2022, 4234−4238
38	S E, Embretson S P Reise . Item Response Theory. New York: Psychology Press, 2013
39	R V, Lindsey J D, Shroyer H, Pashler M C Mozer . Improving students’ long-term knowledge retention through personalized review. Psychological Science, 2014, 25( 3): 639–647
40	M C, Mozer R V Lindsey . Predicting and improving memory retention: psychological theory matters in the big data era. In: Jones M N, ed. Big Data in Cognitive Science. London: Routledge, Taylor & Francis Group, 2017, 34−64
41	Glorot X, Bengio Y. Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the 13th International Conference on Artificial Intelligence and Statistics. 2010, 249−256
42	T, Mikolov I, Sutskever K, Chen G, Corrado J Dean . Distributed representations of words and phrases and their compositionality. In: Proceedings of the 26th International Conference on Neural Information Processing Systems. 2013, 3111−3119

[1]

FCS-23008-OF-HL_suppl_1

Download

[1]	Mingzhen LI, Changxi LIU, Jianjin LIAO, Xuegui ZHENG, Hailong YANG, Rujun SUN, Jun XU, Lin GAN, Guangwen YANG, Zhongzhi LUAN, Depei QIAN. Towards optimized tensor code generation for deep learning on sunway many-core processor[J]. Front. Comput. Sci., 2024, 18(2): 182101-.
[2]	Mingzhi YUAN, Kexue FU, Zhihao LI, Manning WANG. Decoupled deep hough voting for point cloud registration[J]. Front. Comput. Sci., 2024, 18(2): 182703-.
[3]	Hanadi AL-MEKHLAFI, Shiguang LIU. Single image super-resolution: a comprehensive review and recent insight[J]. Front. Comput. Sci., 2024, 18(1): 181702-.
[4]	Yufei ZENG, Zhixin LI, Zhenbin CHEN, Huifang MA. Aspect-level sentiment analysis based on semantic heterogeneous graph convolutional network[J]. Front. Comput. Sci., 2023, 17(6): 176340-.
[5]	Yamin HU, Hao JIANG, Zongyao HU. Measuring code maintainability with deep neural networks[J]. Front. Comput. Sci., 2023, 17(6): 176214-.
[6]	Muazzam MAQSOOD, Sadaf YASMIN, Saira GILLANI, Maryam BUKHARI, Seungmin RHO, Sang-Soo YEO. An efficient deep learning-assisted person re-identification solution for intelligent video surveillance in smart cities[J]. Front. Comput. Sci., 2023, 17(4): 174329-.
[7]	Tian WANG, Jiakun LI, Huai-Ning WU, Ce LI, Hichem SNOUSSI, Yang WU. ResLNet: deep residual LSTM network with longer input for action recognition[J]. Front. Comput. Sci., 2022, 16(6): 166334-.
[8]	Donghong HAN, Yanru KONG, Jiayi HAN, Guoren WANG. A survey of music emotion recognition[J]. Front. Comput. Sci., 2022, 16(6): 166335-.
[9]	Yi WEI, Mei XUE, Xin LIU, Pengxiang XU. Data fusing and joint training for learning with noisy labels[J]. Front. Comput. Sci., 2022, 16(6): 166338-.
[10]	Pinzhuo TIAN, Yang GAO. Improving meta-learning model via meta-contrastive loss[J]. Front. Comput. Sci., 2022, 16(5): 165331-.
[11]	Tian WANG, Shiye LEI, Youyou JIANG, Choi CHANG, Hichem SNOUSSI, Guangcun SHAN, Yao FU. Accelerating temporal action proposal generation via high performance computing[J]. Front. Comput. Sci., 2022, 16(4): 164317-.
[12]	Kaimin WEI, Tianqi LI, Feiran HUANG, Jinpeng CHEN, Zefan HE. Cancer classification with data augmentation based on generative adversarial networks[J]. Front. Comput. Sci., 2022, 16(2): 162601-.
[13]	Yu OU, Lang LI. Side-channel analysis attacks based on deep learning network[J]. Front. Comput. Sci., 2022, 16(2): 162303-.
[14]	Qiang LIN, Yusheng HAO, Caihong LIU. Wi-Fi based non-invasive detection of indoor wandering using LSTM model[J]. Front. Comput. Sci., 2021, 15(6): 156505-.
[15]	Anirban DUTTA, Gudmalwar ASHISHKUMAR, Ch V Rama RAO. Performance analysis of ASR system in hybrid DNN-HMM framework using a PWL euclidean activation function[J]. Front. Comput. Sci., 2021, 15(4): 154705-.

Viewed

Full text

Abstract

Cited

Shared

Discussed