A probabilistic generative model for tracking multi-knowledge concept mastery probability

doi:10.1007/s11704-023-3008-x

Frontiers of Computer Science

2024, Vol. 18

Issue (3): 183602 https://doi.org/10.1007/s11704-023-3008-x

本期目录

A probabilistic generative model for tracking multi-knowledge concept mastery probability

Hengyu LIU¹, Tiancheng ZHANG¹(

), Fan LI¹, Minghe YU², Ge YU¹

¹. School of Computer Science and Engineering, Northeastern University, Shenyang 110169, China
². Software College, Northeastern University, Shenyang 110169, China

全文: PDF(10173 KB) HTML

Abstract：

Knowledge tracing aims to track students’ knowledge status over time to predict students’ future performance accurately. In a real environment, teachers expect knowledge tracing models to provide the interpretable result of knowledge status. Markov chain-based knowledge tracing (MCKT) models, such as Bayesian Knowledge Tracing, can track knowledge concept mastery probability over time. However, as the number of tracked knowledge concepts increases, the time complexity of MCKT predicting student performance increases exponentially (also called explaining away problem). When the number of tracked knowledge concepts is large, we cannot utilize MCKT to track knowledge concept mastery probability over time. In addition, the existing MCKT models only consider the relationship between students’ knowledge status and problems when modeling students’ responses but ignore the relationship between knowledge concepts in the same problem. To address these challenges, we propose an inTerpretable pRobAbilistiC gEnerative moDel (TRACED), which can track students’ numerous knowledge concepts mastery probabilities over time. To solve explain away problem, we design long and short-term memory (LSTM)-based networks to approximate the posterior distribution, predict students’ future performance, and propose a heuristic algorithm to train LSTMs and probabilistic graphical model jointly. To better model students’ exercise responses, we proposed a logarithmic linear model with three interactive strategies, which models students’ exercise responses by considering the relationship among students’ knowledge status, knowledge concept, and problems. We conduct experiments with four real-world datasets in three knowledge-driven tasks. The experimental results show that TRACED outperforms existing knowledge tracing methods in predicting students’ future performance and can learn the relationship among students, knowledge concepts, and problems from students’ exercise sequences. We also conduct several case studies. The case studies show that TRACED exhibits excellent interpretability and thus has the potential for personalized automatic feedback in the real-world educational environment.

Key words： probabilistic graphical model deep learning knowledge tracing learner modeling

收稿日期: 2023-01-04 出版日期: 2023-04-26

Corresponding Author(s): Tiancheng ZHANG

引用本文:

. [J]. Frontiers of Computer Science, 2024, 18(3): 183602.
Hengyu LIU, Tiancheng ZHANG, Fan LI, Minghe YU, Ge YU. A probabilistic generative model for tracking multi-knowledge concept mastery probability. Front. Comput. Sci., 2024, 18(3): 183602.

链接本文:

https://academic.hep.com.cn/fcs/CN/10.1007/s11704-023-3008-x
https://academic.hep.com.cn/fcs/CN/Y2024/V18/I3/183602

Fig.1

Fig.2

	Notation	Description
Dataset description	N	The total number of students
	M	The total number of problems
	K	The total number of knowledge concepts
	$S i$	Student $i$ ’s exercise record
	$r i$	The response result sequence of student $i$ ’s exercise record
	$e i$	The problem sequence of student $i$ ’s exercise record
	$τ i$	The time sequence of student $i$ ’s exercise record
	$Q j, k$	Problem $j$ contains knowledge concept $k$ or not
Model parameters	$E e, j$	The distributed representation of problem $j$
	$E c, k$	The distributed representation of knowledge concept $k$
	$π k$	The probability that students initially master the knowledge concept $k$
	$θ s, j$	Problem $j$ ’s slipping parameters
	$θ g, j$	Problem $j$ ’s guessing parameters
	$θ l, k$	Knowledge concept $k$ ’s learning parameters
	$θ f, k$	Knowledge concept $k$ ’s forgetting parameters
	$b l, k$	Knowledge concept $k$ ’s learning bias
	$b f, k$	Knowledge concept $k$ ’s forgetting bias
	$w e, j$	Problem $j$ ’s bias
	$w c, k$	Knowledge concept $k$ ’s bias
	$Z ∗ ∗, b ∗$	The parameters in LSTM
Random variable	$u i, k t$	Student $i$ masters knowledge concept $k$ or not at the $t$ -th exercise record
	$s j$	Students made a mistake on problem $j$
	$g j$	Students answer problem $j$ by guessing
	$f k$	Students forget knowledge concept $k$
	$l k$	Students master knowledge concept $k$ through learning
Hyperparameter	$d e$	The dimension of distributed representation
	$d h$	The dimension of hidden state in the LSTM which approximates posterior distribution
	$d p$	The dimension of hidden state in the LSTM which predicts students' future performance
	$Δ τ^$	The time interval for calculating knowledge concepts exercise frequency

Tab.1

Fig.3

Tab.2

Tab.3

Tab.4

Tab.5

Tab.6

Fig.4

Fig.5

Fig.6

1	Q, Liu S, Tong C, Liu H, Zhao E, Chen H, Ma S Wang . Exploiting cognitive structure for adaptive learning. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2019, 627−635
2	R, Wu Q, Liu Y, Liu E, Chen Y, Su Z, Chen G Hu . Cognitive modelling for predicting examinee performance. In: Proceedings of the 24th International Conference on Artificial Intelligence. 2015, 1017−1024
3	F, Ai Y, Chen Y, Guo Y, Zhao Z, Wang G, Fu G Wang . Concept-aware deep knowledge tracing and exercise recommendation in an online learning system. In: Proceedings of the 12th International Conference on Educational Data Mining. 2019
4	C, Piech J, Bassen J, Huang S, Ganguli M, Sahami L Guibas . Deep knowledge tracing. In: Proceedings of the 28th International Conference on Neural Information Processing Systems. 2015, 505−513
5	Y, Chen Q, Liu Z, Huang L, Wu E, Chen R, Wu Y, Su G Hu . Tracking knowledge proficiency of students with educational priors. In: Proceedings of 2017 ACM on Conference on Information and Knowledge Management. 2017, 989−998
6	S, Sun X, Hu C, Bu F, Liu Y, Zhang W Luo . Genetic algorithm for Bayesian knowledge tracing: a practical application. In: Proceedings of the 13th International Conference on Sensing and Imaging. 2022, 282−293
7	T L, Wong D, Zou G, Cheng J K T, Tang Y, Cai F L Wang . Enhancing skill prediction through generalising Bayesian knowledge tracing. International Journal of Mobile Learning and Organisation, 2021, 15( 4): 358–373
8	F, Liu X, Hu C, Bu K Yu . Fuzzy Bayesian knowledge tracing. IEEE Transactions on Fuzzy Systems, 2022, 30( 7): 2412–2425
9	K, Zhang Y Yao . A three learning states Bayesian knowledge tracing model. Knowledge-Based Systems, 2018, 148: 189–201
10	M P, Wellman M Henrion . Explaining ’explaining away’. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1993, 15( 3): 287–292
11	der Linden W J, van R K Hambleton . Handbook of Modern Item Response Theory. New York: Springer, 2013
12	H, Cen K, Koedinger B Junker . Learning factors analysis–A general method for cognitive model evaluation and improvement. In: Proceedings of the 8th International Conference on Intelligent Tutoring Systems. 2006, 164−175
13	Jr P I, Pavlik H, Cen K R Koedinger . Performance factors analysis–A new alternative to knowledge tracing. In: Proceedings of the Artificial Intelligence in Education: Building Learning Systems that Care: From Knowledge Representation to Affective Modelling, Proceedings of the 14th International Conference on Artificial Intelligence in Education. 2009
14	Vie J J, Kashima H. Knowledge tracing machines: factorization machines for knowledge tracing. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence, AAAI 2019, the 31st Innovative Applications of Artificial Intelligence Conference, IAAI 2019, the 9th AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019. 2019, 750−757
15	B, Choffin F, Popineau Y, Bourda J J Vie . DAS3H: modeling student learning and forgetting for optimally scheduling distributed practice of skills. In: Proceedings of the 12th International Conference on Educational Data Mining. 2019
16	A T, Corbett J R Anderson . Knowledge tracing: modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, 1994, 4( 4): 253–278
17	G, Gorgun O Bulut . Considering disengaged responses in Bayesian and deep knowledge tracing. In: Proceedings of the 23rd International Conference on Artificial Intelligence in Education. 2022, 591−594
18	la Torre J de . Dina model and parameter estimation: a didactic. Journal of Educational and Behavioral Statistics, 2009, 34( 1): 115–130
19	H, Liu T, Zhang F, Li Y, Gu G Yu . Tracking knowledge structures and proficiencies of students with learning transfer. IEEE Access, 2021, 9: 55413–55421
20	D P Ausubel . Educational Psychology: A Cognitive View. New York: Holt, Rinehart and Winston, 1968
21	Su Y, Liu Q, Liu Q, Huang Z, Yin Y, Chen E, Ding C, Wei S, Hu G. Exercise-enhanced sequential modeling for student performance prediction. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence and 30th Innovative Applications of Artificial Intelligence Conference and 8th AAAI Symposium on Educational Advances in Artificial Intelligence. 2018, 297
22	Q, Liu Z, Huang Y, Yin E, Chen H, Xiong Y, Su G Hu . EKT: exercise-aware knowledge tracing for student performance prediction. IEEE Transactions on Knowledge and Data Engineering, 2021, 33( 1): 100–115
23	Y, Lu D, Wang Q, Meng P Chen . Towards interpretable deep learning models for knowledge tracing. In: Proceedings of the 21st International Conference on Artificial Intelligence in Education. 2020, 185−190
24	J, Zhang X, Shi I, King D Y Yeung . Dynamic key-value memory networks for knowledge tracing. In: Proceedings of the 26th International Conference on World Wide Web. 2017, 765−774
25	X, Sun X, Zhao Y, Ma X, Yuan F, He J Feng . Muti-behavior features based knowledge tracking using decision tree improved DKVMN. In: Proceedings of the ACM Turing Celebration Conference-China. 2019, 72
26	S, Pandey G Karypis . A self attentive model for knowledge tracing. In: Proceedings of the 12th International Conference on Educational Data Mining. 2019
27	S, Pandey J Srivastava . RKT: relation-aware self-attention for knowledge tracing. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2020, 1205−1214
28	A, Ghosh N, Heffernan A S Lan . Context-aware attentive knowledge tracing. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2020, 2330−2339
29	J, Zhu W, Yu Z, Zheng C, Huang Y, Tang G P C Fung . Learning from interpretable analysis: Attention-based knowledge tracing. In: Proceedings of the 21st International Conference on Artificial Intelligence in Education. 2020, 364−368
30	M, Yu F, Li H, Liu T, Zhang G Yu . ContextKT: a context-based method for knowledge tracing. Applied Sciences, 2022, 12( 17): 8822
31	Heng-yu L I U, Tian-cheng Z, Pei-wen W U, Ge Y U. A review of knowledge tracking. Journal of East China Normal University (Natural Science), 2019, 2019(5): 1
32	M J, Anzanello F S Fogliatto . Learning curve models and applications: literature review and research directions. International Journal of Industrial Ergonomics, 2011, 41(5): 573−583
33	von Foerster H. Understanding Understanding: Essays on Cybernetics and Cognition. Springer Science & Business Media, 2007
34	L R Rabiner . A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 1989, 77( 2): 257–286
35	G E, Hinton P, Dayan B J, Frey R M Neal . The “wake-sleep” algorithm for unsupervised neural networks. Science, 1995, 268( 5214): 1158–1161
36	J, Fogarty R S, Baker S E Hudson . Case studies in the use of ROC curve analysis for sensor-based estimates in human computer interaction. In: Proceedings of Graphics Interface 2005. 2005, 129−136
37	H, Liu Q, Fu L, Du T, Zhang G, Yu S, Han D Zhang . Learning rate perturbation: a generic plugin of learning rate schedule towards flatter local minima. In: Proceedings of the 31st ACM International Conference on Information & Knowledge Management. 2022, 4234−4238
38	S E, Embretson S P Reise . Item Response Theory. New York: Psychology Press, 2013
39	R V, Lindsey J D, Shroyer H, Pashler M C Mozer . Improving students’ long-term knowledge retention through personalized review. Psychological Science, 2014, 25( 3): 639–647
40	M C, Mozer R V Lindsey . Predicting and improving memory retention: psychological theory matters in the big data era. In: Jones M N, ed. Big Data in Cognitive Science. London: Routledge, Taylor & Francis Group, 2017, 34−64
41	Glorot X, Bengio Y. Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the 13th International Conference on Artificial Intelligence and Statistics. 2010, 249−256
42	T, Mikolov I, Sutskever K, Chen G, Corrado J Dean . Distributed representations of words and phrases and their compositionality. In: Proceedings of the 26th International Conference on Neural Information Processing Systems. 2013, 3111−3119

[1]

FCS-23008-OF-HL_suppl_1

Download

Viewed

Full text

Abstract

Cited

Shared

Discussed