1. School of Computer Science, Northwestern Polytechnical University, Xi’an 710129, China 2. MIIT Lab of Big Data Storage and Management, Xi’an 710129, China
Learning-outcome prediction (LOP) is a long-standing and critical problem in educational routes. Many studies have contributed to developing effective models while often suffering from data shortage and low generalization to various institutions due to the privacy-protection issue. To this end, this study proposes a distributed grade prediction model, dubbed FecMap, by exploiting the federated learning (FL) framework that preserves the private data of local clients and communicates with others through a global generalized model. FecMap considers local subspace learning (LSL), which explicitly learns the local features against the global features, and multi-layer privacy protection (MPP), which hierarchically protects the private features, including model-shareable features and not-allowably shared features, to achieve client-specific classifiers of high performance on LOP per institution. FecMap is then achieved in an iteration manner with all datasets distributed on clients by training a local neural network composed of a global part, a local part, and a classification head in clients and averaging the global parts from clients on the server. To evaluate the FecMap model, we collected three higher-educational datasets of student academic records from engineering majors. Experiment results manifest that FecMap benefits from the proposed LSL and MPP and achieves steady performance on the task of LOP, compared with the state-of-the-art models. This study makes a fresh attempt at the use of federated learning in the learning-analytical task, potentially paving the way to facilitating personalized education with privacy protection.
Y, Zhang R, An S, Liu J, Cui X Shang . Predicting and understanding student learning performance using multi-source sparse attention convolutional neural networks. IEEE Transactions on Big Data, 2023, 9( 1): 118–132
2
Y, Zhang H, Dai Y, Yun S, Liu A, Lan X Shang . Meta-knowledge dictionary learning on 1-bit response data for student knowledge diagnosis. Knowledge-Based Systems, 2020, 205: 106290
3
P, Symeonidis D Malakoudis . Multi-modal matrix factorization with side information for recommending massive open online courses. Expert Systems with Applications, 2019, 118: 261–271
4
Y, Zhang Y, Yun H, Dai J, Cui X Shang . Graphs regularized robust matrix factorization and its application on student grade prediction. Applied Sciences, 2020, 10( 5): 1755
5
H Bydžovská . Student performance prediction using collaborative filtering methods. In: Proceedings of the 17th International Conference on Artificial Intelligence in Education. 2015, 550−553
6
Al-Shehri H, Al-Qarni A, Al-Saati L, Batoaq A, Badukhen H, Alrashed S, Alhiyafi J, Olatunji S O. Student performance prediction using support vector machine and k-nearest neighbor. In: Proceedings of the 30th IEEE Canadian Conference on Electrical and Computer Engineering. 2017, 1−4
7
A, Polyzou G Karypis . Feature extraction for next-term prediction of poor student performance. IEEE Transactions on Learning Technologies, 2019, 12( 2): 237–248
8
Y, Zhang Y, Yun R, An J, Cui H, Dai X Shang . Educational data mining techniques for student performance prediction: method review and comparison analysis. Frontiers in Psychology, 2021, 12: 698490
9
T, Li S, Hu A, Beirami V Smith . Ditto: Fair and robust federated learning through personalization. In: Proceedings of the 38th International Conference on Machine Learning. 2021, 6357−6368
10
T, Li A K, Sahu M, Zaheer M, Sanjabi A, Talwalkar V Smith . Federated optimization in heterogeneous networks. In: Proceedings of Machine Learning and Systems. 2020, 429−450
11
L, Collins H, Hassani A, Mokhtari S Shakkottai . Exploiting shared representations for personalized federated learning. In: Proceedings of the 38th International Conference on Machine Learning. 2021, 2089−2099
12
F, Haddadpour M Mahdavi . On the convergence of local descent methods in federated learning. 2019, arXiv preprint arXiv: 1910.14425
13
T, Li A K, Sahu A, Talwalkar V Smith . Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine, 2020, 37( 3): 50–60
14
Q, Li Z, Wen Z, Wu S, Hu N, Wang Y, Li X, Liu B He . A survey on federated learning systems: vision, hype and reality for data privacy and protection. IEEE Transactions on Knowledge and Data Engineering, 2023, 35( 4): 3347–3366
15
Tan A Z, Yu H, Cui L, Yang Q. Towards personalized federated learning. IEEE Transactions on Neural Networks and Learning Systems, 2022, 1–17
16
Y, Li X, Liu X, Zhang Y, Shao Q, Wang Y Geng . Personalized federated learning via maximizing correlation with sparse and hierarchical extensions. 2021, arXiv preprint arXiv: 2107.05330
17
B, McMahan E, Moore D, Ramage S, Hampson B A Y Arcas . Communication-efficient learning of deep networks from decentralized data. In: Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. 2017, 1273−1282
18
Y R, Chen A, Rezapour W G Tzeng . Privacy-preserving ridge regression on distributed data. Information Sciences, 2018, 451-452: 34–49
19
D K, Dennis T, Li V Smith . Heterogeneity for the win: One-shot federated clustering. In: Proceedings of the 38th International Conference on Machine Learning. 2021, 2611−2620
20
M, Ribero J, Henderson S, Williamson H Vikalo . Federating recommendations using differentially private prototypes. Pattern Recognition, 2022, 129: 108746
21
P, Zhou K, Wang L, Guo S, Gong B Zheng . A privacy-preserving distributed contextual federated online learning framework with big data support in social recommender systems. IEEE Transactions on Knowledge and Data Engineering, 2021, 33( 3): 824–838
22
Y, Zhang Y, Xu S, Wei Y, Wang Y, Li X Shang . Doubly contrastive representation learning for federated image recognition. Pattern Recognition, 2023, 139: 109507
23
X, Ma J, Zhang S, Guo W Xu . Layer-wised model aggregation for personalized federated learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022, 10082–10091
24
X C, Li D C, Zhan Y, Shao B, Li S Song . FedPHP: Federated personalization with inherited private models. In: Proceedings of Joint European Conference on Machine Learning and Knowledge Discovery in Databases. 2021, 587−602
25
X, Zhang Y, Li W, Li K, Guo Y Shao . Personalized federated learning via variational bayesian inference. In: Proceedings of International Conference on Machine Learning. 2022, 26293−26310
26
V, Smith C K, Chiang M, Sanjabi A Talwalkar . Federated multi-task learning. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 4427–4437
27
M, Duan D, Liu X, Chen R, Liu Y, Tan L Liang . Self-balancing federated learning with global imbalanced data in mobile systems. IEEE Transactions on Parallel and Distributed Systems, 2021, 32( 1): 59–71
28
C I, Bercea B, Wiestler D, Rueckert S Albarqouni . Federated disentangled representation learning for unsupervised brain anomaly detection. Nature Machine Intelligence, 2022, 4( 8): 685–695
29
Q, Wu X, Chen Z, Zhou J Zhang . FedHome: Cloud-edge based personalized federated learning for in-home health monitoring. IEEE Transactions on Mobile Computing, 2022, 21( 8): 2818–2832
30
N, Wang Y, Chen Y, Hu W, Lou Y T Hou . FeCo: Boosting intrusion detection capability in IoT networks via contrastive learning. In: Proceedings of IEEE INFOCOM 2022-IEEE Conference on Computer Communications. 2022, 1409−1418
31
G, Long M, Xie T, Shen T, Zhou X, Wang J Jiang . Multi-center federated learning: clients clustering for better personalization. World Wide Web, 2023, 26( 1): 481–500
32
F, Sattler K R, Müller W Samek . Clustered federated learning: Model-agnostic distributed multitask optimization under privacy constraints. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32( 8): 3710–3722
33
X, Li M, Jiang X, Zhang M, Kamp Q Dou . FedBN: Federated learning on non-IID features via local batch normalization. In: Proceedings of the 9th International Conference on Learning Representations. 2021
34
Y, Zhang S, Wei S, Liu Y, Wang Y, Xu Y, Li X Shang . Graph-regularized federated learning with shareable side information. Knowledge-Based Systems, 2022, 257: 109960
35
L, Yang J, Huang W, Lin J Cao . Personalized federated learning on non-IID data via group-based meta-learning. ACM Transactions on Knowledge Discovery from Data, 2023, 17( 4): 49
36
K A, Bonawitz H, Eichner W, Grieskamp D, Huba A, Ingerman V, Ivanov C, Kiddon J, Konecný S, Mazzocchi B, McMahan Overveldt T, Van D, Petrou D, Ramage J Roselander . Towards federated learning at scale: System design. In: Proceedings of Machine Learning and Systems. 2019, 374−388
37
X C, Li L, Gan D C, Zhan Y, Shao B, Li S Song . Aggregate or not? Exploring where to privatize in DNN based federated learning under different non-IID scenes. 2021, arXiv preprint arXiv: 2107.11954
38
Liang P P, Liu T, Ziyin L, Allen N B, Auerbach R P, Brent D, Salakhutdinov R, Morency L P. Think locally, act globally: federated learning with local and global representations. 2020, arXiv preprint arXiv: 2001.01523
39
M G, Arivazhagan V, Aggarwal A K, Singh S Choudhary . Federated learning with personalization layers. 2019, arXiv preprint arXiv: 1912.00818
40
der Maaten L, van G Hinton . Visualizing data using t-SNE. Journal of Machine Learning Research, 2008, 9( 86): 2579–2605
