Simplified multi-view graph neural network for multilingual knowledge graph completion
Bingbing DONG1,2, Chenyang BU1,2, Yi ZHU1,2,3, Shengwei JI4, Xindong WU1,2()
1. Key Laboratory of Knowledge Engineering with Big Data (the Ministry of Education of China),Hefei University of Technology, Hefei 230009, China 2. School of Computer Science and Information Engineering, Hefei University of Technology, Hefei 230009, China 3. School of Information Engineering, Yangzhou University, Yangzhou 225127, China 4. School of Artificial Intelligence and Big Data, Hefei University, Hefei 230000, China
Knowledge graph completion (KGC) aims to fill in missing entities and relations within knowledge graphs (KGs) to address their incompleteness. Most existing KGC models suffer from knowledge coverage as they are designed to operate within a single KG. In contrast, Multilingual KGC (MKGC) leverages seed pairs from different language KGs to facilitate knowledge transfer and enhance the completion of the target KG. Previous studies on MKGC based on graph neural networks (GNNs) have primarily focused on using relation-aware GNNs to capture the combined features of neighboring entities and relations. However, these studies still have some shortcomings, particularly in the context of MKGCs. First, each language’s specific semantics, structures, and expressions contribute to the increased heterogeneity of the KG. Therefore, the completion of MKGCs necessitates a thorough consideration of the heterogeneity of the KG and the effective integration of its heterogeneous features. Second, MKGCs typically have a large graph scale due to the need to store and manage information from multiple languages. However, current relation-aware GNNs often inherit complex GNN operations, resulting in unnecessary complexity. Therefore, it is necessary to simplify GNN operations. To address these limitations, we propose a Simplified Multi-view Graph Neural Network (SM-GNN) for MKGC. SM-GNN incorporates two simplified multi-view GNNs as components. One GNN is utilized for learning multi-view graph features to complete the KG. The other generates new alignment pairs, facilitating knowledge transfer between different views of the KG. We simplify the two multi-view GNNs by retaining feature propagation while discarding linear transformation and nonlinear activation to reduce unnecessary complexity and effectively leverage graph contextual information. Extensive experiments demonstrate that our proposed model outperforms competing baselines. The code and dataset are available at the website of github.com/dbbice/SM-GNN.
J, Yan C, Wang W, Cheng M, Gao A Zhou . A retrospective of knowledge graphs. Frontiers of Computer Science, 2018, 12( 1): 55–74
2
S, Ji S, Pan E, Cambria P, Marttinen P S Yu . A survey on knowledge graphs: representation, acquisition, and applications. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33( 2): 494–514
3
Y, Liu K, Zhang Z, Huang K, Wang Y, Zhang Q, Liu E Chen . Enhancing hierarchical text classification through knowledge graph integration. In: Proceedings of the Association for Computational Linguistics. 2023, 5797−5810
4
Wu X, Jiang T, Zhu Y, Bu C. Knowledge graph for China’s genealogy. In: Proceedings of 2020 IEEE International Conference on Knowledge Graph. 2020, 529−535
5
Bu C, Zhang J, Yu X, Wu L, Wu X. Which companies are likely to invest: knowledge-graph-based recommendation for investment promotion. In: Proceedings of 2022 IEEE International Conference on Data Mining. 2022, 11−20
6
Q, Lin R, Mao J, Liu F, Xu E Cambria . Fusing topology contexts and logical rules in language models for knowledge graph completion. Information Fusion, 2023, 90: 253–264
7
D, Xu T, Xu S, Wu J, Zhou E Chen . Relation-enhanced negative sampling for multimodal knowledge graph completion. In: Proceedings of the 30th ACM International Conference on Multimedia. 2022, 3857−3866
8
F, Liu Y, Shen T, Zhang H Gao . Entity-related paths modeling for knowledge base completion. Frontiers of Computer Science, 2020, 14( 5): 145311
9
S, Choudhary T, Luthra A, Mittal R Singh . A survey of knowledge graph embedding and their applications. 2021, arXiv preprint arXiv: 2107.07842
10
A, Rossi D, Barbosa D, Firmani A, Matinata P Merialdo . Knowledge graph embedding for link prediction: a comparative analysis. ACM Transactions on Knowledge Discovery from Data, 2021, 15( 2): 14
11
A, Bordes N, Usunier A, Garcia-Durán J, Weston O Yakhnenko . Translating embeddings for modeling multi-relational data. In: Proceedings of the 26th International Conference on Neural Information Processing Systems. 2013, 2787−2795
12
Z, Sun Z-H, Deng J-Y, Nie J Tang . RotatE: knowledge graph embedding by relational rotation in complex space. In: Proceedings of the 7th International Conference on Learning Representations. 2019
13
R, Li Y, Cao Q, Zhu G, Bi F, Fang Y, Liu Q Li . How does knowledge graph embedding extrapolate to unseen data: a semantic evidence view. In: Proceedings of the 36th AAAI Conference on Artificial Intelligence. 2022, 5781−5791
14
Q, Wang Z, Mao B, Wang L Guo . Knowledge graph embedding: a survey of approaches and applications. IEEE Transactions on Knowledge and Data Engineering, 2017, 29( 12): 2724–2743
15
X, Chen M, Chen C, Fan A, Uppunda Y, Sun C Zaniolo . Multilingual knowledge graph completion via ensemble knowledge transfer. In: Proceedings of the Association for Computational Linguistics. 2020, 3227−3238
16
Z, Huang Z, Li H, Jiang T, Cao H, Lu B, Yin K, Subbian Y, Sun W Wang . Multilingual knowledge graph completion with self-supervised adaptive graph alignment. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. 2022, 474−485
17
V, Tong D Q, Nguyen T T, Huynh T T, Nguyen Q V H, Nguyen M Niepert . Joint multilingual knowledge graph completion and alignment. In: Proceedings of the Association for Computational Linguistics. 2022, 4646−4658
18
Z, Li Q, Zhang F, Zhu D, Li C, Zheng Y Zhang . Knowledge graph representation learning with simplifying hierarchical feature propagation. Information Processing & Management, 2023, 60( 4): 103348
19
T, Dettmers P, Minervini P, Stenetorp S Riedel . Convolutional 2D knowledge graph embeddings. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018, 1811−1818
20
Z, Yao W, Zhang M, Chen Y, Huang Y, Yang H Chen . Analogical inference enhanced knowledge graph embedding. In: Proceedings of the 37th AAAI Conference on Artificial Intelligence. 2023, 4801−4808
21
T, Le N, Le B Le . Knowledge graph embedding by relational rotation and complex convolution for link prediction. Expert Systems with Applications, 2023, 214: 119122
22
N, Fanourakis V, Efthymiou D, Kotzinos V Christophides . Knowledge graph embedding methods for entity alignment: experimental review. Data Mining and Knowledge Discovery, 2023, 37( 5): 2070–2137
23
C, Liu L, Li X, Yao L Tang . A survey of recommendation algorithms based on knowledge graph embedding. In: Proceedings of the 2019 IEEE International Conference on Computer Science and Educational Informatization. 2019, 168−171
24
Z, Shokrzadeh M R, Feizi-Derakhshi M A, Balafar J B Mohasefi . Knowledge graph-based recommendation system enhanced by neural collaborative filtering and knowledge graph embedding. Ain Shams Engineering Journal, 2024, 15( 1): 102263
25
Z, Wang J, Zhang J, Feng Z Chen . Knowledge graph embedding by translating on hyperplanes. In: Proceedings of the 28th AAAI Conference on Artificial Intelligence. 2014, 1112−1119
26
Y, Lin Z, Liu M, Sun Y, Liu X Zhu . Learning entity and relation embeddings for knowledge graph completion. In: Proceedings of the 29th AAAI Conference on Artificial Intelligence. 2015, 2181−2187
27
Yao L, Mao C S, Luo Y. KG-BERT: Bert for knowledge graph completion. 2019, arXiv preprint arXiv:1909.03193
28
L, Yao C, Mao Y Luo . Graph convolutional networks for text classification. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 2019, 7370−7377
29
X, He K, Deng X, Wang Y, Li Y D, Zhang M Wang . LightGCN: simplifying and powering graph convolution network for recommendation. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2020, 639−648
30
K L, Mao J M, Zhu X, Xiao B, Lu Z W, Wang X Q He . UltraGCN: Ultra simplification of graph convolutional networks for recommendation. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 2021, 1253−1262
31
M, Schlichtkrull T N, Kipf P, Bloem Den Berg R, Van I, Titov M Welling . Modeling relational data with graph convolutional networks. In: Proceedings of the 15th International Conference on the Semantic Web. 2018, 593−607
32
S, Vashishth S, Sanyal V, Nitin P P Talukdar . Composition-based multi-relational graph convolutional networks. In: Proceedings of the 8th International Conference on Learning Representations. 2020
33
F, Wu A H Jr, Souza T, Zhang C, Fifty T, Yu K Q Weinberger . Simplifying graph convolutional networks. In: Proceedings of the 36th International Conference on Machine Learning. 2019, 6861−6871
34
H, Wang S, Dai W, Su H, Zhong Z, Fang Z, Huang S, Feng Z, Chen Y, Sun D Yu . Simple and effective relation-based embedding propagation for knowledge representation learning. In: Proceedings of the 31st International Joint Conference on Artificial Intelligence. 2022, 2755−2761
35
T, Jiang C, Bu Y, Zhu X Wu . Integrating symbol similarities with knowledge graph embedding for entity alignment: an unsupervised framework. Intelligent Computing, 2023, 2: 0021
36
T T, Jiang C Y, Bu Y, Zhu X D Wu . Combining embedding-based and symbol-based methods for entity alignment. Pattern Recognition, 2022, 124: 108433
37
T T, Jiang C Y, Bu Y, Zhu X D Wu . Two-stage entity alignment: combining hybrid knowledge graph embedding with similarity-based relation alignment. In: Proceedings of the 16th Pacific Rim International Conference on Artificial Intelligence. 2019, 162−175
38
H, Zhu R, Xie Z, Liu M Sun . Iterative entity alignment via joint knowledge embeddings. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence. 2017, 4258−4264
39
J, Zhu C, Huang Meo P De . DFMKE: a dual fusion multi-modal knowledge graph embedding framework for entity alignment. Information Fusion, 2023, 90: 111–119
40
M, Chen Y, Tian M, Yang C Zaniolo . Multilingual knowledge graph embeddings for cross-lingual knowledge alignment. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence. 2017, 1511−1517
41
Z, Sun W, Hu Q, Zhang Y Qu . Bootstrapping entity alignment with knowledge graph embedding. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence. 2018, 4396−4402
42
Wang Z, Lv Q, Lan X, Zhang Y. Cross-lingual knowledge graph alignment via graph convolutional networks. In: Proceedings of 2018 Conference on Empirical Methods in Natural Language Processing. 2018, 349−357
43
H, Singh S, Chakrabarti P, Jain S R, Choudhury Mausam. Multilingual knowledge graph completion with joint relation and entity alignment. In: Proceedings of the 3rd Conference on Automated Knowledge Base Construction. 2021
44
M U, Akhtar J, Liu Z, Xie X, Liu S, Ahmed B Huang . Entity alignment based on relational semantics augmentation for multilingual knowledge graphs. Knowledge-Based Systems, 2022, 252: 109494
45
Y, Wu X, Liu Y, Feng Z, Wang R, Yan D Zhao . Relation-aware entity alignment for heterogeneous knowledge graphs. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence. 2019, 5278−5284
46
B, Yang W T, Yih X, He J, Gao L Deng . Embedding entities and relations for learning and inference in knowledge bases. In: Proceedings of the 3rd International Conference on Learning Representations. 2015
47
Devlin J, Chang M-W, Lee K, Toutanova K. BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019, 4171−4186
48
J Demšar . Statistical comparisons of classifiers over multiple data sets. The Journal of Machine Learning Research, 2006, 7: 1–30
49
X, Mao W, Wang Y, Wu M Lan . Are negative samples necessary in entity alignment?: an approach with high performance, scalability and robustness. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 2021, 1263−1273