Semantic-aware entity alignment for low resource language knowledge graph
Junfei TANG1,2, Ran SONG1,2, Yuxin HUANG1,2, Shengxiang GAO1,2, Zhengtao YU1,2()
1. Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China 2. Yunnan Key Laboratory of Artificial Intelligence, Kunming University of Science and Technology, Kunming 650500, China
Entity alignment (EA) is an important technique aiming to find the same real entity between two different source knowledge graphs (KGs). Current methods typically learn the embedding of entities for EA from the structure of KGs for EA. Most EA models are designed for rich-resource languages, requiring sufficient resources such as a parallel corpus and pre-trained language models. However, low-resource language KGs have received less attention, and current models demonstrate poor performance on those low-resource KGs. Recently, researchers have fused relation information and attributes for entity representations to enhance the entity alignment performance, but the relation semantics are often ignored. To address these issues, we propose a novel Semantic-aware Graph Neural Network (SGNN) for entity alignment. First, we generate pseudo sentences according to the relation triples and produce representations using pre-trained models. Second, our approach explores semantic information from the connected relations by a graph neural network. Our model captures expanded feature information from KGs. Experimental results using three low-resource languages demonstrate that our proposed SGNN approach out performs better than state-of-the-art alignment methods on three proposed datasets and three public datasets.
B, Yang T Mitchell . Leveraging knowledge bases in LSTMs for improving machine reading. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2019, 1436−1446
2
Y, Cao L, Hou J, Li Z Liu . Neural collective entity linking. In: Proceedings of the 27th International Conference on Computational Linguistics. 2018, 675−686
3
Yang Z, Qi P, Zhang S, Bengio Y, Cohen W, Salakhutdinov R, Manning C D. HotpotQA: a dataset for diverse, explainable multi-hop question answering. In: Proceedings of 2018 Conference on Empirical Methods in Natural Language Processing. 2018, 2369−2380
4
M, Franco-Salvador P, Rosso M Montes-y-Gómez . A systematic study of knowledge graph analysis for cross-language plagiarism detection. Information Processing & Management, 2016, 52( 4): 550–570
5
J, Lehmann R, Isele M, Jakob A, Jentzsch D, Kontokostas P N, Mendes S, Hellmann M, Morsey Kleef P, van S, Auer C Bizer . DBpedia–A large-scale, multilingual knowledge base extracted from Wikipedia. Semantic Web, 2015, 6( 2): 167–195
6
F M, Suchanek G, Kasneci G Weikum . Yago: a core of semantic knowledge. In: Proceedings of the 16th International Conference on World Wide Web. 2007, 697−706
7
R, Navigli S P Ponzetto . BabelNet: the automatic construction, evaluation and application of a wide-coverage multilingual semantic network. Artificial Intelligence, 2012, 193: 217–250
8
M, Chen Y, Tian M, Yang C Zaniolo . Multilingual knowledge graph embeddings for cross-lingual knowledge alignment. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence. 2017, 1511−1517
9
Z, Sun W, Hu C Li . Cross-lingual entity alignment via joint attribute-preserving embedding. In: Proceedings of the 16th International Semantic Web Conference. 2017, 628−644
10
P, Veličković G, Cucurull A, Casanova A, Romero P, Liò Y Bengio . Graph attention networks. In: Proceedings of the ICLR 2018. 2018
11
A, Conneau K, Khandelwal N, Goyal V, Chaudhary G, Wenzek F, Guzmán E, Grave M, Ott L, Zettlemoyer V Stoyanov . Unsupervised cross-lingual representation learning at scale. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020, 8440−8451
12
Petroni F, Rocktäschel T, Riedel S, Lewis P, Bakhtin A, Wu Y, Miller A H, Riedel S. Language models as knowledge bases? In: Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019, 2463−2473
13
Jiang Z, Anastasopoulos A, Araki J, Ding H, Neubig G. X-FACTR: multilingual factual knowledge retrieval from pretrained language models. In: Proceedings of 2020 Conference on Empirical Methods in Natural Language Processing. 2020, 5943−5959
14
A, Sanchez-Gonzalez N, Heess J T, Springenberg J, Merel M A, Riedmiller R, Hadsell P W Battaglia . Graph networks as learnable physics engines for inference and control. In: Proceedings of the 35th International Conference on Machine Learning. 2018, 4467−4476
15
Y, Wu D, Lian Y, Xu L, Wu E Chen . Graph convolutional networks with Markov random field reasoning for social spammer detection. In: Proceedings of the AAAI Conference on Artificial Intelligence. 2020, 1054−1061
16
A, Fout J, Byrd B, Shariat A Ben-Hur . Protein interface prediction using graph convolutional networks. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 6533−6542
17
H, Dai E B, Khalil Y, Zhang B, Dilkina L Song . Learning combinatorial optimization algorithms over graphs. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 6351−6361
18
T N, Kipf M Welling . Semi-supervised classification with graph convolutional networks. In: Proceedings of the 5th International Conference on Learning Representations. 2017
19
Y, Li D, Tarlow M, Brockschmidt R S Zemel . Gated graph sequence neural networks. In: Proceedings of the 4th International Conference on Learning Representations. 2016
20
K, Xu W, Hu J, Leskovec S Jegelka . How powerful are graph neural networks? In: Proceedings of the 7th International Conference on Learning Representations. 2019
21
M, Defferrard X, Bresson P Vandergheynst . Convolutional neural networks on graphs with fast localized spectral filtering. In: Proceedings of the 30th International Conference on Neural Information Processing Systems. 2016, 3844−3852
22
W L, Hamilton R, Ying J Leskovec . Inductive representation learning on large graphs. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 1025−1035
23
A, Bordes N, Usunier A, Garcia-Durán J, Weston O Yakhnenko . Translating embeddings for modeling multi-relational data. In: Proceedings of the 26th International Conference on Neural Information Processing Systems. 2013, 2787−2795
24
S, Ji S, Pan E, Cambria P, Marttinen P S Yu . A survey on knowledge graphs: representation, acquisition, and applications. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33( 2): 494–514
25
Wang Z, Zhang J, Feng J, Chen Z. Knowledge graph embedding by translating on hyperplanes. In: Proceedings of the 28th AAAI Conference on Artificial Intelligence. 2014, 1112−1119
26
Lin Y, Liu Z, Sun M, Liu Y, Zhu X. Learning entity and relation embeddings for knowledge graph completion. In: Proceedings of the 29th AAAI Conference on Artificial Intelligence. 2015, 2181−2187
27
H, Zhu R, Xie Z, Liu M Sun . Iterative entity alignment via joint knowledge embeddings. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence. 2017, 4258−4264
28
Z, Sun W, Hu Q, Zhang Y Qu . Bootstrapping entity alignment with knowledge graph embedding. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence. 2018, 4396−4402
29
Q, Zhu X, Zhou J, Wu J, Tan L Guo . Neighborhood-aware attentional representation for multilingual knowledge graphs. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence. 2019, 1943−1949
30
Wang Z, Lv Q, Lan X, Zhang Y. Cross-lingual knowledge graph alignment via graph convolutional networks. In: Proceedings of 2018 Conference on Empirical Methods in Natural Language Processing. 2018, 349−357
31
Y, Cao Z, Liu C, Li Z, Liu J, Li T S Chua . Multi-channel graph neural network for entity alignment. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019, 1452−1461
32
K, Xu L, Wang M, Yu Y, Feng Y, Song Z, Wang D Yu . Cross-lingual knowledge graph alignment via graph matching neural network. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019, 3156−3161
33
Y, Wu X, Liu Y, Feng Z, Wang D Zhao . Neighborhood matching network for entity alignment. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020, 6477−6487
34
Liu Z, Cao Y, Pan L, Li J, Chua T S. Exploring and evaluating attributes, values, and structures for entity alignment. In: Proceedings of 2020 Conference on Empirical Methods in Natural Language Processing. 2020, 6355−6364
35
Wu Y, Liu X, Feng Y, Wang Z, Yan R, Zhao D. Relation-aware entity alignment for heterogeneous knowledge graphs. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence. 2019, 5278−5284
36
Wu Y, Liu X, Feng Y, Wang Z, Zhao D. Jointly learning entity and relation representations for entity alignment. In: Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019, 240−249
37
Zhu Y, Liu H, Wu Z, Du Y. Relation-aware neighborhood matching model for entity alignment. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence. 2021, 4749−4756
38
D, Yu Y, Yang R, Zhang Y Wu . Knowledge embedding based graph convolutional network. In: Proceedings of the Web Conference 2021. 2021, 1619−1628
39
Duchi J, Hazan E, Singer Y. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 2011, 12(61): 2121–2159
40
M D Zeiler . ADADELTA: an adaptive learning rate method. 2012, arXiv preprint arXiv: 1212.5701
