Semantic-aware entity alignment for low resource language knowledge graph

doi:10.1007/s11704-023-2542-x

Front. Comput. Sci.

2024, Vol. 18

Issue (4) : 184319 https://doi.org/10.1007/s11704-023-2542-x

Artificial Intelligence

Semantic-aware entity alignment for low resource language knowledge graph

Junfei TANG^1,², Ran SONG^1,², Yuxin HUANG^1,², Shengxiang GAO^1,², Zhengtao YU^1,²(

)

¹. Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China
². Yunnan Key Laboratory of Artificial Intelligence, Kunming University of Science and Technology, Kunming 650500, China

Download: PDF(9116 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

Entity alignment (EA) is an important technique aiming to find the same real entity between two different source knowledge graphs (KGs). Current methods typically learn the embedding of entities for EA from the structure of KGs for EA. Most EA models are designed for rich-resource languages, requiring sufficient resources such as a parallel corpus and pre-trained language models. However, low-resource language KGs have received less attention, and current models demonstrate poor performance on those low-resource KGs. Recently, researchers have fused relation information and attributes for entity representations to enhance the entity alignment performance, but the relation semantics are often ignored. To address these issues, we propose a novel Semantic-aware Graph Neural Network (SGNN) for entity alignment. First, we generate pseudo sentences according to the relation triples and produce representations using pre-trained models. Second, our approach explores semantic information from the connected relations by a graph neural network. Our model captures expanded feature information from KGs. Experimental results using three low-resource languages demonstrate that our proposed SGNN approach out performs better than state-of-the-art alignment methods on three proposed datasets and three public datasets.

Keywords graph neural network knowledge graph entity alignment low-resource language

Corresponding Author(s): Zhengtao YU

About author:

* Both are co-first authors.

Just Accepted Date: 02 February 2023 Issue Date: 16 June 2023

Cite this article:

Junfei TANG,Ran SONG,Yuxin HUANG, et al. Semantic-aware entity alignment for low resource language knowledge graph[J]. Front. Comput. Sci., 2024, 18(4): 184319.

URL:

https://academic.hep.com.cn/fcs/EN/10.1007/s11704-023-2542-x
https://academic.hep.com.cn/fcs/EN/Y2024/V18/I4/184319

Fig.1 Performance of an EA model for different KGs (a) e, e', e'' are equivalent entities from Chinese, Vietnamese, and English KGs respectively; (b) the EA model performs suitably for rich-resource languages but falters for the low-resource language

Fig.2 An example to description the important of relation in entity alignment. KG1 is a Chinese language KG, KG2 is a Vietnamese language KG.Renaissance,Rome,Florence,Michelangelo have been aligned

Fig.3 Overall architecture of the proposed SGNN model.

a i

e i

and

r i

denote entity attribute, entity and relation, respectively;

E i i n i t

and

X i i n i t

are the pre-trained entity features and pre-trained relation features of KG

i

, respectively;

G i r

is the relation sub-graph which constructed according to

T i r

;

X i

is the relation representation which has aggregated by GAT;

E i 0

is the initial entity representation of GCN;

S k

is the similarity matrix of channel k

Fig.4 Generating pseudo sentences by concatenating head entity, relation, and tail entity from the relation triple

Fig.5 Building the relation sub-graph from the original KG. It is likely that

r 1

and

r 2

have the same head entity,

r 1

and

r 4

have the same head entity, and

r 1

and

r 3

have the same head entity. Thus,

r 2, r 3

, and

r 4

are connected with

r 1

to built the relation sub-graph

Tab.1 Summary of our construct datasets

Tab.2 Summary of DBP15K

Tab.3 Overall performance on our datasets

Model	$D B P 15 K Z H ? E N$		$D B P 15 K J A ? E N$		$D B P 15 K F R ? E N$
Model	Hits@1	Hits@10	Hits@1	Hits@10	Hits@1	Hits@10
GCN-Align(2018) [30]	50.82	79.15	53.09	82.96	54.49	84.73
RDGCN(2019) [35]	70.75	84.55	76.74	89.54	88.64	95.72
AttrGNN(2020) [34]	79.60	92.93	78.33	92.08	91.85	97.77
KE-GCN(2021) [38]	56.20	84.20	57.00	85.20	57.20	85.40
SGNN	82.83	94.17	83.73	93.76	91.54	97.88

Tab.4 Overall performance of baseline models on the DBP15K datasets

Model	$Z H ? V I$
Model	Hits@1	Hits@10
SGNN	45.17	71.28
SGNN-w/o structure	23.03	38.02
SGNN-w/o attribute	29.10	55.63
SGNN-w/o ps	43.01	69.30
SGNN-mBERT	43.97	71.02
SGNN-random	40.01	65.69
SGNN-w/o sub-graph	38.97	63.71

Tab.5 Result of ablation study in structure channel

Fig.6 Results of entity alignment w.r.t the proportion of seed alignments. (a) Different training proportion in ZH-VI; (b) different training proportion in

D B P 15 K Z H ? E N

Tab.6 Case study of relational semantics in entity alignment

1	B, Yang T Mitchell . Leveraging knowledge bases in LSTMs for improving machine reading. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2019, 1436−1446
2	Y, Cao L, Hou J, Li Z Liu . Neural collective entity linking. In: Proceedings of the 27th International Conference on Computational Linguistics. 2018, 675−686
3	Yang Z, Qi P, Zhang S, Bengio Y, Cohen W, Salakhutdinov R, Manning C D. HotpotQA: a dataset for diverse, explainable multi-hop question answering. In: Proceedings of 2018 Conference on Empirical Methods in Natural Language Processing. 2018, 2369−2380
4	M, Franco-Salvador P, Rosso M Montes-y-Gómez . A systematic study of knowledge graph analysis for cross-language plagiarism detection. Information Processing & Management, 2016, 52( 4): 550–570
5	J, Lehmann R, Isele M, Jakob A, Jentzsch D, Kontokostas P N, Mendes S, Hellmann M, Morsey Kleef P, van S, Auer C Bizer . DBpedia–A large-scale, multilingual knowledge base extracted from Wikipedia. Semantic Web, 2015, 6( 2): 167–195
6	F M, Suchanek G, Kasneci G Weikum . Yago: a core of semantic knowledge. In: Proceedings of the 16th International Conference on World Wide Web. 2007, 697−706
7	R, Navigli S P Ponzetto . BabelNet: the automatic construction, evaluation and application of a wide-coverage multilingual semantic network. Artificial Intelligence, 2012, 193: 217–250
8	M, Chen Y, Tian M, Yang C Zaniolo . Multilingual knowledge graph embeddings for cross-lingual knowledge alignment. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence. 2017, 1511−1517
9	Z, Sun W, Hu C Li . Cross-lingual entity alignment via joint attribute-preserving embedding. In: Proceedings of the 16th International Semantic Web Conference. 2017, 628−644
10	P, Veličković G, Cucurull A, Casanova A, Romero P, Liò Y Bengio . Graph attention networks. In: Proceedings of the ICLR 2018. 2018
11	A, Conneau K, Khandelwal N, Goyal V, Chaudhary G, Wenzek F, Guzmán E, Grave M, Ott L, Zettlemoyer V Stoyanov . Unsupervised cross-lingual representation learning at scale. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020, 8440−8451
12	Petroni F, Rocktäschel T, Riedel S, Lewis P, Bakhtin A, Wu Y, Miller A H, Riedel S. Language models as knowledge bases? In: Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019, 2463−2473
13	Jiang Z, Anastasopoulos A, Araki J, Ding H, Neubig G. X-FACTR: multilingual factual knowledge retrieval from pretrained language models. In: Proceedings of 2020 Conference on Empirical Methods in Natural Language Processing. 2020, 5943−5959
14	A, Sanchez-Gonzalez N, Heess J T, Springenberg J, Merel M A, Riedmiller R, Hadsell P W Battaglia . Graph networks as learnable physics engines for inference and control. In: Proceedings of the 35th International Conference on Machine Learning. 2018, 4467−4476
15	Y, Wu D, Lian Y, Xu L, Wu E Chen . Graph convolutional networks with Markov random field reasoning for social spammer detection. In: Proceedings of the AAAI Conference on Artificial Intelligence. 2020, 1054−1061
16	A, Fout J, Byrd B, Shariat A Ben-Hur . Protein interface prediction using graph convolutional networks. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 6533−6542
17	H, Dai E B, Khalil Y, Zhang B, Dilkina L Song . Learning combinatorial optimization algorithms over graphs. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 6351−6361
18	T N, Kipf M Welling . Semi-supervised classification with graph convolutional networks. In: Proceedings of the 5th International Conference on Learning Representations. 2017
19	Y, Li D, Tarlow M, Brockschmidt R S Zemel . Gated graph sequence neural networks. In: Proceedings of the 4th International Conference on Learning Representations. 2016
20	K, Xu W, Hu J, Leskovec S Jegelka . How powerful are graph neural networks? In: Proceedings of the 7th International Conference on Learning Representations. 2019
21	M, Defferrard X, Bresson P Vandergheynst . Convolutional neural networks on graphs with fast localized spectral filtering. In: Proceedings of the 30th International Conference on Neural Information Processing Systems. 2016, 3844−3852
22	W L, Hamilton R, Ying J Leskovec . Inductive representation learning on large graphs. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 1025−1035
23	A, Bordes N, Usunier A, Garcia-Durán J, Weston O Yakhnenko . Translating embeddings for modeling multi-relational data. In: Proceedings of the 26th International Conference on Neural Information Processing Systems. 2013, 2787−2795
24	S, Ji S, Pan E, Cambria P, Marttinen P S Yu . A survey on knowledge graphs: representation, acquisition, and applications. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33( 2): 494–514
25	Wang Z, Zhang J, Feng J, Chen Z. Knowledge graph embedding by translating on hyperplanes. In: Proceedings of the 28th AAAI Conference on Artificial Intelligence. 2014, 1112−1119
26	Lin Y, Liu Z, Sun M, Liu Y, Zhu X. Learning entity and relation embeddings for knowledge graph completion. In: Proceedings of the 29th AAAI Conference on Artificial Intelligence. 2015, 2181−2187
27	H, Zhu R, Xie Z, Liu M Sun . Iterative entity alignment via joint knowledge embeddings. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence. 2017, 4258−4264
28	Z, Sun W, Hu Q, Zhang Y Qu . Bootstrapping entity alignment with knowledge graph embedding. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence. 2018, 4396−4402
29	Q, Zhu X, Zhou J, Wu J, Tan L Guo . Neighborhood-aware attentional representation for multilingual knowledge graphs. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence. 2019, 1943−1949
30	Wang Z, Lv Q, Lan X, Zhang Y. Cross-lingual knowledge graph alignment via graph convolutional networks. In: Proceedings of 2018 Conference on Empirical Methods in Natural Language Processing. 2018, 349−357
31	Y, Cao Z, Liu C, Li Z, Liu J, Li T S Chua . Multi-channel graph neural network for entity alignment. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019, 1452−1461
32	K, Xu L, Wang M, Yu Y, Feng Y, Song Z, Wang D Yu . Cross-lingual knowledge graph alignment via graph matching neural network. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019, 3156−3161
33	Y, Wu X, Liu Y, Feng Z, Wang D Zhao . Neighborhood matching network for entity alignment. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020, 6477−6487
34	Liu Z, Cao Y, Pan L, Li J, Chua T S. Exploring and evaluating attributes, values, and structures for entity alignment. In: Proceedings of 2020 Conference on Empirical Methods in Natural Language Processing. 2020, 6355−6364
35	Wu Y, Liu X, Feng Y, Wang Z, Yan R, Zhao D. Relation-aware entity alignment for heterogeneous knowledge graphs. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence. 2019, 5278−5284
36	Wu Y, Liu X, Feng Y, Wang Z, Zhao D. Jointly learning entity and relation representations for entity alignment. In: Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019, 240−249
37	Zhu Y, Liu H, Wu Z, Du Y. Relation-aware neighborhood matching model for entity alignment. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence. 2021, 4749−4756
38	D, Yu Y, Yang R, Zhang Y Wu . Knowledge embedding based graph convolutional network. In: Proceedings of the Web Conference 2021. 2021, 1619−1628
39	Duchi J, Hazan E, Singer Y. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 2011, 12(61): 2121–2159
40	M D Zeiler . ADADELTA: an adaptive learning rate method. 2012, arXiv preprint arXiv: 1212.5701

[1]

FCS-22542-OF-JT_suppl_1

Download

[1]	Bingbing DONG, Chenyang BU, Yi ZHU, Shengwei JI, Xindong WU. Simplified multi-view graph neural network for multilingual knowledge graph completion[J]. Front. Comput. Sci., 2025, 19(7): 197324-.
[2]	Yanping ZHENG, Lu YI, Zhewei WEI. A survey of dynamic graph neural networks[J]. Front. Comput. Sci., 2025, 19(6): 196323-.
[3]	Shuo WANG, Xinjun MAO, Shuo YANG, Menghan WU, Zhang ZHANG. ROS package search for robot software development: a knowledge graph-based approach[J]. Front. Comput. Sci., 2025, 19(6): 196320-.
[4]	Yao WU, Hong HUANG, Yu SONG, Hai JIN. Soft-GNN: towards robust graph neural networks via self-adaptive data utilization[J]. Front. Comput. Sci., 2025, 19(4): 194311-.
[5]	Tao HE, Ming LIU, Yixin CAO, Zekun WANG, Zihao ZHENG, Bing QIN. Exploring & exploiting high-order graph structure for sparse knowledge graph completion[J]. Front. Comput. Sci., 2025, 19(2): 192306-.
[6]	Jingyu LIU, Shi CHEN, Li SHEN. A comprehensive survey on graph neural network accelerators[J]. Front. Comput. Sci., 2025, 19(2): 192104-.
[7]	Qi LIU, Qinghua ZHANG, Fan ZHAO, Guoyin WANG. Uncertain knowledge graph embedding: an effective method combining multi-relation and multi-path[J]. Front. Comput. Sci., 2024, 18(3): 183311-.
[8]	Jiaqi LIU, Zhiwen YU, Bin GUO, Cheng DENG, Luoyi FU, Xinbing WANG, Chenghu ZHOU. EvolveKG: a general framework to learn evolving knowledge graphs[J]. Front. Comput. Sci., 2024, 18(3): 183309-.
[9]	Miao ZHANG, Tingting HE, Ming DONG. Meta-path reasoning of knowledge graph for commonsense question answering[J]. Front. Comput. Sci., 2024, 18(1): 181303-.
[10]	Yongquan LIANG, Qiuyu SONG, Zhongying ZHAO, Hui ZHOU, Maoguo GONG. BA-GNN: Behavior-aware graph neural network for session-based recommendation[J]. Front. Comput. Sci., 2023, 17(6): 176613-.
[11]	Jinwei LUO, Mingkai HE, Weike PAN, Zhong MING. BGNN: Behavior-aware graph neural network for heterogeneous session-based recommendation[J]. Front. Comput. Sci., 2023, 17(5): 175336-.
[12]	Yuan GAO, Xiang WANG, Xiangnan HE, Huamin FENG, Yongdong ZHANG. Rumor detection with self-supervised learning on texts and social graph[J]. Front. Comput. Sci., 2023, 17(4): 174611-.
[13]	Shuang LIU, Fan ZHANG, Baiyang ZHAO, Renjie GUO, Tao CHEN, Meishan ZHANG. APPCorp: a corpus for Android privacy policy document structure analysis[J]. Front. Comput. Sci., 2023, 17(3): 173320-.
[14]	Zhe XUE, Junping DU, Xin XU, Xiangbin LIU, Junfu WANG, Feifei KOU. Few-shot node classification via local adaptive discriminant structure learning[J]. Front. Comput. Sci., 2023, 17(2): 172316-.
[15]	Yongchun ZHU, Fuzhen ZHUANG, Xiangliang ZHANG, Zhiyuan QI, Zhiping SHI, Juan CAO, Qing HE. Combat data shift in few-shot learning with knowledge graph[J]. Front. Comput. Sci., 2023, 17(1): 171305-.

Viewed

Full text

Abstract

Cited

Shared

Discussed