Meta-path reasoning of knowledge graph for commonsense question answering

doi:10.1007/s11704-022-2336-6

Front. Comput. Sci.

2024, Vol. 18

Issue (1) : 181303 https://doi.org/10.1007/s11704-022-2336-6

Artificial Intelligence

Meta-path reasoning of knowledge graph for commonsense question answering

Miao ZHANG^1,^2,³, Tingting HE^2,^3,⁴(

), Ming DONG^2,^3,⁴(

)

¹. National Engineering Research Center for E-Learning, Central China Normal University, Wuhan 430079, China
². Hubei Provincial Key Laboratory of Artificial Intelligence and Smart Learning, Central China Normal University, Wuhan 430079, China
³. National Language Resources Monitoring & Research Center for Network Media, Central China Normal University, Wuhan 430079, China
⁴. School of Computer, Central China Normal University, Wuhan 430079, China

Download: PDF(4232 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

Commonsense question answering (CQA) requires understanding and reasoning over QA context and related commonsense knowledge, such as a structured Knowledge Graph (KG). Existing studies combine language models and graph neural networks to model inference. However, traditional knowledge graph are mostly concept-based, ignoring direct path evidence necessary for accurate reasoning. In this paper, we propose MRGNN (Meta-path Reasoning Graph Neural Network), a novel model that comprehensively captures sequential semantic information from concepts and paths. In MRGNN, meta-paths are introduced as direct inference evidence and an original graph neural network is adopted to aggregate features from both concepts and paths simultaneously. We conduct sufficient experiments on the CommonsenceQA and OpenBookQA datasets, showing the effectiveness of MRGNN. Also, we conduct further ablation experiments and explain the reasoning behavior through the case study.

Keywords question answering knowledge graph graph neural network meta-path reasoning

Corresponding Author(s): Tingting HE,Ming DONG

About author:

Changjian Wang and Zhiying Yang contributed equally to this work.

Just Accepted Date: 21 November 2022 Issue Date: 21 February 2023

Cite this article:

Miao ZHANG,Tingting HE,Ming DONG. Meta-path reasoning of knowledge graph for commonsense question answering[J]. Front. Comput. Sci., 2024, 18(1): 181303.

URL:

https://academic.hep.com.cn/fcs/EN/10.1007/s11704-022-2336-6
https://academic.hep.com.cn/fcs/EN/Y2024/V18/I1/181303

Notations	Definitions
$Q$	The QA context
$G$	A knowledge graph $G = {V, R}$
$V = {v}$	The set of vertexes
$R = {r}$	The set of relations
$P = {P}$	The set of meta-paths
$G P$	A subgraph with meta-paths
$W$	Weight matrix
$R d$	$d$ -dimensional euclidean space
$α, β$	Normalized attention weight
$σ (?)$	Activation function
$⊕$	The concatenate operation

Tab.1 Notations and definitions

Fig.1 QA example and triples extracted from KG

Fig.2 The illustrative example in building MRGNN. (a) Three types of entities (from question,option and knowledge graph); (b)

G

consists entities and relations extracted from knowledge graph; (c) The progress of meta-path construction

Fig.3 Overview of MRGNN architecture

Dataset	Train	Dev	Test	Choices
$CQA i n ? h o u s e$	8500	1221	1241	5
$CQA o f f i c i a l$	9741	1221	1140	5
OBQA	4957	500	500	4

Tab.2 Statistics of datasets

Methods	BERT-Large		RoBERTa-Large
Methods	IHdev-Acc./%	IHtest-Acc./%	IHdev-Acc./%	IHtest-Acc./%
w/o KG	61.0( $± 0.8$ )	55.4( $± 0.4$ )	73.1 $± 0.5$ )	68.7( $± 0.6$ )
RGCN	63.0( $± 0.3$ )	57.1( $± 0.4$ )	72.7( $± 0.2$ )	68.4 ( $± 0.7$ )
GAT	63.4( $± 0.4$ )	58.2( $± 1.0$ )	74.0( $± 0.2$ )	71.2( $± 0.7$ )
GconAttn	62.3( $± 0.4$ )	57.2( $± 0.5$ )	72.6( $± 0.4$ )	68.6( $± 0.1$ )
KagNet	62.3( $± 0.4$ )	57.2( $± 0.5$ )	73.5( $± 0.2$ )	69.0( $± 0.8$ )
RN	63.4( $± 0.3$ )	58.9( $± 0.1$ )	74.6( $± 0.9$ )	69.1( $± 0.2$ )
MHGRN	63.3( $± 0.5$ )	60.6( $± 0.6$ )	74.45( $± 0.2$ )	71.1( $± 0.2$ )
PathGenerator	?	59.07( $± 0.3$ )	?	72.7( $± 0.4$ )
HGN	?	60.9( $± 0.2$ )	?	73.6( $± 0.3$ )
RaB-PR	63.3	60.8	75.6	73.7
QA-GNN	65.1( $± 0.2$ )	?	76.5( $± 0.2$ )	73.4( $± 0.4$ )
MRGNN(w/ concat)	65.4( $± 0.1$ )	61.8( $± 0.3$ )	75.5 ( $± 0.6$ )	73.1( $± 0.6$ )
MRGNN(w/ retrieve)	65.0( $± 0.2$ )	62.0( $± 0.2$ )	75.3( $± 0.7$ )	73.6( $± 0.5$ )
MRGNN(w/ syntactic)	65.7( $± 0.2$ )	62.3( $± 0.4$ )	75.8( $± 0.8$ )	73.5( $± 0.4$ )

Tab.3 Performance of baseline models on CommonsenceQA in-house split

Tab.4 Performance on the CommonsenseQA in official-split (dev)

Methods	RoBERTa-Large	AristoRoBERTaV7
Fine-tuned w/o KG	64.8( $± 2.3$ )	78.4( $± 1.6$ )
RGCN	62.4( $± 1.6$ )	74.6( $± 2.5$ )
GconAttn	64.7( $± 1.5$ )	71.8( $± 1.2$ )
RN	65.2( $± 1.2$ )	75.3( $± 1.4$ )
GAT	65.0( $± 1.3$ )	78.2( $± 1.2$ )
MHGRN	66.8( $± 1.2$ )	80.6
PathGenerator	?	79.2( $± 0.8$ )
QA-GNN	67.8( $± 2.7$ )	82.7( $± 1.6$ )
HGN	66.2	84.3
RaB-PR	68.3	83.6
MRGNN(w/ concat)	68.4( $± 2.1$ )	83.1( $± 1.3$ )
MRGNN(w/ retrieve)	69.5( $± 1.7$ )	83.6( $± 0.9$ )
MRGNN(w/ syntactic)	69.0( $± 1.6$ )	83.4( $± 1.2$ )

Tab.5 Test accuracy comparison on OpenBookQA. For baselines, the accuracy is reported from [2,3]

Tab.6 Comparisons with methods on the leaderboard for OpenBookQA

Fig.4 Ablation study on GNN components, using the CommonsenseQA in-house split set. N is the GNN layers

Fig.5 A sample of Case Study. The question is taken from the CommonsenseQA dataset. The broad lines show the meta-path changes that the model pays attention to as the GNN message passes

1	Lin B Y, Chen X, Chen J, Ren X. KagNet: knowledge-aware graph networks for commonsense reasoning. In: Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019, 2829−2839
2	Feng Y, Chen X, Lin B Y, Wang P, Yan J, Ren X. Scalable multi-hop relational reasoning for knowledge-aware question answering. In: Proceedings of 2020 Conference on Empirical Methods in Natural Language Processing. 2020, 1295−1309
3	Yasunaga M, Ren H, Bosselut A, Liang P, Leskovec J. QA-GNN: reasoning with language models and knowledge graphs for question answering. In: Proceedings of 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021, 535−546
4	Talmor A, Herzig J, Lourie N, Berant J. CommonsenseQA: a question answering challenge targeting commonsense knowledge. In: Proceedings of 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019, 4149−4158
5	S, Hao H Zhang . Performance analysis of PHY layer for RIS-assisted wireless communication systems with retransmission protocols. Journal of King Saud University-Computer and Information Sciences, 2022, 34( 8): 5388–5404
6	Devlin J, Chang M W, Lee K, Toutanova K. BERT: pretraining of deep bidirectional transformers for language understanding. In: Proceedings of 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019, 4171−4186
7	Lan Z, Chen M, Goodman S, Gimpel K, Sharma P, Soricut R. ALBERT: a lite BERT for self-supervised learning of language representations. In: Proceedings of the 8th International Conference on Learning Representations. 2020
8	C, Raffel N, Shazeer A, Roberts K, Lee S, Narang M, Matena Y, Zhou W, Li P J Liu . Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 2020, 21( 1): 140
9	D, Khashabi S, Min T, Khot A, Sabharwal O, Tafjord P, Clark H Hajishirzi . UNIFIEDQA: crossing format boundaries with a single QA system. In: Proceedings of Findings of the Association for Computational Linguistics: EMNLP 2020. 2020, 1896−1907
10	Speer R, Chin J, Havasi C. ConceptNet 5.5: an open multilingual graph of general knowledge. In: Proceedings of the 31st AAAI Conference on Artificial Intelligence. 2017, 4444−4451
11	Y, Liu M, Ott N, Goyal J, Du M, Joshi D, Chen O, Levy M, Lewis L, Zettlemoyer V Stoyanov . RoBERTa: a robustly optimized BERT pretraining approach. 2016, arXiv preprint arXiv: 1907.11692
12	Shen T, Geng X, Qin T, Guo D, Tang D, Duan N, Long G, Jiang D. Multi-task learning for conversational question answering over a large-scale knowledge base. In: Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019, 2442−2451
13	T N, Kipf M Welling . Semi-supervised classification with graph convolutional networks. 2017, arXiv preprint arXiv: 1609.02907
14	M, Schlichtkrull T N, Kipf P, Bloem Den Berg R, Van I, Titov M Welling . Modeling relational data with graph convolutional networks. In: Proceedings of the 15th International Conference on the Semantic Web. 2018, 593−607
15	P, Veličković A, Casanova P, Liò G, Cucurull A, Romero Y Bengio . Graph attention networks. In: Proceedings of the 6th International Conference on Learning Representations. 2018
16	A, Bordes N, Usunier A, Garcia-Durán J, Weston O Yakhnenko . Translating embeddings for modeling multi-relational data. In: Proceedings of the 26th International Conference on Neural Information Processing Systems. 2013, 2787−2795
17	Y, Sun J Han . Mining Heterogeneous Information Networks: Principles and Methodologies. Synthesis Lectures on Data Mining and Knowledge Discovery. San Rafael: Morgan & Claypool Publishers, 2012
18	A, Santoro D, Raposo D G T, Barrett M, Malinowski R, Pascanu P W, Battaglia T Lillicrap . A simple neural network module for relational reasoning. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 4974−4983
19	M, Heilman N A Smith . Good question! statistical ranking for question generation. In: Proceedings of Human Language Technologies: Conference of the North American Chapter of the Association for Computational Linguistics. 2010, 609−617
20	S, Hochreiter J Schmidhuber . Long short-term memory. Neural Computation, 1997, 9( 8): 1735–1780
21	Mihaylov T, Clark P, Khot T, Sabharwal A. Can a suit of armor conduct electricity? A new dataset for open book question answering. In: Proceedings of 2018 Conference on Empirical Methods in Natural Language Processing. 2018, 2381−2391
22	Mihaylov T, Frank A. Knowledgeable reader: enhancing cloze-style reading comprehension with external commonsense knowledge. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018, 821−832
23	A M, Monea A N Marginean . Medical question entailment based on textual inference and fine-tuned BioMed-RoBERTa. In: Proceedings of the 17th IEEE International Conference on Intelligent Computer Communication and Processing. 2021, 319−326
24	Wang X, Kapanipathi P, Musa R, Yu M, Talamadupula K, Abdelaziz I, Chang M, Fokoue A, Makni B, Mattei N, Witbrock M. Improving natural language inference using external knowledge in the science questions domain. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence and the 31st Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence. 2019, 7208−7215
25	X, Zhan Y, Huang X, Dong Q, Cao X Liang . PathReasoner: explainable reasoning paths for commonsense question answering. Knowledge-Based Systems, 2022, 235: 107612
26	J, Yan M, Raman A, Chan T, Zhang R, Rossi H, Zhao S, Kim N, Lipka X Ren . Learning contextualized knowledge structures for commonsense reasoning. In: Proceedings of the Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. 2021, 4038−4051
27	J, Wang X, Li Z, Tan X, Zhao W Xiao . Relation-aware bidirectional path reasoning for commonsense question answering. In: Proceedings of the 25th Conference on Computational Natural Language Learning. 2021, 445−453
28	A, Saxena A, Tripathi P P Talukdar . Improving multi-hop question answering over knowledge graphs using knowledge base embeddings. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020, 4498−4507
29	J, Bao N, Duan Z, Yan M, Zhou T Zhao . Constraint-based question answering with knowledge graph. In: Proceedings of the COLING 2016, 26th International Conference on Computational Linguistics: Technical Papers. 2016, 2503−2514
30	R, Das S, Dhuliawala M, Zaheer L, Vilnis I, Durugkar A, Krishnamurthy A, Smola A McCallum . Go for a walk and arrive at the answer: reasoning over paths in knowledge bases using reinforcement learning. In: Proceedings of the 6th International Conference on Learning Representations. 2018
31	D, Chen M, Yang H T, Zheng Y, Li Y Shen . Answer-enhanced path-aware relation detection over knowledge base. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2019, 1021−1024
32	S, Ji S, Pan E, Cambria P, Marttinen P S Yu . A survey on knowledge graphs: representation, acquisition, and applications. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33( 2): 494–514
33	Bollacker K, Evans C, Paritosh P, Sturge T, Taylor J. Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of 2008 ACM SIGMOD International Conference on Management of Data. 2008, 1247−1250
34	S, Kundu T, Khot A, Sabharwal P Clark . Exploiting explicit paths for multi-hop reading comprehension. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019, 2737−2747
35	Chen W, Xiong W, Yan X, Wang W Y. Variational knowledge graph reasoning. In: Proceedings of 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2018, 1823−1832
36	W T, Yih M W, Chang X, He J Gao . Semantic parsing via staged query graph generation: question answering with knowledge base. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 2015, 1321−1331
37	Y, Lan J Jiang . Query graph generation for answering multi-hop complex questions from knowledge bases. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020, 969−974

[1]

FCS-22336-OF-MZ_suppl_1

Download

[1]	Jingyu LIU, Shi CHEN, Li SHEN. A comprehensive survey on graph neural network accelerators[J]. Front. Comput. Sci., 2025, 19(2): 192104-.
[2]	Junfei TANG, Ran SONG, Yuxin HUANG, Shengxiang GAO, Zhengtao YU. Semantic-aware entity alignment for low resource language knowledge graph[J]. Front. Comput. Sci., 2024, 18(4): 184319-.
[3]	Qi LIU, Qinghua ZHANG, Fan ZHAO, Guoyin WANG. Uncertain knowledge graph embedding: an effective method combining multi-relation and multi-path[J]. Front. Comput. Sci., 2024, 18(3): 183311-.
[4]	Jiaqi LIU, Zhiwen YU, Bin GUO, Cheng DENG, Luoyi FU, Xinbing WANG, Chenghu ZHOU. EvolveKG: a general framework to learn evolving knowledge graphs[J]. Front. Comput. Sci., 2024, 18(3): 183309-.
[5]	Yongquan LIANG, Qiuyu SONG, Zhongying ZHAO, Hui ZHOU, Maoguo GONG. BA-GNN: Behavior-aware graph neural network for session-based recommendation[J]. Front. Comput. Sci., 2023, 17(6): 176613-.
[6]	Jinwei LUO, Mingkai HE, Weike PAN, Zhong MING. BGNN: Behavior-aware graph neural network for heterogeneous session-based recommendation[J]. Front. Comput. Sci., 2023, 17(5): 175336-.
[7]	Yuan GAO, Xiang WANG, Xiangnan HE, Huamin FENG, Yongdong ZHANG. Rumor detection with self-supervised learning on texts and social graph[J]. Front. Comput. Sci., 2023, 17(4): 174611-.
[8]	Shuang LIU, Fan ZHANG, Baiyang ZHAO, Renjie GUO, Tao CHEN, Meishan ZHANG. APPCorp: a corpus for Android privacy policy document structure analysis[J]. Front. Comput. Sci., 2023, 17(3): 173320-.
[9]	Zhe XUE, Junping DU, Xin XU, Xiangbin LIU, Junfu WANG, Feifei KOU. Few-shot node classification via local adaptive discriminant structure learning[J]. Front. Comput. Sci., 2023, 17(2): 172316-.
[10]	Yongchun ZHU, Fuzhen ZHUANG, Xiangliang ZHANG, Zhiyuan QI, Zhiping SHI, Juan CAO, Qing HE. Combat data shift in few-shot learning with knowledge graph[J]. Front. Comput. Sci., 2023, 17(1): 171305-.
[11]	Rui LIU, Yahong HAN. Instance-sequence reasoning for video question answering[J]. Front. Comput. Sci., 2022, 16(6): 166708-.
[12]	Yunhao SUN, Guanyu LI, Jingjing DU, Bo NING, Heng CHEN. A subgraph matching algorithm based on subgraph index for knowledge graph[J]. Front. Comput. Sci., 2022, 16(3): 163606-.
[13]	Chuan SHI, Jiayu DING, Xiaohuan CAO, Linmei HU, Bin WU, Xiaoli LI. Entity set expansion in knowledge graph: a heterogeneous information network perspective[J]. Front. Comput. Sci., 2021, 15(1): 151307-.
[14]	Dongsheng WANG. Answering contextual questions based on ontologies and question templates[J]. Front Comput Sci Chin, 2011, 5(4): 405-418.

Viewed

Full text

Abstract

Cited

Shared

Discussed