Please wait a minute...
Frontiers of Computer Science

ISSN 2095-2228

ISSN 2095-2236(Online)

CN 10-1014/TP

Postal Subscription Code 80-970

2018 Impact Factor: 1.129

Front. Comput. Sci.    2021, Vol. 15 Issue (3) : 153323    https://doi.org/10.1007/s11704-020-9420-6
RESEARCH ARTICLE
Nested relation extraction with iterative neural network
Yixuan CAO1,2, Dian CHEN1,2, Zhengqi XU1,2, Hongwei LI1,2, Ping LUO1,2()
1. Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS), Institute of Computing Technology, CAS, Beijing 100190, China
2. University of Chinese Academy of Sciences, Beijing 100049, China
 Download: PDF(1014 KB)  
 Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks
Abstract

Most existing researches on relation extraction focus on binary flat relations like BornIn relation between a Person and a Location. But a large portion of objective facts described in natural language are complex, especially in professional documents in fields such as finance and biomedicine that require precise expressions. For example, “the GDP of the United States in 2018 grew 2.9% compared with 2017” describes a growth rate relation between two other relations about the economic index, which is beyond the expressive power of binary flat relations. Thus, we propose the nested relation extraction problem and formulate it as a directed acyclic graph (DAG) structure extraction problem. Then, we propose a solution using the Iterative Neural Network which extracts relations layer by layer. The proposed solution achieves 78.98 and 97.89 F1 scores on two nested relation extraction tasks, namely semantic cause-and-effect relation extraction and formula extraction. Furthermore, we observe that nested relations are usually expressed in long sentences where entities are mentioned repetitively, which makes the annotation difficult and errorprone. Hence, we extend our model to incorporate a mentioninsensitive mode that only requires annotations of relations on entity concepts (instead of exact mentions) while preserving most of its performance. Our mention-insensitive model performs better than the mention sensitive model when the random level in mention selection is higher than 0.3.

Keywords nested relation extraction      mention insensitive relation      iterative neural network     
Corresponding Author(s): Ping LUO   
Just Accepted Date: 09 April 2020   Issue Date: 27 January 2021
 Cite this article:   
Yixuan CAO,Dian CHEN,Zhengqi XU, et al. Nested relation extraction with iterative neural network[J]. Front. Comput. Sci., 2021, 15(3): 153323.
 URL:  
https://academic.hep.com.cn/fcs/EN/10.1007/s11704-020-9420-6
https://academic.hep.com.cn/fcs/EN/Y2021/V15/I3/153323
1 P Ernst, A Siu, G Weikum. Highlife: higher-arity fact harvesting. In: Proceedings of the 2018 World Wide Web Conference. 2018, 1013–1022
https://doi.org/10.1145/3178876.3186000
2 N Hassan, F Arslan, C Li, M Tremayne. Toward automated fact-checking: detecting check-worthy factual claims by claimbuster. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2017, 1803–1812
https://doi.org/10.1145/3097983.3098131
3 M Mintz, S Bills, R Snow, J Dan. Distant supervision for relation extraction without labeled data. In: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP. 2009, 1003–1011
https://doi.org/10.3115/1690219.1690287
4 C C Aggarwal, C Zhai. Mining Text Data. Springer Science & Business Media, 2012
https://doi.org/10.1007/978-1-4614-3223-4
5 M Miwa, M Bansal. End-to-end relation extraction using LSTMs on sequences and tree structures. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016, 1105–1116
https://doi.org/10.18653/v1/P16-1105
6 Y Xu, L Mou, G Li, Y Chen, H Peng, Z Jin. Classifying relations via long short term memory networks along shortest dependency paths. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 2015, 1785–1794
https://doi.org/10.18653/v1/D15-1206
7 P Zhou, W Shi, J Tian, Z Qi, B Li, H Hao, B Xu. Attention-based bidirectional long short-term memory networks for relation classification. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016, 207–212
https://doi.org/10.18653/v1/P16-2034
8 Y Zhang, P Qi, C D Manning. Graph convolution over pruned dependency trees improves relation extraction. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018, 2205–2215
https://doi.org/10.18653/v1/D18-1244
9 A Katiyar, C Cardie. Going out on a limb: joint extraction of entity mentions and relations without dependency trees. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017, 917–928
https://doi.org/10.18653/v1/P17-1085
10 F Christopoulou, M Miwa, S Ananiadou. A walk-based model on entity graphs for relation extraction. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018, 81–88
https://doi.org/10.18653/v1/P18-2014
11 W Zeng, Y Lin, Z Liu, M Sun. Incorporating relation paths in neural relation extraction. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2017, 1768–1777
https://doi.org/10.18653/v1/D17-1186
12 F M Suchanek, G Kasneci, G Weikum. Yago: a large ontology from wikipedia and wordnet. Journal of Web Semantics, 2008, 6(3): 203–217
https://doi.org/10.1016/j.websem.2008.06.001
13 D Zhou, D Zhong, Y He. Biomedical relation extraction: from binary to complex. Computational and Mathematical Methods in Medicine, 2014
https://doi.org/10.1155/2014/298473
14 R McDonald, F Pereira, S Kulick, S Winters, Y Jin, P White. Simple algorithms for complex relation extraction with applications to biomedical IE. In: Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics. 2005, 491–498
https://doi.org/10.3115/1219840.1219901
15 J Li, Y Sun, R J Johnson, D Sciaky, C H Wei, R Leaman, A P Davis, C J Mattingly, T C Wiegers, Z Lu. BioCreative V CDR task corpus: a resource for chemical disease relation extraction. Database: the Journal of Biological Databases & Curation, 2016, 2016: baw068
https://doi.org/10.1093/database/baw068
16 Y Peng, C H Wei, Z Lu. Improving chemical disease relation extraction with rich features and weakly labeled data. Journal of Cheminformatics, 2016, 8(1): 53
https://doi.org/10.1186/s13321-016-0165-z
17 P Verga, E Strubell, A McCallum. Simultaneously self-attending to all mentions for full-abstract biological relation extraction. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 2018, 872–884
https://doi.org/10.18653/v1/N18-1080
18 L Cui, F Wei, M Zhou. Neural open information extraction. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018, 407–413
https://doi.org/10.18653/v1/P18-2065
19 V Reshadat, M Hoorali, H Faili. A hybrid method for open information extraction based on shallow and deep linguistic analysis. Interdisciplinary Information Sciences, 2016, 22(1): 87–100
https://doi.org/10.4036/iis.2016.R.03
20 V Reshadat, H Faili. A new open information extraction system using sentence difficulty estimation. Computing and Informatics, 2019, 38(4): 986–1008
https://doi.org/10.31577/cai_2019_4_986
21 M Sun, X Li, X Wang, M Fan, Y Feng, P Li. Logician: a unified end-to-end neural approach for open-domain information extraction. In: Proceedings of the 11th ACM International Conference on Web Search & Data Mining. 2018
https://doi.org/10.1145/3159652.3159712
22 Y Chen, L Xu, K Liu, D Zeng, J Zhao. Event extraction via dynamic multi-pooling convolutional neural networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 2015, 167–176
https://doi.org/10.3115/v1/P15-1017
23 P Blunsom, d N Freitas, E Grefenstette, K M Hermann. A deep architecture for semantic parsing. In: Proceedings of the ACL 2014 Workshop on Semantic Parsing. 2014
24 C Liang, J Berant, Q Le, K D Forbus, N Lao. Neural symbolic machines: learning semantic parsers on freebase with weak supervision. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017, 23–33
https://doi.org/10.18653/v1/P17-1003
25 Y Wang, J Berant, P Liang. Building a semantic parser overnight. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 2015, 1332–1342
https://doi.org/10.3115/v1/P15-1129
26 C Xiao, M Dymetman, C Gardent. Sequence-based structured prediction for semantic parsing. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016, 1341–1350
https://doi.org/10.18653/v1/P16-1127
27 J Berant, A Chou, R Frostig, P Liang. Semantic parsing on freebase from question-answer pairs. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. 2013, 1533–1544
28 D Hershcovich, O Abend, A Rappoport. A transition-based directed acyclic graph parser for UCCA. In: Proceedings of the 55th AnnualMeeting of the Association for Computational Linguistics. 2017, 1127–1138
https://doi.org/10.18653/v1/P17-1104
29 X Zhu, P Sobihani, H Guo. Long short-term memory over recursive structures. In: Proceedings of the 32nd International Conference on Machine Learning. 2015, 1604–1612
30 K S Tai, R Socher, C D Manning. Improved semantic representations from tree-structured long short-term memory networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 2015, 1556–1566
https://doi.org/10.3115/v1/P15-1150
31 R Agerri, G Rigau. Robust multilingual named entity recognition with shallow semi-supervised features. Artificial Intelligence, 2016, 238: 63–82
https://doi.org/10.1016/j.artint.2016.05.003
32 J Aguilar, C Beller, P McNamee, B Van Durme, S Strassel, Z Song, J Ellis. A comparison of the events and relations across ACE, ERE, TAC-KBP, and framenet annotation standards. In: Proceedings of the 2nd Workshop on EVENTS: Definition, Detection, Coreference, and Representation. 2014, 45–53
https://doi.org/10.3115/v1/W14-2907
33 R Girju, P Nakov, V Nastase, S Szpakowicz, P Turney, D Yuret. Semeval- 2007 task 04: classification of semantic relations between nominals. In: Proceedings of the 4th International Workshop on Semantic Evaluations. 2007, 13–18
https://doi.org/10.3115/1621474.1621477
34 I Hendrickx, S N Kim, Z Kozareva, P Nakov, D Ó Séaghdha, S Padó, M Pennacchiotti, L Romano, S Szpakowicz. Semeval-2010 task 8: multiway classification of semantic relations between pairs of nominals. In: Proceedings of the Workshop on Semantic Evaluations: Recent Achievements and Future Directions. 2009, 94–99
https://doi.org/10.3115/1621969.1621986
35 Y Wang, X Liu, S Shi. Deep neural solver for math word problems. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2017, 845–854
https://doi.org/10.18653/v1/D17-1088
36 L Wang, D Zhang, J Zhang, X Xu, L Gao, B T Dai, H T Shen. Templatebased math word problem solvers with recursive neural networks. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 2019, 7144–7151
https://doi.org/10.1609/aaai.v33i01.33017144
37 D Zeng, K Liu, Y Chen, J Zhao. Distant supervision for relation extraction via piecewise convolutional neural networks. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 2015, 1753–1762
https://doi.org/10.18653/v1/D15-1203
38 S Hochreiter, J Schmidhuber. Long short-term memory. Neural Computation, 1997, 9(8): 1735–1780
https://doi.org/10.1162/neco.1997.9.8.1735
39 M T Luong, H Pham, C D Manning. Effective approaches to attentionbased neural machine translation. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 2015, 1412–1421
https://doi.org/10.18653/v1/D15-1166
40 Y Cao, H Li, P Luo, J Yao. Towards automatic numerical cross-checking: extracting formulas from text. In: Proceedings of the 2018 World Wide Web Conference. 2018, 1795–1804
https://doi.org/10.1145/3178876.3186166
41 A Vaswani, N Shazeer, N Parmar, J Uszkoreit, L Jones, A N Gomez, Ł Kaiser, I Polosukhin. Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 6000–6010
42 M D Zeiler. Adadelta: an adaptive learning rate method. 2012, arXiv preprint arXiv: 1212.5071
43 D Huang, J Yao, C Lin, Q Zhou, J Yin. Using intermediate representations to solve math word problems. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018, 419–428
https://doi.org/10.18653/v1/P18-1039
44 J Gu, Z Lu, H Li, V O Li. Incorporating copying mechanism in sequenceto- sequence learning. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016, 1631–1640
https://doi.org/10.18653/v1/P16-1154
45 G Klein, Y Kim, Y Deng, J Senellart, A Rush. OpenNMT: open-source toolkit for neural machine translation. In: Proceedings of Annual Meeting of the Association for Computational Linguistics, System Demonstrations. 2017, 67–72
https://doi.org/10.18653/v1/P17-4012
46 S Zheng, F Wang, H Bao, Y Hao, P Zhou, B Xu. Joint extraction of entities and relations based on a novel tagging scheme. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017, 1227–1236
https://doi.org/10.18653/v1/P17-1113
47 Y Cao, D Chen, H Li, P Luo. Nested relation extraction with iterative neural network. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 2019, 1001–1010
https://doi.org/10.1145/3357384.3358003
[1] Highlights Download
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed