A syntactic path-based hybrid neural network for negation scope detection

doi:10.1007/s11704-018-7368-6

Front. Comput. Sci.

2020, Vol. 14

Issue (1) : 84-94 https://doi.org/10.1007/s11704-018-7368-6

RESEARCH ARTICLE

A syntactic path-based hybrid neural network for negation scope detection

Lydia LAZIB, Bing QIN, Yanyan ZHAO(

), Weinan ZHANG, Ting LIU

Research Center for Social Computing and Information Retrieval, Harbin Institute of Technology, Harbin 150001, China

Download: PDF(414 KB)
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

The automatic detection of negation is a crucial task in a wide-range of natural language processing (NLP) applications, including medical data mining, relation extraction, question answering, and sentiment analysis. In this paper, we present a syntactic path-based hybrid neural network architecture, a novel approach to identify the scope of negation in a sentence. Our hybrid architecture has the particularity to capture salient information to determine whether a token is in the scope or not, without relying on any human intervention. This approach combines a bidirectional long shortterm memory (Bi-LSTM) network and a convolutional neural network (CNN). The CNN model captures relevant syntactic features between the token and the cue within the shortest syntactic path in both constituency and dependency parse trees. The Bi-LSTM learns the context representation along the sentence in both forward and backward directions. We evaluate our model on the Bioscope corpus, and get 90.82% F-score (78.31% PCS) on the abstract sub-corpus, outperforming features-dependent approaches.

Keywords natural language processing negation scope detection convolutional neural network recurrent neural network syntactic path

Corresponding Author(s): Yanyan ZHAO

Just Accepted Date: 14 June 2018 Online First Date: 06 August 2018 Issue Date: 24 September 2019

Cite this article:

Lydia LAZIB,Bing QIN,Yanyan ZHAO, et al. A syntactic path-based hybrid neural network for negation scope detection[J]. Front. Comput. Sci., 2020, 14(1): 84-94.

URL:

https://academic.hep.com.cn/fcs/EN/10.1007/s11704-018-7368-6
https://academic.hep.com.cn/fcs/EN/Y2020/V14/I1/84

1	R Morante, A Liekens, W Daelemans. Learning the scope of negation in biomedical texts. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics. 2008, 715–724 https://doi.org/10.3115/1613715.1613805
2	W Chapman, W Bridewell, P Hanbury, G F Cooper, B G Buchanan. A simple algorithm for identifying negated findings and diseases in discharge summaries. Journal of Biomedical Informatics, 2001, 34(5): 301–310 https://doi.org/10.1006/jbin.2001.1029
3	P G Mutalik, A Deshpande, P M Nadkarni. Use of general-purpose negation detection to augment concept indexing of medical documents. Journal of the American Medical Informatics Association, 2001, 8(6): 598–609 https://doi.org/10.1136/jamia.2001.0080598
4	Y Huang, H J Lowe. A novel hybrid approach to automated negation detection in clinical radiology reports. Journal of the American Medical Informatics Association, 2007, 14(3): 304–311 https://doi.org/10.1197/jamia.M2284
5	V Vincze, G Szarvas, R Farkas, G Mra, J Csirik. The BioScope corpus: biomedical texts annotated for uncertainty, negation and their scopes. BMC Bioinformatics, 2008, 9(11): S9 https://doi.org/10.1186/1471-2105-9-S11-S9
6	R Morante, W Daelemans. A metalearning approach to processing the scope of negation. In: Proceedings of the 13th Conference on Computational Natural Language Learning, Association for Computational Linguistics. 2009, 21–29 https://doi.org/10.3115/1596374.1596381
7	B Zou, G Zhou, Q Zhu. Tree kernel-based negation and speculation scope detection with structured syntactic parse features. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2013, 968–976
8	A Abu-Jbara, R Dragomir. Umichigan: a conditional random field model for resolving the scope of negation. In: Proceedings of the 1st Joint Conference on Lexical and Computational Semantics–Volume 1: Proceedings of the Main Conference and the Shared Task, and Volume 2: Proceedings of the 6th International Workshop on Semantic Evaluation, Association for Computational Linguistics. 2012, 328–334
9	S Agarwal, H Yu. Biomedical negation scope detection with conditional random fields. Journal of the American Medical Informatics Association, 2010, 17(6): 696–701 https://doi.org/10.1136/jamia.2010.003228
10	L Lazib, Y Zhao, B Qin, T Liu. Negation scope detection with conditional random field model. High Technology Letters, 2017, 23(2): 191–197
11	K Cho, B Van Merrinboer, D Bahdanau, Y Bengio. On the properties of neural machine translation: encoder-decoder approaches. In: Proceedings of the 8th Workshop on Syntax, Semantics and Sturucture in Statistical Translation (SSST-8). 2014
12	D Zeng, K Liu, Y Chen, J Zhao. Distant supervision for relation extraction via piecewise convolutional neural networks. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2015, 1753–1762 https://doi.org/10.18653/v1/D15-1203
13	D Tang, B Qin, T Liu. Document modeling with gated recurrent neural network for sentiment classification. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2015, 1426–1432 https://doi.org/10.18653/v1/D15-1167
14	L Lazib, Y Zhao, B Qin, T Liu. Negation scope detection with recurrent neural networks models in review texts. In: Proceedings of International Conference of Young Computer Scientists, Engineers and Educators. 2016, 494–508 https://doi.org/10.1007/978-981-10-2053-7_44
15	L Lazib, Y Zhao, B Qin, T Liu. Negation scope detection with recurrent neural networks models in review texts. International Journal of High Performance Computing and Networking, 2019, 13(2): 211–221 https://doi.org/10.1504/IJHPCN. 2016.10011341
16	Z Qian, P Li, Q Zhu, G Zhou, Z Luo, W Luo. Speculation and negation scope detection via convolutional neural networks. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2016, 815–825 https://doi.org/10.18653/v1/D16-1078
17	F Fancellu, A Lopez, B L Webber. Neural networks for negation scope detection. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016, 495–504 https://doi.org/10.18653/v1/P16-1047
18	M Schuster, K K Paliwal . Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 1997, 45(11): 2673–2681 https://doi.org/10.1109/78.650093
19	T Mikolov, M Karafit, L Burget, J Černocký, S Khudanpur. Recurrent neural network based language model. In: Proceedings of 11th Annual Conference of the International Speech Communication Association. 2010, 1045–1048
20	Y LeCun, Y Bengio. Convolutional networks for images, speech, and time series. The Handbook of Brain Theory and Neural Networks, 1995, 3361(10): 1995
21	Y Liu, F Wei, S Li, H Ji, M Zhou, H Wang. A dependency-based neural network for relation classification. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 2015, 285–290 https://doi.org/10.3115/v1/P15-2047
22	R Cai, X Zhang, H Wang. Bidirectional recurrent convolutional neural network for relation classification. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016, 756–765 https://doi.org/10.18653/v1/P16-1072
23	Y Xu, L Mou, G Li, Y Chen, H Peng, Z Jin. Classifying relations via long short-term memory networks along shortest dependency paths. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2015, 1785–1794 https://doi.org/10.18653/v1/D15-1206
24	L Øvrelid, E Velldal, S Oepen. Syntactic scope resolution in uncertainty analysis. In: Proceedings of the 23rd International Conference on Computational Linguistics. 2010, 1379–1387
25	J Lafferty, A McCallum, F C Pereira. Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of Probabilistic Models for Segmenting and Labeling Sequence Data. 2001, 282–289
26	J P White. UWashington: negation resolution using machine learning methods. In: Proceedings of the 1st Joint Conference on Lexical and Computational Semantics-Volume 1: Proceedings of the Main Conference and the Shared Task, and Volume 2: Proceedings of the 6th International Workshop on Semantic Evaluation. Association for Computational Linguistics. 2012, 335–339
27	Z Huang, W Xu, K Yu. Bidirectional LSTM-CRF models for sequence tagging. 2015, arXiv preprint arXiv:1508.01991
28	P Wang, Y Qian, F K Soong, L He , H Zhao. A unified tagging solution: bidirectional LSTM recurrent neural network with word embedding. 2015, arXiv preprint arXiv: 1511.00215
29	M Taboada, C Anthony, K Voll. Methods for creating semantic orientation dictionaries. In: Proceedings of the 5th Conference on Language Resources and Evaluation. 2006, 427–432
30	D Zeng, K Liu, S Lai, G Zhou, J Zhao. Relation classification via convolutional deep neural network. In: Proceedings of the 25th International Conference on Computational Linguistics: Technical Papers. 2014, 2335–2344
31	B Zhang, J Su, D Xiong, Y Lu, H Duan, J Yao. Shallow convolutional neural network for implicit discourse relation recognition. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2015, 2230–2235 https://doi.org/10.18653/v1/D15-1266
32	S Hochreiter, J Schmidhuber. Long short-term memory. Neural Computation, 1997, 9(8): 1735–1780 https://doi.org/10.1162/neco.1997.9.8.1735
33	A Graves, J Schmidhuber. Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Networks, 2005, 18(5): 602–610 https://doi.org/10.1016/j.neunet.2005.06.042
34	M Sundermeyer, R Schlter, H Ney. LSTM neural networks for language modeling. In: Proceedings of the 13th Annual Conference of the International Speech Communication Association. 2013, 194–197
35	R Kadari, Y Zhang, W Zhang, T Liu. CCG supertagging with bidirectional long short-term memory networks. Natural Language Engineering, 2018, 24(1): 77–90 https://doi.org/10.1017/S1351324917000250
36	R Kadari, Y Zhang, W Zhang, T Liu. CCG supertagging via Bidirectional LSTM-CRF neural architecture. Neurocomputing, 2018, 283: 31–37 https://doi.org/10.1016/j.neucom.2017.12.050
37	A Graves, N Jaitly. Towards end-to-end speech recognition with recurrent neural networks. In: Proceedings of the 31st International Conference on Machine Learning. 2014, 1764–1772
38	H Sak, A W Senior, F Beaufays. Long short-term memory recurrent neural network architectures for large scale acoustic modeling. In: Proceedings of the 15th Annual Conference of the International Speech Communication Association. 2014, 338–342
39	R Collobert, J Weston, L Bottou, M Karlen, K Kavukcuoglu, P Kuksa. Natural language processing (almost) from scratch. Journal of Machine Learning Research, 2011, 12(Aug): 2493–2537
40	N Collier, H S Park, N Ogata, Y Tateishi, C Nobata, T Ohta, T Sekimizu, H Imai, K Ibushi, J I Tsujii. The GENIA project: corpus-based knowledge acquisition and information extraction from genome research papers. In: Proceedings of the 9th Conference on European Chapter of the Association for Computational Linguistics. 1999, 271–272 https://doi.org/10.3115/977035.977081
41	F Chollet. Keras on GitHub, 2015
42	D Klein, C D Manning. Accurate unlexicalized parsing. In: Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics. 2003 https://doi.org/10.3115/1075096.1075150

[1]

Download

[1]	Huiying ZHANG, Yu ZHANG, Xin GENG. Practical age estimation using deep label distribution learning[J]. Front. Comput. Sci., 2021, 15(3): 153318-.
[2]	Qianchen YU, Zhiwen YU, Zhu WANG, Xiaofeng WANG, Yongzhi WANG. Estimating posterior inference quality of the relational infinite latent feature model for overlapping community detection[J]. Front. Comput. Sci., 2020, 14(6): 146323-.
[3]	Xin CHEN, He JIANG, Zhenyu CHEN, Tieke HE, Liming NIE. Automatic test report augmentation to assist crowdsourced testing[J]. Front. Comput. Sci., 2019, 13(5): 943-959.
[4]	Wenhao ZHENG, Hongyu ZHOU, Ming LI, Jianxin WU. CodeAttention: translating source code to comments by exploiting the code constructs[J]. Front. Comput. Sci., 2019, 13(3): 565-578.
[5]	Anna ZHU, Seiichi UCHIDA. Scene word recognition from pieces to whole[J]. Front. Comput. Sci., 2019, 13(2): 292-301.
[6]	Qingying SUN, Zhongqing WANG, Shoushan LI, Qiaoming ZHU, Guodong ZHOU. Stance detection via sentiment information and neural network model[J]. Front. Comput. Sci., 2019, 13(1): 127-138.
[7]	Jun ZHANG, Bineng ZHONG, Pengfei WANG, Cheng WANG, Jixiang DU. Robust feature learning for online discriminative tracking without large-scale pre-training[J]. Front. Comput. Sci., 2018, 12(6): 1160-1172.
[8]	Qianjun ZHANG, Lei ZHANG. Convolutional adaptive denoising autoencoders for hierarchical feature extraction[J]. Front. Comput. Sci., 2018, 12(6): 1140-1148.
[9]	Lili HUANG, Jiefeng PENG, Ruimao ZHANG, Guanbin LI, Liang LIN. Learning deep representations for semantic image parsing: a comprehensive overview[J]. Front. Comput. Sci., 2018, 12(5): 840-857.
[10]	Zhongqing WANG, Shoushan LI, Guodong ZHOU. Personal summarization from profile networks[J]. Front. Comput. Sci., 2017, 11(6): 1085-1097.
[11]	Yang-Yen OU, Ta-Wen KUAN, Anand PAUL, Jhing-Fa WANG, An-Chao TSAI. Spoken dialog summarization system with HAPPINESS/SUFFERING factor recognition[J]. Front. Comput. Sci., 2017, 11(3): 429-443.
[12]	Kai CHEN,Guiguang DING,Jungong HAN. Attribute-based supervised deep learning model for action recognition[J]. Front. Comput. Sci., 2017, 11(2): 219-229.
[13]	Feifei ZHANG,Yongbin YU,Qirong MAO,Jianping GOU,Yongzhao ZHAN. Pose-robust feature learning for facial expression recognition[J]. Front. Comput. Sci., 2016, 10(5): 832-844.
[14]	Yi ZHENG,Qi LIU,Enhong CHEN,Yong GE,J. Leon ZHAO. Exploiting multi-channels deep convolutional neural networks for multivariate time series classification[J]. Front. Comput. Sci., 2016, 10(1): 96-112.
[15]	Wenge RONG,Baolin PENG,Yuanxin OUYANG,Chao LI,Zhang XIONG. Structural information aware deep semi-supervised recurrent neural network for sentiment analysis[J]. Front. Comput. Sci., 2015, 9(2): 171-184.

Viewed

Full text

Abstract

Cited

Shared

Discussed