Please wait a minute...
Frontiers of Computer Science

ISSN 2095-2228

ISSN 2095-2236(Online)

CN 10-1014/TP

Postal Subscription Code 80-970

2018 Impact Factor: 1.129

Front. Comput. Sci.    2021, Vol. 15 Issue (1) : 151302    https://doi.org/10.1007/s11704-019-9164-3
RESEARCH ARTICLE
Improving neural sentence alignment with word translation
Ying DING, Junhui LI, Zhengxian GONG(), Guodong ZHOU
School of Computer Science and Technology, Soochow University, Suzhou 215006, China
 Download: PDF(519 KB)  
 Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks
Abstract

Sentence alignment is a basic task in natural language processing which aims to extract high-quality parallel sentences automatically. Motivated by the observation that aligned sentence pairs contain a larger number of aligned words than unaligned ones, we treat word translation as one of the most useful external knowledge. In this paper, we show how to explicitly integrate word translation into neural sentence alignment. Specifically, this paper proposes three cross-lingual encoders to incorporate word translation: 1) Mixed Encoder that learns words and their translation annotation vectors over sequences where words and their translations are mixed alternatively; 2) Factored Encoder that views word translations as features and encodes words and their translations by concatenating their embeddings; and 3) Gated Encoder that uses gate mechanism to selectively control the amount of word translations moving forward. Experimentation on NIST MT and Opensubtitles Chinese-English datasets on both non-monotonicity and monotonicity scenarios demonstrates that all the proposed encoders significantly improve sentence alignment performance.

Keywords sentence alignment      word translation      mixed encoder      factored encoder      gated encoder     
Corresponding Author(s): Zhengxian GONG   
Just Accepted Date: 18 September 2019   Issue Date: 10 October 2020
 Cite this article:   
Ying DING,Junhui LI,Zhengxian GONG, et al. Improving neural sentence alignment with word translation[J]. Front. Comput. Sci., 2021, 15(1): 151302.
 URL:  
https://academic.hep.com.cn/fcs/EN/10.1007/s11704-019-9164-3
https://academic.hep.com.cn/fcs/EN/Y2021/V15/I1/151302
1 D Bahdanau, K Cho, Y Bengio. Neural machine translation by jointly learning to align and translate. In: Proceedings of International Conference on Learning Representations. 2015
2 A Vaswani, N Shazeer, N Parmar, J Uszkoreit, L Jones, A Gomez, L Kaiser, I Polosukhin. Attention is all you need. In: Proceedings of the 31st Conference on Neural Information Processing Systems. 2017, 6000–6010
3 K M Hermann, P Blunsom. Multilingual models for compositional distributed semantics. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. 2014, 58–68
https://doi.org/10.3115/v1/P14-1006
4 J Y Nie, M Simard, P Isabelle, R Durand. Cross-language information retrieval based on parallel texts and automatic mining of parallel texts from the web. In: Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 1999, 74–81
https://doi.org/10.1145/312624.312656
5 G D S Martino, S Romeo, A Barro�n-Cedeno, S Joty, L Marquez, A Moschitti, P Nakov. Cross-language question re-ranking. In: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2017, 1145–1148
6 D Wu. Alignment. Handbook of Natural Language Processing. CRC Press. 2010
7 F Gregoire, P Langlais. A deep neural network approach to parallel sentence extraction. 2017, arXiv preprint arXiv:1709.09783
8 J Grover, P Mitra. Bilingual word embeddings with bucketed CNN for parallel sentence extraction. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics–Student Research Workshop. 2017, 11–16
https://doi.org/10.18653/v1/P17-3003
9 Y Ding, J Li, G Zhou. Word-pair relevance network for sentence alignment. Journal of Chinese Information Processing, 2019
10 Y Ding, J H Li, Z X Gong, G D Zhou. Word-pair relevance modeling with multi-view neural attention mechanism for sentence alignment. Journal of Computer Science and Technology, 2019
11 L Liu, M Utiyama, A Finch, E Sumita. Neural machine translation with supervised attention. In: Proceedings of the 26th International Conference on Computational Linguistics: Technical Papers. 2016, 3093–3102
12 H Mi, Z Wang, A Ittycheriah. Supervised attentions for neural machine translation. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 2016, 2283–2288
https://doi.org/10.18653/v1/D16-1249
13 P Arthur, G Neubig, S Nakamura. Incorporating discrete translation lexicons into neural machine translation. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 2016, 1557–1567
https://doi.org/10.18653/v1/D16-1162
14 W A Gale, K W Church. A program for aligning sentences in bilingual corpora. In: Proceedings of the Meeting on the Association for Computational Linguistics. 1991, 177–184
https://doi.org/10.3115/981344.981367
15 S F Chen. Aligning sentences in bilingual corpora using lexical information. Computer Knowledge & Technology, 1993, 46(3): 9–16
https://doi.org/10.3115/981574.981576
16 D Wu. Aligning a parallel English-Chinese corpus statistically with lexical criteria. Computer Science, 1994, 4(4): 80–87
https://doi.org/10.3115/981732.981744
17 R C Moore. Fast and accurate sentence alignment of bilingual corpora. In: Processing of the 5th Conference of the Association for Machine Translation in the Americas. 2002, 135–144
https://doi.org/10.1007/3-540-45820-4_14
18 P F Brown, V J D Pietra, S A D Pietra, R L Mercer. The mathematics of statistical machine translation: parameter estimation. Computational linguistics, 1993, 19(2): 263–311
19 F Braune, A Fraser. Improved unsupervised sentence alignment for symmetrical and asymmetrical parallel corpora. In: Processings of the 23rd International Conference on Computational Linguistics. 2010, 81–89
20 X Ma. Champollion: a robust parallel text sentence aligner. In: Processings of the 5th International Conference on Language Resources and Evaluation. 2006, 489–492
21 P Li, M Sun, P Xue. Fast-Champollion: a fast and robust sentence alignment algorithm. In: Proceedings of the 23rd International Conference on Computational Linguistics. 2010, 710–718
22 X Quan, C Kit, Y Song. Non-monotonic sentence alignment via semisupervised learning. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics. 2013, 622–630
23 R Chatterjee, M Negri, M Turchi, M Federico, L Specia, F Blain. Guiding neural machine translation decoding with external knowledge. In: Proceedings of the 2nd Conference on Machine Translation. 2017, 157–168
https://doi.org/10.18653/v1/W17-4716
24 T Q Nguyen, D Chiang. Improving lexical choice in neural machine translation. 2017, arXiv preprint arXiv:1710.01329
https://doi.org/10.18653/v1/N18-1031
25 X Wang, Z Tu, D Xiong, M Zhang. Translating phrases in neural machine translation. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2017, 1421–1431
https://doi.org/10.18653/v1/D17-1149
26 X Wang, Z Lu, Z Tu, H Li, D Xiong, M Zhang. Neural machine translation advised by statistical machine translation. In: Proceedings of the 31st AAAI Conference on Artificial Intelligence. 2017, 3330–3336
27 K Chen, R Wang, M Utiyama, L Liu, A Tamura, E Sumita, T Zhao. Neural machine translation with source dependency representation. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2017, 2846–2852
https://doi.org/10.18653/v1/D17-1304
28 J Li, D Xiong, Z Tu, M Zhu, M Zhang, G Zhou. Modeling source syntax for neural machine translation. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017, 688–697
https://doi.org/10.18653/v1/P17-1064
29 A Eriguchi, K Hashimoto, Y Tsuruoka. Tree-to-sequence attentional neural machine translation. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016, 823–833
https://doi.org/10.18653/v1/P16-1078
30 R Sennrich, B Haddow. Linguistic input features improve neural machine translation. In: Proceedings of the 1st Conference on Machine Translation. 2016, 83–91
https://doi.org/10.18653/v1/W16-2209
31 R Aharoni, Y Goldberg. Towards string-to-tree neural machine translation. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Short Papers). 2017, 132–140
https://doi.org/10.18653/v1/P17-2021
32 H Chen, S Huang, D Chiang, J Chen. Improved neural machine translation with a syntax-aware encoder and decoder. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017, 1936–1945
https://doi.org/10.18653/v1/P17-1177
33 D Han, J Li, Y Li, M Zhang, G Zhou. Explicitly modeling word translations in neural machine translation. ACM Transactions on Asian and Low-Resource Language Information Processing, 2019, 19(1): 1–17
https://doi.org/10.1145/3342353
34 P Langlais, M Simard, J Veronis. Methods and practical issues in evaluating alignment techniques. In: Processings of the 36th Annual Meeting of the Association for Computational Linguistics and the 17th International Conference on Computational Linguistics. 1998, 711–717
https://doi.org/10.3115/980845.980964
35 C Kit, J J Webster, K K Sin, H Pan, H Li. Clause alignment for bilingual Hong Kong legal texts: a lexicalbased approach. International Journal of Corpus Linguistics, 2004, 9(1): 29–52
https://doi.org/10.1075/ijcl.9.1.02kit
36 F J Och, H Ney. A systematic comparison of various statistical alignment models. Computational Linguistics, 2003, 29(1): 19–51
https://doi.org/10.1162/089120103321337421
37 K Cho, B van Merrienboer, C Gulcehre, D Bahdanau, F Bougares, H Schwenk, Y Bengio. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 2014, 1724–1734
https://doi.org/10.3115/v1/D14-1179
38 I Sutskever, R Salakhutdinov, J B Tenenbaum. Modelling relational data using bayesian clustered tensor factorization. In: Proceedings of the 22nd International Conference on Neural Information Processing Systems. 2009, 1821–1828
39 R Jenatton, N L Roux, A Bordes, G Obozinski. A latent factor model for highly multi-relational data. In: Proceedings of the 25th International Conference on Neural Information Processing Systems. 2012, 3167–3175
40 R Collobert, J Weston. A unified architecture for natural language processing: deep neural networks with multitask learning. In: Proceedings of the 25th International Conference on Machine Learning. 2008, 160–167
https://doi.org/10.1145/1390156.1390177
41 W Y Zou, R Socher, D Cer, C D Manning. Bilingual word embeddings for phrase-based machine translation. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. 2013, 1393–1398
[1] Article highlights Download
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed