Fusion of discourse structural position encoding for neural machine translation

Xiaomian KANG; Chengqing ZONG

doi:10.11959/j.issn.2096-6652.202016

您当前的位置：

首页 >

文章列表页 >

Fusion of discourse structural position encoding for neural machine translation

Regular Papers | 更新时间：2024-06-05

- Fusion of discourse structural position encoding for neural machine translation
- Chinese Journal of Intelligent Science and Technology Vol. 2, Issue 2, Pages: 144-152(2020)
- 作者机构：
  
  1. 中国科学院自动化研究所模式识别国家重点实验室，北京 100190
  2. 中国科学院大学，北京 100049
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-6652.202016
  CLC： TP391
- Published Online：2020-06，
  
  Published：20 June 2020
- 稿件说明：
移动端阅览
Xiaomian KANG, Chengqing ZONG. Fusion of discourse structural position encoding for neural machine translation[J]. Chinese journal of intelligent science and technology, 2020, 2(2): 144-152.
DOI：

Xiaomian KANG, Chengqing ZONG. Fusion of discourse structural position encoding for neural machine translation[J]. Chinese journal of intelligent science and technology, 2020, 2(2): 144-152. DOI： 10.11959/j.issn.2096-6652.202016.

摘要

现有的文档级神经机器翻译方法在翻译一个句子时大多只利用文档的上下文词汇信息，而忽视了跨句子的篇章语义单元之间的结构关系。针对此问题，提出了多种篇章结构位置编码策略，利用基于修辞结构理论的篇章树结构，对篇章树上位于不同篇章单元的单词之间的位置关系进行了表示。实验表明，通过位置编码的方式，在基于Transformer框架的神经机器翻译模型中有效地融合了源端的篇章结构信息，译文质量得到了显著提升。

Abstract

Most of existing document-level neural machine translation (DocNMT) methods focus on exploring the utilization of the lexical information of context

which ignore the structural relationships among the cross-sentence discourse semantic units.Therefore

multiple discourse structural position encoding strategies were proposed to represent the positional relationships among the words in discourse units over the discourse tree based on rhetorical structure theory (RST).Experimental results show that the source-side discourse structural position information is effectively fused into the DocNMT models underlying the Transformer architecture by the position encoding

and the translation quality is improved significantly.

关键词

Keywords

references

郑南宁 . 人工智能新时代 [J ] . 智能科学与技术学报 , 2019 , 1 ( 1 ): 1 - 3 .

ZHENG N N . The new era of artificial intelligence [J ] . Chinese Journal of Intelligent Science and Technology , 2019 , 1 ( 1 ): 1 - 3 .

张钹 . 人工智能进入后深度学习时代 [J ] . 智能科学与技术学报 , 2019 , 1 ( 1 ): 4 - 6 .

ZHANG B . Artificial intelligence is entering the post deep-learning era [J ] . Chinese Journal of f Intelligent Science And Technology , 2019 , 1 ( 1 ): 4 - 6 .

宗成庆 . 统计自然语言处理 [M ] . 北京 : 清华大学出版社 , 2013 .

ZONG C Q . Statistical natural language processing [M ] . Beijing : Tsinghua University PressPress , 2013 .

杜倩龙 , 宗成庆 , 苏克毅 . 利用上下文相似度增强词对齐效果的自然语言推理方法 [J ] . 智能科学与技术学报 , 2020 , 2 ( 1 ): 26 - 35 .

DU Q L , ZONG C Q , SU K Y . Enhancing alignment with context similarity for natural language inference [J ] . Chinese Journal of Intelligent Science and Technology , 2020 , 2 ( 1 ): 26 - 35 .

BROWN G , BROWN G D , BROWN G R , et al . Discourse analysis [M ] . Cambridge : Cambridge University PressPress , 1983 .

HALLIDAY M A K , HASAN R . Cohesion in English [M ] . >[S.l.] . RoutledgePress , 2014 .

GONG Z X , ZHANG M , ZHOU G D . Cache-based document-level statistical machine translation [C ] // The Conference on Empirical Methods in Natural Language Processing . Stroudsburg:ACL , 2011 : 909 - 919 .

XIONG D , DING Y , ZHANG M , et al . Lexical chain based cohesion models for document-level statistical machine translation [C ] // The Conference on Empirical Methods in Natural Language Processing . Stroudsburg:ACL , 2013 : 1563 - 1573 .

TU M , ZHOU Y , ZONG C Q . Enhancing grammatical cohesion:generating transitional expressions for SMT [C ] // The Annual Meeting of the Association for Computational Linguistics . Stroudsburg:ACL , 2014 : 850 - 860 .

HARDMEIER C . Discourse in statistical machine translation [D ] . Uppsala:Uppsala University , 2014 .

WANG L Y , TU Z P , WAY A , et al . Exploiting cross-sentence context for neural machine translation [C ] // The Conference on Empirical Methods in Natural Language Processing . Stroudsburg:ACL , 2017 : 2826 - 2831 .

VOITA E , SERDYUKOV P , SENNRICH R , et al . Context-aware neural machine translation learns anaphora resolution [C ] // The Annual Meeting of the Association for Computational Linguistics . Stroudsburg:ACL , 2019 : 1264 - 1274 .

ZHANG J C , LUAN H B , SUN M S , et al . Improving the transformer translation model with document-level context [C ] // The Conference on Empirical Methods in Natural Language Processing . Stroudsburg:ACL , 2018 : 533 - 542 .

MICULICICH L , RAM D , PAPPAS N , et al . Document-level neural machine translation with hierarchical attention networks [C ] // The Conference on Empirical Methods in Natural Language Processing . Stroudsburg:ACL , 2018 : 2947 - 2954 .

YANG Z X , ZHANG J C , MENG F D , et al . Enhancing context modeling with a query-guided capsule network for document-level translation [C ] // The Conference on Empirical Methods in Natural Language Processing . Stroudsburg:ACL , 2019 : 1527 - 1537 .

TU Z P , LIU Y , SHI S M , et al . Learning to remember translation history with a continuous cache [J ] . Transactions of the ACL , 2018 , 6 : 407 - 420 .

XIONG H , HE Z J , WU H , et al . Modeling coherence for discourse neural machine translation [C ] // The AAAI Conference on Artificial Intelligence . Palo Alto:AAAI Press , 2019 : 7338 - 7345 .

MARUF S , MARTINS A F T , HAFFARI G . Selective attention for context-aware neural machine translation [C ] // TheConference of the North American Chapter of the ACL . Stroudsburg:ACL , 2019 : 3092 - 3102 .

BAWDEN R , SENNRICH R , BIRCH A , et al . Evaluating discourse phenomena in neural machine translation [C ] // The Conference of the North American Chapter of the ACL . Stroudsburg:ACL , 2018 : 1304 - 1313 .

VOITA E , SENNRICH R , TITOV I . When a good translation is wrong in context:Context-aware machine translation improves on deixis,ellipsis,and lexical cohesion [C ] // The Annual Meeting of the Associationfor Computational Linguistics . Stroudsburg:ACL , 2019 : 1198 - 1212 .

KANG X M , ZONG C Q , XUE N W . A survey of discourse representations for Chinese discourse annotation [J ] . ACM Transactions on Asian and Low-Resource Language Information Processing , 2019 , 18 ( 3 ): 1 - 25 .

ROTHWELL A D . Thematic progression as a functional resource in analysing texts [J ] . Circulo de Linguistica Aplicada a la Communication , 2001 ,( 5 ):2.

ASHER N , ALEX L . Logics of conversation [M ] . Cambridge : Cambridge University PressPress , 2003 .

MANN W C , THOMPSON S A . Rhetorical structure theory:toward a functional theory of text organization [J ] . Text ＆ Talk , 1988 , 8 ( 3 ): 243 - 281 .

HERNAULT H , PRENDINGER H , DUVERLE D A . HILDA:adiscourse parser using support vector machine classification [J ] . Dialogue＆ Discourse , 2010 , 1 ( 3 ): 1 - 33 .

FENG V W , HIRST G . Text-level discourse parsing with rich linguistic features [C ] // The Annual Meeting of the Association for Computational Linguistics . Stroudsburg:ACL , 2012 : 60 - 68 .

JI Y F , EISENSTEIN J . Representation learning for text-level discourse parsing [C ] // The Annual Meeting of the Association for Computational Linguistics . Stroudsburg:ACL , 2014 : 13 - 24 .

BHATIA P , JI Y F , EISENSTEIN J . Better document-level sentiment analysis from RST discourse parsing [C ] // The Conference on Empirical Methods in Natural Language Processing . Stroudsburg:ACL , 2015 : 2212 - 2218 .

GERANI S , MEHDAD Y , CARENINI G , et al . Abstractive summarization of product reviews using discourse structure [C ] // The Conference on Empirical Methods in Natural Language Processing . Stroudsburg:ACL , 2014 : 1602 - 1613 .

GUZMAN F , JOTY S , LIUIS A , et al . Using discourse structure improves machine translation evaluation [C ] // The Annual Meeting of the Association for Computational Linguistics . Stroudsburg:ACL , 2014 : 687 - 698 .

TU M , ZHOU Y , ZONG C Q . A novel translation framework based on rhetorical structure theory [C ] // The Annual Meeting of the Association for Computational Linguistics . Stroudsburg:ACL , 2013 : 370 - 374 .

VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all you need [C ] // The 31st Annual Conference on Advances in Neural Information Processing Systems . Boston:MIT Press , 2017 : 5998 - 6008 .

BAHDANAU D , CHO K , BENGIO Y . Neural machine translation by jointly learning to align and translate [J ] . arXiv:1409.0473 , 2014

GEHRING J , AULI M , GRAGIER D , et al . Convolutional sequence to sequence learning [C ] // The 34th International Conference on Machine Learning . New York:ACM Press , 2017 : 1243 - 1252 .

SHAW P , USZKOREIT J , VASWANI A . Self-attention with relative position representations [C ] // The Conference of the North American Chapter of the Association for Computational Linguistics . Stroudsburg:ACL , 2018 : 464 - 468

WANG X , TU Z P , WANG L Y , et al . Self-attention with structural position representations [C ] // The Conference on Empirical Methods in Natural Language Processing . Stroudsburg:ACL , 2019 : 1403 - 1409 .

SENNRICH R , DADDOW B , BIRCH A . Neural machine translation of rare words with subword units [C ] // The Annual Meeting of the Associationfor Computational Linguistics . Stroudsburg:ACL , 2016 : 1715 - 1725 .

Views

270

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

Postal code：100079
Tel：（010）53879206 Email：cjist@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：35685 Visits Today：160

⁰