融合篇章结构位置编码的神经机器翻译

亢晓勉; 宗成庆

doi:10.11959/j.issn.2096-6652.202016

您当前的位置：

首页 >

文章列表页 >

融合篇章结构位置编码的神经机器翻译

学术论文 | 更新时间：2024-06-05

- 融合篇章结构位置编码的神经机器翻译
- Fusion of discourse structural position encoding for neural machine translation
- 智能科学与技术学报 2020年2卷第2期页码：144-152
- 作者机构：
  
  1. 中国科学院自动化研究所模式识别国家重点实验室，北京 100190
  2. 中国科学院大学，北京 100049
- 作者简介：
  
  [ "亢晓勉（1991- ），男，中国科学院自动化研究所模式识别国家重点实验室博士生，主要研究方向为机器翻译、篇章分析" ]
  [ "宗成庆（1963- ），男，博士，中国科学院自动化研究所模式识别国家重点实验室研究员、博士生导师，主要研究方向为机器翻译、自然语言处理和文本数据挖掘等" ]
- 基金信息：
  
  国家自然科学基金资助项目;The National Natural Science Foundation of China(U1836221)
- DOI：10.11959/j.issn.2096-6652.202016
  中图分类号： TP391
- 网络出版日期：2020-06，
  
  纸质出版日期：2020-06-20
- 稿件说明：
移动端阅览
亢晓勉, 宗成庆. 融合篇章结构位置编码的神经机器翻译[J]. 智能科学与技术学报, 2020,2(2):144-152.

Xiaomian KANG, Chengqing ZONG. Fusion of discourse structural position encoding for neural machine translation[J]. Chinese journal of intelligent science and technology, 2020, 2(2): 144-152.
亢晓勉, 宗成庆. 融合篇章结构位置编码的神经机器翻译[J]. 智能科学与技术学报, 2020,2(2):144-152. DOI： 10.11959/j.issn.2096-6652.202016.

Xiaomian KANG, Chengqing ZONG. Fusion of discourse structural position encoding for neural machine translation[J]. Chinese journal of intelligent science and technology, 2020, 2(2): 144-152. DOI： 10.11959/j.issn.2096-6652.202016.

摘要

现有的文档级神经机器翻译方法在翻译一个句子时大多只利用文档的上下文词汇信息，而忽视了跨句子的篇章语义单元之间的结构关系。针对此问题，提出了多种篇章结构位置编码策略，利用基于修辞结构理论的篇章树结构，对篇章树上位于不同篇章单元的单词之间的位置关系进行了表示。实验表明，通过位置编码的方式，在基于Transformer框架的神经机器翻译模型中有效地融合了源端的篇章结构信息，译文质量得到了显著提升。

Abstract

Most of existing document-level neural machine translation (DocNMT) methods focus on exploring the utilization of the lexical information of context

which ignore the structural relationships among the cross-sentence discourse semantic units.Therefore

multiple discourse structural position encoding strategies were proposed to represent the positional relationships among the words in discourse units over the discourse tree based on rhetorical structure theory (RST).Experimental results show that the source-side discourse structural position information is effectively fused into the DocNMT models underlying the Transformer architecture by the position encoding

and the translation quality is improved significantly.

关键词

Keywords

references

郑南宁 . 人工智能新时代 [J ] . 智能科学与技术学报 , 2019 , 1 ( 1 ): 1 - 3 .

ZHENG N N . The new era of artificial intelligence [J ] . Chinese Journal of Intelligent Science and Technology , 2019 , 1 ( 1 ): 1 - 3 .

张钹 . 人工智能进入后深度学习时代 [J ] . 智能科学与技术学报 , 2019 , 1 ( 1 ): 4 - 6 .

ZHANG B . Artificial intelligence is entering the post deep-learning era [J ] . Chinese Journal of f Intelligent Science And Technology , 2019 , 1 ( 1 ): 4 - 6 .

宗成庆 . 统计自然语言处理 [M ] . 北京 : 清华大学出版社 , 2013 .

ZONG C Q . Statistical natural language processing [M ] . Beijing : Tsinghua University PressPress , 2013 .

杜倩龙 , 宗成庆 , 苏克毅 . 利用上下文相似度增强词对齐效果的自然语言推理方法 [J ] . 智能科学与技术学报 , 2020 , 2 ( 1 ): 26 - 35 .

DU Q L , ZONG C Q , SU K Y . Enhancing alignment with context similarity for natural language inference [J ] . Chinese Journal of Intelligent Science and Technology , 2020 , 2 ( 1 ): 26 - 35 .

BROWN G , BROWN G D , BROWN G R , et al . Discourse analysis [M ] . Cambridge : Cambridge University PressPress , 1983 .

HALLIDAY M A K , HASAN R . Cohesion in English [M ] . >[S.l.] . RoutledgePress , 2014 .

GONG Z X , ZHANG M , ZHOU G D . Cache-based document-level statistical machine translation [C ] // The Conference on Empirical Methods in Natural Language Processing . Stroudsburg:ACL , 2011 : 909 - 919 .

XIONG D , DING Y , ZHANG M , et al . Lexical chain based cohesion models for document-level statistical machine translation [C ] // The Conference on Empirical Methods in Natural Language Processing . Stroudsburg:ACL , 2013 : 1563 - 1573 .

TU M , ZHOU Y , ZONG C Q . Enhancing grammatical cohesion:generating transitional expressions for SMT [C ] // The Annual Meeting of the Association for Computational Linguistics . Stroudsburg:ACL , 2014 : 850 - 860 .

HARDMEIER C . Discourse in statistical machine translation [D ] . Uppsala:Uppsala University , 2014 .

WANG L Y , TU Z P , WAY A , et al . Exploiting cross-sentence context for neural machine translation [C ] // The Conference on Empirical Methods in Natural Language Processing . Stroudsburg:ACL , 2017 : 2826 - 2831 .

VOITA E , SERDYUKOV P , SENNRICH R , et al . Context-aware neural machine translation learns anaphora resolution [C ] // The Annual Meeting of the Association for Computational Linguistics . Stroudsburg:ACL , 2019 : 1264 - 1274 .

ZHANG J C , LUAN H B , SUN M S , et al . Improving the transformer translation model with document-level context [C ] // The Conference on Empirical Methods in Natural Language Processing . Stroudsburg:ACL , 2018 : 533 - 542 .

MICULICICH L , RAM D , PAPPAS N , et al . Document-level neural machine translation with hierarchical attention networks [C ] // The Conference on Empirical Methods in Natural Language Processing . Stroudsburg:ACL , 2018 : 2947 - 2954 .

YANG Z X , ZHANG J C , MENG F D , et al . Enhancing context modeling with a query-guided capsule network for document-level translation [C ] // The Conference on Empirical Methods in Natural Language Processing . Stroudsburg:ACL , 2019 : 1527 - 1537 .

TU Z P , LIU Y , SHI S M , et al . Learning to remember translation history with a continuous cache [J ] . Transactions of the ACL , 2018 , 6 : 407 - 420 .

XIONG H , HE Z J , WU H , et al . Modeling coherence for discourse neural machine translation [C ] // The AAAI Conference on Artificial Intelligence . Palo Alto:AAAI Press , 2019 : 7338 - 7345 .

MARUF S , MARTINS A F T , HAFFARI G . Selective attention for context-aware neural machine translation [C ] // TheConference of the North American Chapter of the ACL . Stroudsburg:ACL , 2019 : 3092 - 3102 .

BAWDEN R , SENNRICH R , BIRCH A , et al . Evaluating discourse phenomena in neural machine translation [C ] // The Conference of the North American Chapter of the ACL . Stroudsburg:ACL , 2018 : 1304 - 1313 .

VOITA E , SENNRICH R , TITOV I . When a good translation is wrong in context:Context-aware machine translation improves on deixis,ellipsis,and lexical cohesion [C ] // The Annual Meeting of the Associationfor Computational Linguistics . Stroudsburg:ACL , 2019 : 1198 - 1212 .

KANG X M , ZONG C Q , XUE N W . A survey of discourse representations for Chinese discourse annotation [J ] . ACM Transactions on Asian and Low-Resource Language Information Processing , 2019 , 18 ( 3 ): 1 - 25 .

ROTHWELL A D . Thematic progression as a functional resource in analysing texts [J ] . Circulo de Linguistica Aplicada a la Communication , 2001 ,( 5 ):2.

ASHER N , ALEX L . Logics of conversation [M ] . Cambridge : Cambridge University PressPress , 2003 .

MANN W C , THOMPSON S A . Rhetorical structure theory:toward a functional theory of text organization [J ] . Text ＆ Talk , 1988 , 8 ( 3 ): 243 - 281 .

HERNAULT H , PRENDINGER H , DUVERLE D A . HILDA:adiscourse parser using support vector machine classification [J ] . Dialogue＆ Discourse , 2010 , 1 ( 3 ): 1 - 33 .

FENG V W , HIRST G . Text-level discourse parsing with rich linguistic features [C ] // The Annual Meeting of the Association for Computational Linguistics . Stroudsburg:ACL , 2012 : 60 - 68 .

JI Y F , EISENSTEIN J . Representation learning for text-level discourse parsing [C ] // The Annual Meeting of the Association for Computational Linguistics . Stroudsburg:ACL , 2014 : 13 - 24 .

BHATIA P , JI Y F , EISENSTEIN J . Better document-level sentiment analysis from RST discourse parsing [C ] // The Conference on Empirical Methods in Natural Language Processing . Stroudsburg:ACL , 2015 : 2212 - 2218 .

GERANI S , MEHDAD Y , CARENINI G , et al . Abstractive summarization of product reviews using discourse structure [C ] // The Conference on Empirical Methods in Natural Language Processing . Stroudsburg:ACL , 2014 : 1602 - 1613 .

GUZMAN F , JOTY S , LIUIS A , et al . Using discourse structure improves machine translation evaluation [C ] // The Annual Meeting of the Association for Computational Linguistics . Stroudsburg:ACL , 2014 : 687 - 698 .

TU M , ZHOU Y , ZONG C Q . A novel translation framework based on rhetorical structure theory [C ] // The Annual Meeting of the Association for Computational Linguistics . Stroudsburg:ACL , 2013 : 370 - 374 .

VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all you need [C ] // The 31st Annual Conference on Advances in Neural Information Processing Systems . Boston:MIT Press , 2017 : 5998 - 6008 .

BAHDANAU D , CHO K , BENGIO Y . Neural machine translation by jointly learning to align and translate [J ] . arXiv:1409.0473 , 2014

GEHRING J , AULI M , GRAGIER D , et al . Convolutional sequence to sequence learning [C ] // The 34th International Conference on Machine Learning . New York:ACM Press , 2017 : 1243 - 1252 .

SHAW P , USZKOREIT J , VASWANI A . Self-attention with relative position representations [C ] // The Conference of the North American Chapter of the Association for Computational Linguistics . Stroudsburg:ACL , 2018 : 464 - 468

WANG X , TU Z P , WANG L Y , et al . Self-attention with structural position representations [C ] // The Conference on Empirical Methods in Natural Language Processing . Stroudsburg:ACL , 2019 : 1403 - 1409 .

SENNRICH R , DADDOW B , BIRCH A . Neural machine translation of rare words with subword units [C ] // The Annual Meeting of the Associationfor Computational Linguistics . Stroudsburg:ACL , 2016 : 1715 - 1725 .

浏览量

270

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

暂无数据