利用上下文相似度增强词对齐效果的自然语言推理方法

杜倩龙; 宗成庆; 苏克毅

doi:10.11959/j.issn.2096-6652.202003

您当前的位置：

首页 >

文章列表页 >

利用上下文相似度增强词对齐效果的自然语言推理方法

学术论文 | 更新时间：2024-06-05

- 利用上下文相似度增强词对齐效果的自然语言推理方法
- Enhancing alignment with context similarity for natural language inference
- 智能科学与技术学报 2020年2卷第1期页码：26-35
- 作者机构：
  
  1. 中国科学院自动化研究所模式识别国家重点实验室，北京 100190
  2. 中国科学院大学，北京 100049
  3. 台湾“中央研究院”资讯科学研究所，台湾台北 11529
- 作者简介：
  
  [ "杜倩龙（1989- ），男，中国科学院自动化研究所博士生，主要研究方向为词义消歧、自然语言推理" ]
  [ "宗成庆（1963- ），男，博士，中国科学院自动化研究所研究员，中国科学院大学岗位教授、博士生导师，主要研究方向为自然语言处理理论与方法、机器翻译、文本数据挖掘、人机对话系统" ]
  [ "苏克毅（1955- ），男，博士，台湾“中央研究院”资讯科学研究所研究员，主要研究方向为机器翻译、机器阅读和自然语言理解" ]
- 基金信息：
  
  国家重点研发计划基金资助项目;The National Key Research and Development Program of China(2017YFB1002103)
- DOI：10.11959/j.issn.2096-6652.202003
  中图分类号： TP391.1
- 网络出版日期：2020-03，
  
  纸质出版日期：2020-03-20
- 稿件说明：
移动端阅览
杜倩龙, 宗成庆, 苏克毅. 利用上下文相似度增强词对齐效果的自然语言推理方法[J]. 智能科学与技术学报, 2020,2(1):26-35.

Qianlong DU, Chengqing ZONG, Keh-Yih SU. Enhancing alignment with context similarity for natural language inference[J]. Chinese journal of intelligent science and technology, 2020, 2(1): 26-35.
杜倩龙, 宗成庆, 苏克毅. 利用上下文相似度增强词对齐效果的自然语言推理方法[J]. 智能科学与技术学报, 2020,2(1):26-35. DOI： 10.11959/j.issn.2096-6652.202003.

Qianlong DU, Chengqing ZONG, Keh-Yih SU. Enhancing alignment with context similarity for natural language inference[J]. Chinese journal of intelligent science and technology, 2020, 2(1): 26-35. DOI： 10.11959/j.issn.2096-6652.202003.

摘要

当前的自然语言推理方法在进行词对齐时往往只关注如何利用上下文信息改善词汇的表示，而忽略了上下文相似度对词汇对齐的帮助。推理方法在融合局部判定信息进行结果预测时，通常赋予局部判定信息相同的权重，而实际上不同词汇对应的局部判定信息对最终结果具有不同的影响。因此，提出一种通过增强词对齐性能改善自然语言推理性能的方法，该方法在词对齐时同时考虑词汇的词义相似度和上下文相似度，并且利用选择门机制对不同的局部判定信息进行加权。实验表明，该方法能够获得与当前最优的模型相当的准确率，而且该方法更加符合人脑推理判定行为过程，且具有很好的可解释性。

Abstract

Previous approaches generally use context information to improve the word representation but ignore the importance of context similarity in aligning tokens.Furthermore

most of them uniformly weight various local decisions during aggregation for the global judgment.However

local decisions related to various tokens can influence the final decision differently.In order to process these problems

an enhanced alignment mechanism was proposed

which jointly considers both token content and context similarity in computing the alignment weight for each token pair.Besides

a selection gate mechanism to weight local decisions differently was also proposed.Experimental results show that our performance is comparable to state-of-the-art approaches but better mimics human behavior

making it more interpretable.

关键词

Keywords

references

KOUYLEKOV M , MAGNINI B . Recognizing textual entailment with tree edit distance algorithms [J ] . Pascal Challenges on RTE , 2005 : 17 - 20 .

BOS J , MARKERT K . Recognizing textual entailment with robust logical inference [C ] // The Conference Human Language Technology and Empirical Methods in Natural Language Processing . Stroudsburg:Association for Computational Linguistics , 2005 : 404 - 426 .

BHASKAR P , BANERJEE S , PAKRAY P , et al . A hybrid question answering system for multiple choice question(MCQ) [C ] // QA4MRE at Conference and Labs of the Evaluation Forum.[S.l.:s.n] . 2013 .

ROMANO L , KOUYLEKOV M , SZPEKTOR I , et al . Investigating a generic paraphrase-based approach for relation extraction [C ] // The 11th Conference of the European Chapter of the Association for Computational Linguistics . Stroudsburg:Association for Computational Linguistics , 2006 : 409 - 416 .

PADÓ S , CER D , GALLEY M , et al . Measuring machine translation quality as semantic equivalence:a metric based on entailment features [J ] . Machine Translation , 2009 , 23 ( 2-3 ): 181 - 193 .

HARABAGIU S , HICKL A . Methods for using textual entailment in open-domain question answering [C ] // The 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics . Stroudsburg:Association for Computational Linguistics , 2006 : 905 - 912 .

HARABAGIU S , HICKL A , LACATUSU F . Satisfying information needs with multi-document summaries [J ] . Information Processing and Management , 2007 , 43 ( 6 ): 1619 - 1642 .

HEILMAN M , SMITH N A . Tree edit models for recognizing textual entailments,paraphrases,and answers to questions [C ] // Human Language Technologies:Conference of the North American Chapter of the Association of Computational Linguistics . Stroudsburg:Association for Computational Linguistics , 2010 : 1011 - 1019 .

BOWMAN S R , ANGELI G , POTTS C , et al . A large annotated corpus for learning natural language inference [C ] // The 2015 Conference on Empirical Methods in Natural Language Processing . Stroudsburg:Association for Computational Linguistics , 2015 : 632 - 642 .

MOU L L , MEN R , LI G , et al . Natural language inference by tree-based convolution and heuristic matching [C ] // The 54th Annual Meeting of the Association for Computational Linguistics . Stroudsburg:Association for Computational Linguistics , 2016 : 130 - 136 .

PARIKH A P , TACKSTROM O , DAS D , et al . A decomposable attention model for natural language inference [C ] // The 2016 Conference on Empirical Methods in Natural Language Processing . Stroudsburg:Association for Computational Linguistics , 2016 : 2249 - 2255 .

CHEN Q , ZHU X , LING Z H , et al . Enhanced LSTM for natural language inference [C ] // The 55th Annual Meeting of the Association for Computational Linguistics . Stroudsburg:Association for Computational Linguistics , 2017 : 1657 - 1668 .

TAY Y , LUU A T , HUI S C , et al . Compare,compress and propagate:enhancing neural architectures with alignment factorization for natural language inference [C ] // The 2018 Conference on Empirical Methods in Natural Language Processing . Stroudsburg:Association for Computational Linguistics , 2018 : 1565 - 1575 .

POLIAK A , NARADOWSKY J , HALDAR A , et al . Hypothesis only baselines in natural language inference [C ] // The 7th Joint Conference on Lexical and Computational Semantics.[S.l.:s.n . ] , 2018 .

PETERS M E , NEUMANN M , IYYER M , et al . Deep contextualized word representations [C ] // The 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies . Stroudsburg:Association for Computational Linguistics , 2018 : 2227 - 2237 .

WANG Z , HAMZA W , FLORIAN R , et al . Bilateral multi-perspective matching for natural language sentences [C ] // The 26th International Joint Conference on Artificial Intelligence . Los Altos:William Kaufman , 2017 : 4144 - 4150 .

VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all you need [C ] // The 31st Conference on Neural Information Processing System (NIPS 2017).[S.l.:s.n] . 2017 .

DEVLIN J , CHANG M , LEE K , et al . BERT:Pre-training of deep bidirectional transformers for language understanding [C ] // The 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies . Stroudsburg:Association for Computational Linguistics , 2019 : 4171 - 4186 .

DU Q L , SU K Y , ZONG C Q . Adopting the word-pair-dependencytriplets with individual comparison for natural language inference [C ] // The 27th International Conference on Computational Linguistics (COLING) . Stroudsburg:Association for Computational Linguistics , 2018 : 414 - 425 .

PENNINGTON J , SOCHER R , MANNING C D . GloVe:global vectors for word representation [C ] // The 2014 Conference on Empirical Methods in Natural Language Processing . Stroudsburg:Association for Computational Linguistics , 2014 : 1532 - 1543 .

CHEN D , FISCH A , WESTON J , et al . Reading Wikipedia to answer open-domain questions [C ] // The 55th Annual Meeting of the Association for Computational Linguistics . Stroudsburg:Association for Computational Linguistics , 2017 : 1870 - 1879 .

WILLIAMS A , NANGIA N , BOWMAN S R , et al . A broad-coverage challenge corpus for sentence understanding through inference [C ] // The 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies . Stroudsburg:Association for Computational Linguistics , 2018 : 1112 - 1122 .

GLOCKNER M , SHWARTZ V , GOLDBERG Y . Breaking NLI systems with sentences that require simple lexical inferences [C ] // The 56th Annual Meeting of the Association for Computational Linguistic . Stroudsburg:Association for Computational Linguistics , 2018 : 650 - 655 .

WANG S , JIANG J . Learning natural language inference with LSTM [C ] // The 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies . Stroudsburg:Association for Computational Linguistics , 2016 : 1442 - 1451 .

SHA L , CHANG B B , SUI Z F , et al . Reading and thinking:reread LSTM unit for textual entailment recognition [C ] // The 26th International Conference on Computational Linguistics (COLING) . Stroudsburg:Association for Computational Linguistics , 2016 : 2870 - 2879 .

GONG Y , LUO H , ZHANG J , et al . Natural language inference over interaction space [C ] // The 6th International Conference On Learning Representations.[S.l.:s.n] . 2018 .

GHAEINI R , HASAN S A , DATLA V , et al . DR-BiLSTM:dependent reading bidirectional LSTM for natural lnaguage inference [C ] // The 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies . Stroudsburg:Association for Computational Linguistics , 2018 : 1460 - 1469 .

CHEN Q , ZHU X D , LING Z H , et al . Neural natural language inference models enhanced with external knowledge [C ] // The 56th Annual Meeting of the Association for Computational Linguistics . Stroudsburg:Association for Computational Linguistics , 2018 : 2406 - 2417 .

GURURANGAN S , SWAYAMDIPTA S , LEVY O , et al . Annotation artifacts in natural language inference data [C ] // The 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies . Stroudsburg:Association for Computational Linguistics , 2018 : 107 - 112 .

SRIVASTAVA N , HINTON G , KRIZHEVSKY A , et al . Dropout:a simple way to prevent neural network from overfitting [J ] . The Journal of Machine Learning Research , 2014 , 15 ( 1 ): 1929 - 1958 .

BALAZS J A , MARRESE-TAYLOR E , LOYOLA P , et al . Refing raw sentence representations for textual entailment recognition via attention [C ] // The 2nd Workshop on Evaluating Vector Space Representations for NLP . Stroudsburg:Association for Computational Linguistics , 2017 : 51 - 55 .

CHEN Q , ZHU X D , LING Z H , et al . Recurrent neural network-based sentence encoder with gated attention for natural language inference [C ] // The 2nd Workshop on Evaluating Vector Space Representation for NLP . Stroudsburg:Association for Computational Linguistics , 2017 : 36 - 40 .

NIE Y , BANSAL M . Shortcut-stacked sentence encoders for multi-domain inference [C ] // The 2nd workshop on Evaluating Vector Space Representations for NLP . Stroudsburg:Association for Computational Linguistics , 2017 : 41 - 45 .

浏览量

269

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

暂无数据