Enhancing alignment with context similarity for natural language inference

Qianlong DU; Chengqing ZONG; Keh-Yih SU

doi:10.11959/j.issn.2096-6652.202003

您当前的位置：

首页 >

文章列表页 >

Enhancing alignment with context similarity for natural language inference

Regular Papers | 更新时间：2024-06-05

- Enhancing alignment with context similarity for natural language inference
- Chinese Journal of Intelligent Science and Technology Vol. 2, Issue 1, Pages: 26-35(2020)
- 作者机构：
  
  1. 中国科学院自动化研究所模式识别国家重点实验室，北京 100190
  2. 中国科学院大学，北京 100049
  3. 台湾“中央研究院”资讯科学研究所，台湾台北 11529
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-6652.202003
  CLC： TP391.1
- Published Online：2020-03，
  
  Published：20 March 2020
- 稿件说明：
移动端阅览
Qianlong DU, Chengqing ZONG, Keh-Yih SU. Enhancing alignment with context similarity for natural language inference[J]. Chinese journal of intelligent science and technology, 2020, 2(1): 26-35.
DOI：

Qianlong DU, Chengqing ZONG, Keh-Yih SU. Enhancing alignment with context similarity for natural language inference[J]. Chinese journal of intelligent science and technology, 2020, 2(1): 26-35. DOI： 10.11959/j.issn.2096-6652.202003.

摘要

当前的自然语言推理方法在进行词对齐时往往只关注如何利用上下文信息改善词汇的表示，而忽略了上下文相似度对词汇对齐的帮助。推理方法在融合局部判定信息进行结果预测时，通常赋予局部判定信息相同的权重，而实际上不同词汇对应的局部判定信息对最终结果具有不同的影响。因此，提出一种通过增强词对齐性能改善自然语言推理性能的方法，该方法在词对齐时同时考虑词汇的词义相似度和上下文相似度，并且利用选择门机制对不同的局部判定信息进行加权。实验表明，该方法能够获得与当前最优的模型相当的准确率，而且该方法更加符合人脑推理判定行为过程，且具有很好的可解释性。

Abstract

Previous approaches generally use context information to improve the word representation but ignore the importance of context similarity in aligning tokens.Furthermore

most of them uniformly weight various local decisions during aggregation for the global judgment.However

local decisions related to various tokens can influence the final decision differently.In order to process these problems

an enhanced alignment mechanism was proposed

which jointly considers both token content and context similarity in computing the alignment weight for each token pair.Besides

a selection gate mechanism to weight local decisions differently was also proposed.Experimental results show that our performance is comparable to state-of-the-art approaches but better mimics human behavior

making it more interpretable.

关键词

Keywords

references

KOUYLEKOV M , MAGNINI B . Recognizing textual entailment with tree edit distance algorithms [J ] . Pascal Challenges on RTE , 2005 : 17 - 20 .

BOS J , MARKERT K . Recognizing textual entailment with robust logical inference [C ] // The Conference Human Language Technology and Empirical Methods in Natural Language Processing . Stroudsburg:Association for Computational Linguistics , 2005 : 404 - 426 .

BHASKAR P , BANERJEE S , PAKRAY P , et al . A hybrid question answering system for multiple choice question(MCQ) [C ] // QA4MRE at Conference and Labs of the Evaluation Forum.[S.l.:s.n] . 2013 .

ROMANO L , KOUYLEKOV M , SZPEKTOR I , et al . Investigating a generic paraphrase-based approach for relation extraction [C ] // The 11th Conference of the European Chapter of the Association for Computational Linguistics . Stroudsburg:Association for Computational Linguistics , 2006 : 409 - 416 .

PADÓ S , CER D , GALLEY M , et al . Measuring machine translation quality as semantic equivalence:a metric based on entailment features [J ] . Machine Translation , 2009 , 23 ( 2-3 ): 181 - 193 .

HARABAGIU S , HICKL A . Methods for using textual entailment in open-domain question answering [C ] // The 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics . Stroudsburg:Association for Computational Linguistics , 2006 : 905 - 912 .

HARABAGIU S , HICKL A , LACATUSU F . Satisfying information needs with multi-document summaries [J ] . Information Processing and Management , 2007 , 43 ( 6 ): 1619 - 1642 .

HEILMAN M , SMITH N A . Tree edit models for recognizing textual entailments,paraphrases,and answers to questions [C ] // Human Language Technologies:Conference of the North American Chapter of the Association of Computational Linguistics . Stroudsburg:Association for Computational Linguistics , 2010 : 1011 - 1019 .

BOWMAN S R , ANGELI G , POTTS C , et al . A large annotated corpus for learning natural language inference [C ] // The 2015 Conference on Empirical Methods in Natural Language Processing . Stroudsburg:Association for Computational Linguistics , 2015 : 632 - 642 .

MOU L L , MEN R , LI G , et al . Natural language inference by tree-based convolution and heuristic matching [C ] // The 54th Annual Meeting of the Association for Computational Linguistics . Stroudsburg:Association for Computational Linguistics , 2016 : 130 - 136 .

PARIKH A P , TACKSTROM O , DAS D , et al . A decomposable attention model for natural language inference [C ] // The 2016 Conference on Empirical Methods in Natural Language Processing . Stroudsburg:Association for Computational Linguistics , 2016 : 2249 - 2255 .

CHEN Q , ZHU X , LING Z H , et al . Enhanced LSTM for natural language inference [C ] // The 55th Annual Meeting of the Association for Computational Linguistics . Stroudsburg:Association for Computational Linguistics , 2017 : 1657 - 1668 .

TAY Y , LUU A T , HUI S C , et al . Compare,compress and propagate:enhancing neural architectures with alignment factorization for natural language inference [C ] // The 2018 Conference on Empirical Methods in Natural Language Processing . Stroudsburg:Association for Computational Linguistics , 2018 : 1565 - 1575 .

POLIAK A , NARADOWSKY J , HALDAR A , et al . Hypothesis only baselines in natural language inference [C ] // The 7th Joint Conference on Lexical and Computational Semantics.[S.l.:s.n . ] , 2018 .

PETERS M E , NEUMANN M , IYYER M , et al . Deep contextualized word representations [C ] // The 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies . Stroudsburg:Association for Computational Linguistics , 2018 : 2227 - 2237 .

WANG Z , HAMZA W , FLORIAN R , et al . Bilateral multi-perspective matching for natural language sentences [C ] // The 26th International Joint Conference on Artificial Intelligence . Los Altos:William Kaufman , 2017 : 4144 - 4150 .

VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all you need [C ] // The 31st Conference on Neural Information Processing System (NIPS 2017).[S.l.:s.n] . 2017 .

DEVLIN J , CHANG M , LEE K , et al . BERT:Pre-training of deep bidirectional transformers for language understanding [C ] // The 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies . Stroudsburg:Association for Computational Linguistics , 2019 : 4171 - 4186 .

DU Q L , SU K Y , ZONG C Q . Adopting the word-pair-dependencytriplets with individual comparison for natural language inference [C ] // The 27th International Conference on Computational Linguistics (COLING) . Stroudsburg:Association for Computational Linguistics , 2018 : 414 - 425 .

PENNINGTON J , SOCHER R , MANNING C D . GloVe:global vectors for word representation [C ] // The 2014 Conference on Empirical Methods in Natural Language Processing . Stroudsburg:Association for Computational Linguistics , 2014 : 1532 - 1543 .

CHEN D , FISCH A , WESTON J , et al . Reading Wikipedia to answer open-domain questions [C ] // The 55th Annual Meeting of the Association for Computational Linguistics . Stroudsburg:Association for Computational Linguistics , 2017 : 1870 - 1879 .

WILLIAMS A , NANGIA N , BOWMAN S R , et al . A broad-coverage challenge corpus for sentence understanding through inference [C ] // The 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies . Stroudsburg:Association for Computational Linguistics , 2018 : 1112 - 1122 .

GLOCKNER M , SHWARTZ V , GOLDBERG Y . Breaking NLI systems with sentences that require simple lexical inferences [C ] // The 56th Annual Meeting of the Association for Computational Linguistic . Stroudsburg:Association for Computational Linguistics , 2018 : 650 - 655 .

WANG S , JIANG J . Learning natural language inference with LSTM [C ] // The 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies . Stroudsburg:Association for Computational Linguistics , 2016 : 1442 - 1451 .

SHA L , CHANG B B , SUI Z F , et al . Reading and thinking:reread LSTM unit for textual entailment recognition [C ] // The 26th International Conference on Computational Linguistics (COLING) . Stroudsburg:Association for Computational Linguistics , 2016 : 2870 - 2879 .

GONG Y , LUO H , ZHANG J , et al . Natural language inference over interaction space [C ] // The 6th International Conference On Learning Representations.[S.l.:s.n] . 2018 .

GHAEINI R , HASAN S A , DATLA V , et al . DR-BiLSTM:dependent reading bidirectional LSTM for natural lnaguage inference [C ] // The 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies . Stroudsburg:Association for Computational Linguistics , 2018 : 1460 - 1469 .

CHEN Q , ZHU X D , LING Z H , et al . Neural natural language inference models enhanced with external knowledge [C ] // The 56th Annual Meeting of the Association for Computational Linguistics . Stroudsburg:Association for Computational Linguistics , 2018 : 2406 - 2417 .

GURURANGAN S , SWAYAMDIPTA S , LEVY O , et al . Annotation artifacts in natural language inference data [C ] // The 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies . Stroudsburg:Association for Computational Linguistics , 2018 : 107 - 112 .

SRIVASTAVA N , HINTON G , KRIZHEVSKY A , et al . Dropout:a simple way to prevent neural network from overfitting [J ] . The Journal of Machine Learning Research , 2014 , 15 ( 1 ): 1929 - 1958 .

BALAZS J A , MARRESE-TAYLOR E , LOYOLA P , et al . Refing raw sentence representations for textual entailment recognition via attention [C ] // The 2nd Workshop on Evaluating Vector Space Representations for NLP . Stroudsburg:Association for Computational Linguistics , 2017 : 51 - 55 .

CHEN Q , ZHU X D , LING Z H , et al . Recurrent neural network-based sentence encoder with gated attention for natural language inference [C ] // The 2nd Workshop on Evaluating Vector Space Representation for NLP . Stroudsburg:Association for Computational Linguistics , 2017 : 36 - 40 .

NIE Y , BANSAL M . Shortcut-stacked sentence encoders for multi-domain inference [C ] // The 2nd workshop on Evaluating Vector Space Representations for NLP . Stroudsburg:Association for Computational Linguistics , 2017 : 41 - 45 .

Views

269

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

Postal code：100079
Tel：（010）53879206 Email：cjist@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：35685 Visits Today：160

⁰