您好，欢迎访问北京市农林科学院 机构知识库！
北京市农林科学院机构知识库

全部

  • 全部
  • 标题
  • 学者
  • 机构
  • 关键词

History Reuse and Bag-of-Words Loss for Long Summary Generation

文献类型： 外文期刊

作者： Liu, Qing 1 ; Chen, Lei 1 ; Yuan, Yuan 1 ; Wu, Huarui 3 ;

作者机构： 1.Chinese Acad Sci, HFIPS, Inst Intelligent Machines, Hefei 230031, Peoples R China

2.Univ Sci & Technol China, Hefei 230026, Peoples R China

3.Beijing Res Ctr Informat Technol Agr, Beijing 100097, Peoples R China

关键词： History; Decoding; Training; Predictive models; Postal services; Computational modeling; Vocabulary; Abstractive summarization; long summaries; history reuse; bag-of-words; word order deviation

期刊名称：IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING （ 影响因子：3.919； 五年影响因子：3.906 ）

ISSN： 2329-9290

年卷期： 2021 年 29 卷

页码：

收录情况： SCI

摘要： Recurrent Neural Network (RNN) based abstractive text summarization models have made great progress over the past few years, largely triggered by the encoder-decoder architecture. However, there has been little work improving the generation of relatively long summaries. In this paper, we concentrate on two prominent problems in long summary generation. First, although significant efforts have been made to assist the encoder in handling long sequences, the decoder struggles with long sequences owing to the limited storage capacity of RNN. We propose a simple and effective approach called history reuse, which first mines critical information from the history summary sequence and then transmits the information to the decoder. Second, since encoder-decoder models are typically trained to produce exactly the same summary as the target summary, certain word order deviations between the predicted summary and target summary are excessively punished. Accordingly, we introduce a fully differentiable loss called bag-of-words (BoW) loss, which takes advantage of the feature of BoW discarding word order information in texts, and computes the difference between the two summaries at the BoW space. Experiments on two benchmark datasets, CNN/Daily Mail and Pubmed, demonstrate that our methods significantly improve the baseline.

  • 相关文献
作者其他论文 更多>>
置顶
购物车
反馈

© 京ICP备09089781号-29 主办单位：北京市农林科学院

学术资源： 中国农业科学院 农业专业知识服务系统 农科联盟资源共建共享平台 中国农业科技文献与信息服务平台 中国农业期刊集成服务平台