您好，欢迎访问北京市农林科学院 机构知识库！
北京市农林科学院机构知识库

全部

  • 全部
  • 标题
  • 学者
  • 机构
  • 关键词

Chinese named entity recognition for agricultural diseases based on entity-related visual prompts injection

文献类型： 外文期刊

作者： Zhang, Chenshuo 1 ; Zhang, Lijie 5 ; Wu, Huarui 2 ; Wang, Chunshan 1 ; Chen, Cheng 2 ; Zhu, Huaji 2 ; Liang, Fangfang 1 ;

作者机构： 1.Hebei Agr Univ, Sch Informat Sci & Technol, Baoding 071001, Peoples R China

2.Natl Engn Res Ctr Informat Technol Agr, Beijing 100097, Peoples R China

3.Beijing Res Ctr Informat Technol Agr, Beijing 100097, Peoples R China

4.Hebei Key Lab Agr Big Data, Baoding 071001, Peoples R China

5.High resolut Earth Observat Syst Data & Applicat T, Baoding 071001, Peoples R China

关键词： Agricultural diseases; Visual prompts; Multimodal named entity recognition; Pre-trained language model

期刊名称：COMPUTERS AND ELECTRONICS IN AGRICULTURE （ 影响因子：8.9； 五年影响因子：9.3 ）

ISSN： 0168-1699

年卷期： 2024 年 227 卷

页码：

收录情况： SCI

摘要： Named entity recognition is a crucial step in information extraction for agricultural diseases. However, most existing works only utilize word embedding models to generate contextual semantic features, which are limited by textual contextual dependencies and suffer from the problem of missing contextual semantic features. There is a semantic correlation between the visual information in agricultural disease images and the target entities in agricultural disease texts, but this visual information has not been fully utilized among the existing works. To solve the aforementioned issues, we propose a Chinese named entity recognition model for agricultural diseases based on entity-related visual prompts injection. First, we introduce an improved object detector with Shape-IoU to obtain local images related to the target entities. Second, we propose a visual feature extraction structure based on multi-scale feature fusion to extract visual features from the local images and encode them as visual prompts. Third, we propose a novel visual-guided attention mechanism to achieve multimodal information fusion within the pretrained language model. Furthermore, the first multimodal named entity recognition (MNER) dataset for agricultural diseases named Disease7000 was also collected and annotated, which contains 7,000 image-text pairs, 4 entity types, and 31,656 samples. Experimental results show that our proposed model achieved the best F1 scores of 89.17% and 75.06% on Disease7000 and the publicly available Twitter2015 dataset, respectively, demonstrating the effectiveness and generalizability of the model. Additionally, the ablation study validate the effectiveness of each module in many aspects.

  • 相关文献
作者其他论文 更多>>
置顶
购物车
反馈

© 京ICP备09089781号-29 主办单位：北京市农林科学院

学术资源： 中国农业科学院 农业专业知识服务系统 农科联盟资源共建共享平台 中国农业科技文献与信息服务平台 中国农业期刊集成服务平台