您好，欢迎访问北京市农林科学院 机构知识库！
北京市农林科学院机构知识库

全部

  • 全部
  • 标题
  • 学者
  • 机构
  • 关键词

Crop disease identification and interpretation method based on multimodal deep learning

文献类型： 外文期刊

作者： Zhou, Ji 1 ; Li, Jiuxi 4 ; Wang, Chunshan 1 ; Wu, Huarui 2 ; Zhao, Chunjiang 2 ; Teng, Guifa 1 ;

作者机构： 1.Hebei Agr Univ, Sch Informat Sci & Technol, Baoding 071001, Peoples R China

2.Natl Engn Res Ctr Informat Technol Agr, Beijing 100097, Peoples R China

3.Beijing Res Ctr Informat Technol Agr, Beijing 100097, Peoples R China

4.Hebei Agr Univ, Sch Mech & Elect Engn, Baoding 071001, Peoples R China

5.Hebei Key Lab Agr Big Data, Baoding 071001, Peoples R China

关键词： Multimodality; Knowledge graph; Image-text pairs; Semantic interpretation; Disease identification

期刊名称：COMPUTERS AND ELECTRONICS IN AGRICULTURE （ 影响因子：5.565； 五年影响因子：5.494 ）

ISSN： 0168-1699

年卷期： 2021 年 189 卷

页码：

收录情况： SCI

摘要： Identification methods of crop diseases based on image modality alone have achieved relative success under limited and restricted conditions. As a data-driven technology, its performance depends on a large amount of image labeling data. Many of the existing methods neglected the role and value of other modal data except images and only relies on low-level image features for disease recognition without utilizing high-level domain knowledge, leading to poor credibility and interpretability of identification results. This paper targets tomato and cucumber common invasive diseases recognition as the research object. First, for the problem of insufficient utilization of multimodal data in existing models, the semantic embedding methods for disease images and disease description texts were examined, and the correlation and complementarity between the two types of modal data was utilized to realize the joint representation learning of disease features. Second, in response to the requirements of reliable identification and interpretability of diseases, the knowledge representation and knowledge embedding mechanism in the field of disease identification was studied, and the high-level domain knowledge graph was used as the external guidance for image feature learning and disease identification. Lastly, a disease identification model based on "image-text" multimodal collaborative representation and knowledge assistance (ITK-Net) was constructed. The proposed model achieved an identification accuracy, precision, sensitivity and specificity of 99.63%, 99%, 99.07% and 99.78% respectively on a dataset composed of "imagetext" pairs. Meanwhile, semantic interpretation was performed on the model inference process. The achievement of this paper can offer a new method for disease identification based on multimodal data and domain knowledge, which might help improve the intelligence level of crop disease identification.

  • 相关文献
作者其他论文 更多>>
置顶
购物车
反馈

© 京ICP备09089781号-29 主办单位：北京市农林科学院

学术资源： 中国农业科学院 农业专业知识服务系统 农科联盟资源共建共享平台 中国农业科技文献与信息服务平台 中国农业期刊集成服务平台