您好，欢迎访问北京市农林科学院 机构知识库！
北京市农林科学院机构知识库

全部

  • 全部
  • 标题
  • 学者
  • 机构
  • 关键词

VEG-MMKG: Multimodal knowledge graph construction for vegetables based on pre-trained model extraction

文献类型： 外文期刊

作者： Lv, Bowen 1 ; Wu, Huarui 1 ; Chen, Wenbai 2 ; Chen, Cheng 1 ; Miao, Yisheng 1 ; Zhao, Chunjiang 1 ;

作者机构： 1.Natl Engn Res Ctr Informat Technol Agr, Beijing 100097, Peoples R China

2.Beijing Informat Sci & Technol Univ, Sch Automat, Beijing 100192, Peoples R China

3.Beijing Acad Agr & Forestry Sci, Informat Technol Res Ctr, Beijing 100097, Peoples R China

4.Minist Agr & Rural Affairs, Key Lab Digital Village Technol, Beijing 100097, Peoples R China

关键词： Knowledge graph; Multimodal fusion; Image-text pairs; Pre-trained model

期刊名称：COMPUTERS AND ELECTRONICS IN AGRICULTURE （ 影响因子：8.9； 五年影响因子：9.3 ）

ISSN： 0168-1699

年卷期： 2024 年 226 卷

页码：

收录情况： SCI

摘要： Knowledge graph technology is of great significance to modern agricultural information management and datadriven decision support. However, agricultural knowledge is rich in types, and agricultural knowledge graph databases built only based on text are not conducive to users' intuitive perception and comprehensive understanding of knowledge. In view of this, this paper proposes a solution to extract knowledge and construct an agricultural multimodal knowledge graph using a pre-trained language model. This paper takes two plants, cabbage and corn, as research objects. First, a text-image collaborative representation learning method with a two-stream structure is adopted to combine the image modal information of vegetables with the text modal information, and the correlation and complementarity between the two types of information are used to achieve entity alignment. In addition, in order to solve the problem of high similarity of vegetable entities in small categories, a cross-modal fine-grained contrastive learning method is introduced, and the problem of insufficient semantic association between modalities is solved by contrastive learning of vocabulary and small areas of images. Finally, a visual multimodal knowledge graph user interface is constructed using the results of image and text matching. Experimental results show that the image and text matching efficiency of the fine-tuned pretrained model on the vegetable dataset is 76.7%, and appropriate images can be matched for text entities. The constructed visual multimodal knowledge graph database allows users to query and filter knowledge according to their needs, providing assistance for subsequent research on various applications in specific fields such as multimodal agricultural intelligent question and answer, crop pest and disease identification, and agricultural product recommendations.

  • 相关文献
作者其他论文 更多>>
置顶
购物车
反馈

© 京ICP备09089781号-29 主办单位：北京市农林科学院

学术资源： 中国农业科学院 农业专业知识服务系统 农科联盟资源共建共享平台 中国农业科技文献与信息服务平台 中国农业期刊集成服务平台