Improved binocular localization of kiwifruit in orchard based on fruit and calyx detection using YOLOv5x for robotic picking

文献类型： 外文期刊

第一作者： Gao, Changqing

作者： Gao, Changqing;Jiang, Hanhui;Liu, Xiaojuan;Li, Haihong;Wu, Zhenchao;Sun, Xiaoming;He, Leilei;Mao, Wulan;Li, Rui;Fu, Longsheng;Fu, Longsheng;Fu, Longsheng;Fu, Longsheng;Mao, Wulan;Majeed, Yaqoob

作者机构：

关键词： Fruit detection; Calyx localization; Binocular stereo vision; YOLOv5x; Robotic harvesting

期刊名称：COMPUTERS AND ELECTRONICS IN AGRICULTURE （ 影响因子：8.3； 五年影响因子：8.3 ）

ISSN： 0168-1699

年卷期： 2024 年 217 卷

页码：

收录情况： SCI

摘要： Localization is the first critical step for picking robots to successfully grasp fruit. However, classical binocular localization methods adopted global matching for kiwifruit, which may result in a large amount of mismatching feature points in complex orchard and thus cause low localization accuracy. Therefore, an improved binocular localization method of calyxes based on deep learning was proposed to accurately detect and locate kiwifruit for robotic harvesting. Calyxes in the binocular images and kiwifruit in the left images of the binocular images were detected using You Only Look Once version 5 xlarge (YOLOv5x). The detected calyxes were matched in the binocular images using kiwifruit and calyx pairing and kiwifruit matching. The matched calyxes in the binocular images were used to locate calyxes using three localization methods. Specifically, three binocular localization methods, i.e., calyx localization (CL), fruit localization (FL), and depth information from depth map (DIDM), were compared to find the optimal one. Ground truth three-dimensional coordinates of calyxes was measured by laser rangefinder and coordinate paper on a self-designed experimental platform. Results showed that YOLOv5x achieved an average precision (AP) of 99.5 % on kiwifruit detection and a mean AP of 93.5 % on kiwifruit and calyx detection with a detection speed of 108 ms per image. Average deviation of X-axis, Y-axis, and Z-axis obtained by the CL method were 7.9 mm, 6.4 mm, and 4.8 mm, respectively. Compared with the FL and DIDM methods, localization error rate of the proposed CL method was reduced by 55.1 % and 53.8 %, respectively. It indicates that the proposed CL method is promising for robotic harvesting.

分类号：

  • 相关文献
作者其他论文 更多>>

微信小程序