文献类型： 外文期刊

作者： Gao, Changqing ¹ ; Jiang, Hanhui ¹ ; Liu, Xiaojuan ¹ ; Li, Haihong ¹ ; Wu, Zhenchao ¹ ; Sun, Xiaoming ¹ ; He, Leilei ¹ ; Mao, Wulan ¹ ; Majeed, Yaqoob ⁶ ; Li, Rui ¹ ; Fu, Longsheng ¹ ;

作者机构： 1.Northwest A&F Univ, Coll Mech & Elect Engn, Yangling 712100, Shaanxi, Peoples R China

2.Minist Agr & Rural Affairs, Key Lab Agr Internet Things, Yangling 712100, Shaanxi, Peoples R China

3.Shaanxi Key Lab Agr Informat Percept & Intelligent, Yangling 712100, Shaanxi, Peoples R China

4.Northwest A&F Univ, Shenzhen Res Inst, Shenzhen 518000, Guangdong, Peoples R China

5.Xinjiang Acad Agr Sci, Inst Agr Mechanizat, Urumqi 830000, Peoples R China

6.Univ Agr Faisalabad, Fac Agr Engn & Technol, Faisalabad 38000, Pakistan

关键词： Fruit detection; Calyx localization; Binocular stereo vision; YOLOv5x; Robotic harvesting

期刊名称：COMPUTERS AND ELECTRONICS IN AGRICULTURE （ 影响因子：8.3； 五年影响因子：8.3 ）

ISSN： 0168-1699

年卷期： 2024 年 217 卷

页码：

收录情况： SCI

摘要： Localization is the first critical step for picking robots to successfully grasp fruit. However, classical binocular localization methods adopted global matching for kiwifruit, which may result in a large amount of mismatching feature points in complex orchard and thus cause low localization accuracy. Therefore, an improved binocular localization method of calyxes based on deep learning was proposed to accurately detect and locate kiwifruit for robotic harvesting. Calyxes in the binocular images and kiwifruit in the left images of the binocular images were detected using You Only Look Once version 5 xlarge (YOLOv5x). The detected calyxes were matched in the binocular images using kiwifruit and calyx pairing and kiwifruit matching. The matched calyxes in the binocular images were used to locate calyxes using three localization methods. Specifically, three binocular localization methods, i.e., calyx localization (CL), fruit localization (FL), and depth information from depth map (DIDM), were compared to find the optimal one. Ground truth three-dimensional coordinates of calyxes was measured by laser rangefinder and coordinate paper on a self-designed experimental platform. Results showed that YOLOv5x achieved an average precision (AP) of 99.5 % on kiwifruit detection and a mean AP of 93.5 % on kiwifruit and calyx detection with a detection speed of 108 ms per image. Average deviation of X-axis, Y-axis, and Z-axis obtained by the CL method were 7.9 mm, 6.4 mm, and 4.8 mm, respectively. Compared with the FL and DIDM methods, localization error rate of the proposed CL method was reduced by 55.1 % and 53.8 %, respectively. It indicates that the proposed CL method is promising for robotic harvesting.