End-to-end stereo matching network with two-stage partition filtering for full-resolution depth estimation and precise localization of kiwifruit for robotic harvesting

文献类型： 外文期刊

第一作者： Jing, Xudong

作者： Jing, Xudong;Jiang, Hanhui;Niu, Shiao;Zhang, Haosen;Murengami, Bryan Gilbert;Wu, Zhenchao;Li, Rui;Fu, Longsheng;Zhou, Chengquan;Ye, Hongbao;Chen, Jinyong;Fu, Longsheng;Fu, Longsheng;Fu, Longsheng;Majeed, Yaqoob

作者机构：

关键词： Two-stage partition filtering; Kiwifruit localization; LaC-Gwc Net; YOLOv8; Robotic harvesting

期刊名称：COMPUTERS AND ELECTRONICS IN AGRICULTURE （ 影响因子：7.7； 五年影响因子：8.4 ）

ISSN： 0168-1699

年卷期： 2024 年 225 卷

页码：

收录情况： SCI

摘要： Full-resolution depth estimation within operational space of robotic arms and accurate localization of kiwifruits is very important for automated harvesting. Depth estimation is expected to be accurate and full-resolution while current depth estimation methods are susceptible to depth missing due to occlusion and uneven illumination. And depth estimation mostly focuses on fruit localization, while obstacles such as branches and wires, which can affect harvesting strategy, have not been considered. This paper localized kiwifruits based on bounding boxes output by YOLOv8m and full-resolution depth from an end-to-end stereo matching network, i.e., LaC-Gwc Net, which was trained after generating a stereo matching dataset by proposing a two-stage partition filtering algorithm. Results showed that LaC-Gwc Net achieved an end-point error (EPE) of 3.8 pixels, which means that accurate depth estimation can also be achieved for thin obstacles such as the branches and the wires. Additionally, YOLOv8m obtained acceptable results in detecting kiwifruits and their calyxes, reaching mean average precision (mAP) of 93.1% and detection speed of 7.0 ms. The methodology obtained only kiwifruit localization error of 4.0 mm on the Z-axis, which meets requirements of robotic harvesting. Furthermore, this study considered the localization of obstacles in kiwifruit orchards, providing high-precision full-resolution depth estimation for agricultural harvesting robots.

分类号：

  • 相关文献
作者其他论文 更多>>

微信小程序