Occluded Apple Fruit Detection and Localization with a Frustum-Based Point-Cloud-Processing Approach for Robotic Harvesting

文献类型： 外文期刊

第一作者： Li, Tao

作者： Li, Tao;Feng, Qingchun;Xie, Feng;Qiu, Quan;Xie, Feng;Zhao, Chunjiang

作者机构：

关键词： agricultural robot; deep learning; fruit detection; point cloud; apple-harvesting robot; RGBD camera

期刊名称：REMOTE SENSING （ 影响因子：5.349； 五年影响因子：5.786 ）

ISSN：

年卷期： 2022 年 14 卷 3 期

页码：

收录情况： SCI

摘要： Precise localization of occluded fruits is crucial and challenging for robotic harvesting in orchards. Occlusions from leaves, branches, and other fruits make the point cloud acquired from Red Green Blue Depth (RGBD) cameras incomplete. Moreover, an insufficient filling rate and noise on depth images of RGBD cameras usually happen in the shade from occlusions, leading to the distortion and fragmentation of the point cloud. These challenges bring difficulties to position locating and size estimation of fruit for robotic harvesting. In this paper, a novel 3D fruit localization method is proposed based on a deep learning segmentation network and a new frustum-based point-cloud-processing method. A one-stage deep learning segmentation network is presented to locate apple fruits on RGB images. With the outputs of masks and 2D bounding boxes, a 3D viewing frustum was constructed to estimate the depth of the fruit center. By the estimation of centroid coordinates, a position and size estimation approach is proposed for partially occluded fruits to determine the approaching pose for robotic grippers. Experiments in orchards were performed, and the results demonstrated the effectiveness of the proposed method. According to 300 testing samples, with the proposed method, the median error and mean error of fruits' locations can be reduced by 59% and 43%, compared to the conventional method. Furthermore, the approaching direction vectors can be correctly estimated.

分类号：

  • 相关文献
作者其他论文 更多>>

微信小程序