Litchi picking points localization in natural environment based on the Litchi-YOSO model and branch morphology reconstruction algorithm

文献类型： 外文期刊

第一作者： Wang, Chenglin

作者： Wang, Chenglin;Han, Qiyu;Li, Chunjiang;Zhang, Tie;Sun, Xing;Zhang, Tie;Sun, Xing

作者机构：

关键词： Litchi; Picking point; Semantic segmentation; Yolact; Branch morphology

期刊名称：COMPUTERS AND ELECTRONICS IN AGRICULTURE （ 影响因子：8.9； 五年影响因子：9.3 ）

ISSN： 0168-1699

年卷期： 2024 年 226 卷

页码：

收录情况： SCI

摘要： Litchi is a fruit widely cultivated in China with high nutritional and economic value. Currently, it is primarily harvested manually, which incurs high costs. Mechanized harvesting can improve efficiency and reduce costs. In the context of Agriculture 4.0, harvesting robots equipped have garnered attention. These robots use vision systems to identify Litchi fruits and branches, and then locate the picking points. However, challenges arise due to the complex morphology of Litchi branches, low contrast with the background, and frequent branch occlusion. Efficient harvesting requires locating the main fruit-bearing branch (MFBB) for whole-bunch picking. The MFBB and non-MFBB have similar geometric features, making it hard to distinguish them by position. This study constructed a Litchi-YOSO(Litchi-you only segment once) model combined with a branch morphology reconstruction algorithm to accurately segment litchi and locate the picking point on the MFBB. The model's backbone incorporates a combination of Fasternet and C2F modules, which maintains computational volume while reducing memory access during computation. The Biformer attention mechanism is added to the backbone to precisely locate branch boundaries by learning regional similarities. The RepGFPN (Reparameterized generalized feature pyramid network) module forms the neck part of the model, optimizing cross-scale fusion to better learn pixel features at different distances. The branch morphology reconstruction algorithm uses the Litchi-YOSO segmentation results to build a sparse matrix from the central line of the branch region, recording connectivity in a compressed format. Boundary points are filtered to reconstruct a "point-line" model. The MFBB is located using Prim's algorithm and the longest root path algorithm. To determine the picking point, Harris corner detection and sliding window algorithms are employed to identify the flattest regions with minimal leaf scars. The experiment showed that the Litchi-YOSO model achieved a mAP@0.5 of 76.45 %, 29.1 MB of model size, and 26.5 ms of average segmentation speed. It can accurately and quickly segment the Litchi fruits and branch areas. The branch morphology reconstruction algorithm was tested on natural environment images, achieving a picking point location success rate of 91.5 % and an average processing time of 120.20 ms. It can rapidly locate picking points for Litchi bunches under various conditions such as occlusion and horizontal growth of branches, providing a foundation for the efficient and precise operation of harvesting robots.

分类号：

  • 相关文献
作者其他论文 更多>>

微信小程序