文献类型： 外文期刊

作者： Zhou, Lingli ¹ ; Hu, Anqi ¹ ; Cheng, Yawen ¹ ; Zhang, Wenxiang ¹ ; Zhang, Bingyuan ¹ ; Lu, Xinyu ¹ ; Wu, Qian ¹ ; Ren, Ni ¹ ;

作者机构： 1.Jiangsu Acad Agr Sci, Inst Agr Informat, Nanjing, Peoples R China

2.Minist Agr & Rural Affairs, Key Lab Intelligent Agr Technol Changjiang Delta, Nanjing, Peoples R China

关键词： harvesting robot; image semantic segmentation; obstacle perception; deep learning; fruit selection; positioning; tomato

期刊名称：FRONTIERS IN PLANT SCIENCE （ 影响因子：4.8； 五年影响因子：5.7 ）

ISSN： 1664-462X

年卷期： 2024 年 15 卷

页码：

收录情况： SCI

摘要： With the advancement of computer vision technology, vision-based target perception has emerged as a predominant approach for harvesting robots to identify and locate fruits. However, little attention has been paid to the fact that fruits may be obscured by stems or other objects. In order to improve the vision detection ability of fruit harvesting robot, a fruit target selection and location approach considering obstacle perception was proposed. To enrich the dataset for tomato harvesting, synthetic data were generated by rendering a 3D simulated model of the tomato greenhouse environment, and automatically producing corresponding pixel-level semantic segmentation labels. An attention-based spatial-relationship feature extraction module (SFM) with lower computational complexity was designed to enhance the ability of semantic segmentation network DeepLab v3+ in accurately segmenting linear-structured obstructions such as stems and wires. An adaptive K-means clustering method was developed to distinguish individual instances of fruits. Furthermore, a barrier-free fruit selection algorithm that integrates information of obstacles and fruit instances was proposed to identify the closest and largest non-occluded fruit as the optimal picking target. The improved semantic segmentation network exhibited enhanced performance, achieving an accuracy of 96.75%. Notably, the Intersection-over-Union (IoU) of wire and stem classes was improved by 5.0% and 2.3%, respectively. Our target selection method demonstrated accurate identification of obstacle types (96.15%) and effectively excluding fruits obstructed by strongly resistant objects (86.67%). Compared to the fruit detection method without visual obstacle avoidance (Yolo v5), our approach exhibited an 18.9% increase in selection precision and a 1.3% reduction in location error. The improved semantic segmentation algorithm significantly increased the segmentation accuracy of linear-structured obstacles, and the obstacle perception algorithm effectively avoided occluded fruits. The proposed method demonstrated an appreciable ability in precisely selecting and locating barrier-free fruits within non-structural environments, especially avoiding fruits obscured by stems or wires. This approach provides a more reliable and practical solution for fruit selection and localization for harvesting robots, while also being applicable to other fruits and vegetables such as sweet peppers and kiwis.