MTA-YOLACT: Multitask-aware network on fruit bunch identification for cherry tomato robotic harvesting

文献类型： 外文期刊

第一作者： Li, Yajun

作者： Li, Yajun;Feng, Qingchun;Liu, Cheng;Xiong, Zicong;Sun, Yuhuan;Xie, Feng;Li, Tao;Zhao, Chunjiang;Li, Yajun;Zhao, Chunjiang;Feng, Qingchun;Liu, Cheng;Xiong, Zicong;Sun, Yuhuan;Xie, Feng;Li, Tao

作者机构：

关键词： Multitask -aware network; Tomato harvesting robot; YOLACT; Decision tree

期刊名称：EUROPEAN JOURNAL OF AGRONOMY （ 影响因子：5.2； 五年影响因子：5.9 ）

ISSN： 1161-0301

年卷期： 2023 年 146 卷

页码：

收录情况： SCI

摘要： Accurate and rapid perception of fruit bunch posture is necessary for the cherry tomato harvesting robot to successfully achieve the bunch's holding and separating. According to the postural relationship of the fruit bunch, bunch pedicel, and plant' main-stem, the robotic end-effector's holding region and approach path could be determined, which were important for successful picking operation. The main goal of this research was to propose a multitask-aware network (MTA-YOLACT), which simultaneously performed region detection on fruit bunch, and region segmentation on pedicel and main-stem. The MTA-YOLACT extended from the pre-trained YOLACT model, included two detector branch networks for detection and instance segment, which shared the same backbone network, and the loss function with weighting coefficients of the two branches was adopted to balance the multi-task learning, according to multi-task's homoscedastic uncertainty during the model training. Furthermore, in order to cluster the fruit bunch, pedicel and main-stem from the same bunch target, a classification and regression tree (CART) model was built, based on the region's positional relationship from the MTAYOLACT output. An image dataset of cherry tomato plants in China greenhouse was built to training and test the model. The results indicated a promising performance of the proposed network, with an F1-score of 95.4% on detecting fruit bunches and the mean Average Precision of 38.7% and 51.9% on the instance segmentation of pedicel and main-stem, which was 1.1% and 3.5% more than original YOLACT. Beyond that, our approach performed a real-time detection and instance segmentation of 13.3 frames per second (FPS). The whole bunch could be identified by the CART model with an average accuracy of 99.83% and the time cost of 9.53 ms. These results demonstrated the research could be a viable support to the harvesting robot's vision unit development and the end-effector's motion planning in the future research.

分类号：

  • 相关文献
作者其他论文 更多>>

微信小程序