In the unstructured litchi orchard,precise identification and localization of litchi fruits and picking points are crucial for litchi-picking robots.Most studies adopt multi-step methods to detect fruit and locate pic...In the unstructured litchi orchard,precise identification and localization of litchi fruits and picking points are crucial for litchi-picking robots.Most studies adopt multi-step methods to detect fruit and locate picking points,which are slow and struggle to cope with complex environments.This study proposes a YOLOv8-iGR model based on YOLOv8n-pose improvement,integrating end-to-end network for both object detection and key point detection.Specifically,this study considers the influence of auxiliary points on picking point and designs four litchi key point strategies.Secondly,the architecture named iSaE is proposed,which combines the capabilities of CNN and attention mechanism.Subsequently,C2f is replaced by Generalized Efficient Layer Aggregation Network(GELAN)to reduce model redundancy and improve detection accuracy.Finally,based on RFAConv,RFAPoseHead is designed to address the issue of parameter sharing in large convolutional kernels,thereby more effectively extracting feature information.Experimental results demonstrate that YOLOv8-iGR achieves an AP of 95.7%in litchi fruit detection,and the Euclidean distance error of picking points is less than 8 pixels across different scenes,meeting the requirements of litchi picking.Additionally,the GFLOPs of the model are reduced by 10.71%.The accuracy of the model’s localization for picking points was tested through field picking experiments.In conclusion,YOLOv8-iGR exhibits outstanding detection performance along with lower model complexity,making it more feasible for implementation on robots.This will provide technical support for the vision system of the litchi-picking robot.展开更多
In order to obtain the determining method of the installing angle and decrease the performance indices (cutting force and wearing rate) of the pick, the relationships among the installing angles (impact angle, inclina...In order to obtain the determining method of the installing angle and decrease the performance indices (cutting force and wearing rate) of the pick, the relationships among the installing angles (impact angle, inclination angle and the skew angle) were studied, and the static model of installing angles of the pick was built. The relationships among the impact angle, the tip angle of pick and the kinematics parameters of the pick were built, too. Moreover, the mechanic models of the maximum clearance angle and the wearing angle of the pick were set up. To research the relationships of the installing angles and the change law of the wearing angle along with the kinematics parameters, the simulation was done. In order to verify the correctness of the models, the cutting experiments were done by employing two picks with different pick tip angles. The results indicate that, the cutting force is the smallest when the direction of the resultant force of pick follows its axis, and the relationship derived among the installing angles should be satisfied. In addition, to decrease the cutting force and the wearing of the pick, the tip angle of pick should not be larger than the half of the difference between the minimum wearing angle and the impact angle of the pick, and the clearance angle must not be less than zero.展开更多
In the complex orchard environment,precise picking point localization is crucial for the automation of fruit picking robots.However,existing methods are prone to positioning errors when dealing with complex scenarios ...In the complex orchard environment,precise picking point localization is crucial for the automation of fruit picking robots.However,existing methods are prone to positioning errors when dealing with complex scenarios such as short peduncles,partial occlusion,or complete misidentification,which can affect the actual work efficiency of the fruit picking robot.This study proposes an enhanced picking point localization method based on semantic reasoning for complex picking scenarios in vineyard.It innovatively designs three modules:the semantic reasoning module(SRM),the ROI threshold adjustment strategy(RTAS),and the picking point location optimization module(PPOM).The SRM is applied to handle the scenarios of grape peduncles being obstructed by obstacles,partial misidentification of peduncles,and complete misidentification of peduncles.The RTAS addresses the issue of low and short peduncles during the picking process.Finally,the PPOM optimizes the final position of the picking point,allowing the robotic arm to perform the picking operation with greater flexibility.Experimental results show that SegFormer achieves an mIoU(mean Intersection over Union)of 84.54%,with B_IoU and P_IoU reaching 73.90%and 75.63%,respectively.Additionally,the success rate of the improved fruit picking point localization algorithm reached 94.96%,surpassing the baseline algorithm by 8.12%.The algorithm's average processing time is 0.5428±0.0063 s,meeting the practical requirements for real-time picking.展开更多
The accurate identification,detection,and segmentation of tea buds and leaves are important factors for realizing intelligent tea picking.A tea picking point location method based on the region-based convolutional neu...The accurate identification,detection,and segmentation of tea buds and leaves are important factors for realizing intelligent tea picking.A tea picking point location method based on the region-based convolutional neural network(R-CNN)Mask-RCNN is proposed,and a tea bud and leaf and picking point recognition model is established.First,tea buds and leaf pictures are collected in a complex environment,the Resnet50 residual network and a feature pyramid network(FPN)are used to extract bud and leaf features,and preliminary classification and preselection box regression training-performed on the feature maps through a regional proposal network(RPN).Second,the regional feature aggregation method(RoIAlign)is used to eliminate the quantization error,and the feature map of the preselected region of interest(ROI)is converted into a fixed-size feature map.The output module of the model realizes the functions of classification,regression and segmentation.Finally,through the output mask image and the positioning algorithm the positioning of the picking points of tea buds and leaves is determined.One hundred tea tree bud and leaf pictures in a complex environment are selected for testing.The experimental results show that the average detection accuracy rate reaches 93.95%and that the recall rate reaches 92.48%.The tea picking location method presented in this paper is more versatile and robust in complex environments.展开更多
基金supported by Natural Science Foundation of Guangdong Province(Grant No.2025A1515011771)Guangzhou Science and Technology Plan Project(Grant No.2024E04J1242,2023B01J0046)+2 种基金Guangdong Provincial Department of Science and Technology(Grant No.2023A0505050130)Key Projects of Guangzhou Science and Technology Program(Grant No.2024B03J1357)Natural Science Foundation of China(Grant No.61863011,32071912).
文摘In the unstructured litchi orchard,precise identification and localization of litchi fruits and picking points are crucial for litchi-picking robots.Most studies adopt multi-step methods to detect fruit and locate picking points,which are slow and struggle to cope with complex environments.This study proposes a YOLOv8-iGR model based on YOLOv8n-pose improvement,integrating end-to-end network for both object detection and key point detection.Specifically,this study considers the influence of auxiliary points on picking point and designs four litchi key point strategies.Secondly,the architecture named iSaE is proposed,which combines the capabilities of CNN and attention mechanism.Subsequently,C2f is replaced by Generalized Efficient Layer Aggregation Network(GELAN)to reduce model redundancy and improve detection accuracy.Finally,based on RFAConv,RFAPoseHead is designed to address the issue of parameter sharing in large convolutional kernels,thereby more effectively extracting feature information.Experimental results demonstrate that YOLOv8-iGR achieves an AP of 95.7%in litchi fruit detection,and the Euclidean distance error of picking points is less than 8 pixels across different scenes,meeting the requirements of litchi picking.Additionally,the GFLOPs of the model are reduced by 10.71%.The accuracy of the model’s localization for picking points was tested through field picking experiments.In conclusion,YOLOv8-iGR exhibits outstanding detection performance along with lower model complexity,making it more feasible for implementation on robots.This will provide technical support for the vision system of the litchi-picking robot.
基金Project(51005232) supported by the National Natural Science Foundation of ChinaProject(20100481176) supported by the China Postdoctoral Science Foundation+1 种基金Project(201104583) supported by the China Postdoctoral Special FundProject(1101106c) supported by Jiangsu Postdoctoral Foundation, China
文摘In order to obtain the determining method of the installing angle and decrease the performance indices (cutting force and wearing rate) of the pick, the relationships among the installing angles (impact angle, inclination angle and the skew angle) were studied, and the static model of installing angles of the pick was built. The relationships among the impact angle, the tip angle of pick and the kinematics parameters of the pick were built, too. Moreover, the mechanic models of the maximum clearance angle and the wearing angle of the pick were set up. To research the relationships of the installing angles and the change law of the wearing angle along with the kinematics parameters, the simulation was done. In order to verify the correctness of the models, the cutting experiments were done by employing two picks with different pick tip angles. The results indicate that, the cutting force is the smallest when the direction of the resultant force of pick follows its axis, and the relationship derived among the installing angles should be satisfied. In addition, to decrease the cutting force and the wearing of the pick, the tip angle of pick should not be larger than the half of the difference between the minimum wearing angle and the impact angle of the pick, and the clearance angle must not be less than zero.
基金supported by the National Natural Science Foundation of China,China(32171909,52205254,32301704)the Guangdong Basic and Applied Basic Research Foundation,China(2023A1515011255,2020B1515120050,2024A1515010199)+1 种基金the Key Research Projects of Ordinary Universities in Guangdong Province,China(2024ZDZX1042,2024ZDZX3057)the Guangdong key R&D projects,China(202080404030001).
文摘In the complex orchard environment,precise picking point localization is crucial for the automation of fruit picking robots.However,existing methods are prone to positioning errors when dealing with complex scenarios such as short peduncles,partial occlusion,or complete misidentification,which can affect the actual work efficiency of the fruit picking robot.This study proposes an enhanced picking point localization method based on semantic reasoning for complex picking scenarios in vineyard.It innovatively designs three modules:the semantic reasoning module(SRM),the ROI threshold adjustment strategy(RTAS),and the picking point location optimization module(PPOM).The SRM is applied to handle the scenarios of grape peduncles being obstructed by obstacles,partial misidentification of peduncles,and complete misidentification of peduncles.The RTAS addresses the issue of low and short peduncles during the picking process.Finally,the PPOM optimizes the final position of the picking point,allowing the robotic arm to perform the picking operation with greater flexibility.Experimental results show that SegFormer achieves an mIoU(mean Intersection over Union)of 84.54%,with B_IoU and P_IoU reaching 73.90%and 75.63%,respectively.Additionally,the success rate of the improved fruit picking point localization algorithm reached 94.96%,surpassing the baseline algorithm by 8.12%.The algorithm's average processing time is 0.5428±0.0063 s,meeting the practical requirements for real-time picking.
基金the Key Research and Development Project of Anhui Province(1804a07020108,201904a06020056,202104a06020012)Independent Project of Anhui Key Laboratory of Smart Agricultural Technology and Equipment(APKLSATE2019X001).
文摘The accurate identification,detection,and segmentation of tea buds and leaves are important factors for realizing intelligent tea picking.A tea picking point location method based on the region-based convolutional neural network(R-CNN)Mask-RCNN is proposed,and a tea bud and leaf and picking point recognition model is established.First,tea buds and leaf pictures are collected in a complex environment,the Resnet50 residual network and a feature pyramid network(FPN)are used to extract bud and leaf features,and preliminary classification and preselection box regression training-performed on the feature maps through a regional proposal network(RPN).Second,the regional feature aggregation method(RoIAlign)is used to eliminate the quantization error,and the feature map of the preselected region of interest(ROI)is converted into a fixed-size feature map.The output module of the model realizes the functions of classification,regression and segmentation.Finally,through the output mask image and the positioning algorithm the positioning of the picking points of tea buds and leaves is determined.One hundred tea tree bud and leaf pictures in a complex environment are selected for testing.The experimental results show that the average detection accuracy rate reaches 93.95%and that the recall rate reaches 92.48%.The tea picking location method presented in this paper is more versatile and robust in complex environments.
