In general, the orientation interpolation of industrial robots has been done based on Euler angle system which can result in singular point （so-called Gimbal Lock）. However, quaternion interpolation has the advantag...In general, the orientation interpolation of industrial robots has been done based on Euler angle system which can result in singular point （so-called Gimbal Lock）. However, quaternion interpolation has the advantage of natural （specifically smooth） orientation interpolation without Gimbal Lock. This work presents the application of quatemion interpolation, specifically Spherical Linear IntERPolation （SLERP）, to the orientation control of the 6-axis articulated robot （RS2） using LabVIEW and RecurDyn. For the comparison of SLERP with linear Euler interpolation in the view of smooth movement （profile） of joint angles （torques）, the two methods are dynamically simulated on RS2 by using both LabVIEW and RecurDyn. Finally, our original work, specifically the implementation of SLERP and linear Euler interpolation on the actual robot, i.e. RS2, is done using LabVIEW motion control tool kit. The SLERP orientation control is shown to be effective in terms of smooth joint motion and torque when compared to a conventional （linear） Euler interpolation.展开更多
Underwater pipeline inspection plays a vital role in the proactive maintenance and management of critical marine infrastructure and subaquatic systems.However,the inspection of underwater pipelines presents a challeng...Underwater pipeline inspection plays a vital role in the proactive maintenance and management of critical marine infrastructure and subaquatic systems.However,the inspection of underwater pipelines presents a challenge due to factors such as light scattering,absorption,restricted visibility,and ambient noise.The advancement of deep learning has introduced powerful techniques for processing large amounts of unstructured and imperfect data collected from underwater environments.This study evaluated the efficacy of the You Only Look Once(YOLO)algorithm,a real-time object detection and localization model based on convolutional neural networks,in identifying and classifying various types of pipeline defects in underwater settings.YOLOv8,the latest evolution in the YOLO family,integrates advanced capabilities,such as anchor-free detection,a cross-stage partial network backbone for efficient feature extraction,and a feature pyramid network+path aggregation network neck for robust multi-scale object detection,which make it particularly well-suited for complex underwater environments.Due to the lack of suitable open-access datasets for underwater pipeline defects,a custom dataset was captured using a remotely operated vehicle in a controlled environment.This application has the following assets available for use.Extensive experimentation demonstrated that YOLOv8 X-Large consistently outperformed other models in terms of pipe defect detection and classification and achieved a strong balance between precision and recall in identifying pipeline cracks,rust,corners,defective welds,flanges,tapes,and holes.This research establishes the baseline performance of YOLOv8 for underwater defect detection and showcases its potential to enhance the reliability and efficiency of pipeline inspection tasks in challenging underwater environments.展开更多
In the unstructured litchi orchard,precise identification and localization of litchi fruits and picking points are crucial for litchi-picking robots.Most studies adopt multi-step methods to detect fruit and locate pic...In the unstructured litchi orchard,precise identification and localization of litchi fruits and picking points are crucial for litchi-picking robots.Most studies adopt multi-step methods to detect fruit and locate picking points,which are slow and struggle to cope with complex environments.This study proposes a YOLOv8-iGR model based on YOLOv8n-pose improvement,integrating end-to-end network for both object detection and key point detection.Specifically,this study considers the influence of auxiliary points on picking point and designs four litchi key point strategies.Secondly,the architecture named iSaE is proposed,which combines the capabilities of CNN and attention mechanism.Subsequently,C2f is replaced by Generalized Efficient Layer Aggregation Network(GELAN)to reduce model redundancy and improve detection accuracy.Finally,based on RFAConv,RFAPoseHead is designed to address the issue of parameter sharing in large convolutional kernels,thereby more effectively extracting feature information.Experimental results demonstrate that YOLOv8-iGR achieves an AP of 95.7%in litchi fruit detection,and the Euclidean distance error of picking points is less than 8 pixels across different scenes,meeting the requirements of litchi picking.Additionally,the GFLOPs of the model are reduced by 10.71%.The accuracy of the model’s localization for picking points was tested through field picking experiments.In conclusion,YOLOv8-iGR exhibits outstanding detection performance along with lower model complexity,making it more feasible for implementation on robots.This will provide technical support for the vision system of the litchi-picking robot.展开更多
To address the challenges of harsh harvesting environments,high labor intensity,and low picking efficiency in tomato harvesting,this study investigates the key technologies related to the end-effector design,detection...To address the challenges of harsh harvesting environments,high labor intensity,and low picking efficiency in tomato harvesting,this study investigates the key technologies related to the end-effector design,detection and recognition,and spatial localization of tomato-picking robots.A non-contact cavity-type end-effector is designed,which effectively prevents tomato damage caused by compression during picking while preserving the peduncle.Additionally,the motion of the robotic arm is simulated for performance analysis.Subsequently,tomato images are captured and annotated for training deep neural network models.Both the original YOLO v8n and the improved YOLO v8n models are used for tomato image detection,with a focus on the impact of varying light intensities and different tomato maturities on recognition and localization accuracy.Experimental results demonstrate that the robot’s vision system achieves optimal recognition and localization performance under light intensities ranging from 20000 to 30000 lx,with an accuracy of 91.5%,an average image detection speed of 15.1 ms per image,and an absolute localization error of 1.55 cm.Furthermore,the prototype tomato-picking robot’s end-effector successfully performed stable grasping of individual tomatoes without damaging the skin,achieving a picking success rate of 83.3%,with an average picking time of approximately 9.5 s per fruit.This study provides a technical support for the automated harvesting of tomato-picking robots.展开更多
Instance segmentation,an important image processing operation for automation in agriculture,is used to precisely delineate individual objects of interestwithin images,which provides foundational information for variou...Instance segmentation,an important image processing operation for automation in agriculture,is used to precisely delineate individual objects of interestwithin images,which provides foundational information for various automated or robotic tasks such as selective harvesting and precision pruning.This study compares the one-stage YOLOv8 and the two-stage Mask R-CNN machine learning models for instance segmentation under varying orchard conditions across two datasets.Dataset 1,collected in dormant season,includes images of dormant apple trees,which were used to train multi-object segmentation models delineating tree branches and trunks.Dataset 2,collected in the early growing season,includes images of apple tree canopies with green foliage and immature(green)apples(also called fruitlet),which were used to train single-object segmentation models delineating only immature green apples.The results showed that YOLOv8 performed better than Mask R-CNN,achieving good precision and near-perfect recall across both datasets at a confidence threshold of 0.5.Specifically,for Dataset 1,YOLOv8 achieved a precision of 0.90 and a recall of 0.95 for all classes.In comparison,Mask R-CNN demonstrated a precision of 0.81 and a recall of 0.81 for the samedataset.With Dataset 2,YOLOv8 achieved a precision of 0.93 and a recall of 0.97.Mask R-CNN,in this single-class scenario,achieved a precision of 0.85 and a recall of 0.88.Additionally,the inference times for YOLOv8 were 10.9 ms for multi-class segmentation(Dataset 1)and 7.8 ms for single-class segmentation(Dataset 2),compared to 15.6 ms and 12.8 ms achieved by Mask R-CNN's,respectively.These findings showYOLOv8's superior accuracy and efficiency in machine learning applications compared to two-stage models,specifically Mask-R-CNN,which suggests its suitability in developing smart and automated orchard operations,particularly when real-time applications are necessary in such cases as robotic harvesting and robotic immature green fruit thinning.展开更多
基金Project supported by the Second Stage of Brain Korea 21 Projectssupported by Basic Science Research Program through the National Research Foundation of Korea (NRF)funded by the Ministry of Education,Science and Technology (2011-0013902)
文摘In general, the orientation interpolation of industrial robots has been done based on Euler angle system which can result in singular point （so-called Gimbal Lock）. However, quaternion interpolation has the advantage of natural （specifically smooth） orientation interpolation without Gimbal Lock. This work presents the application of quatemion interpolation, specifically Spherical Linear IntERPolation （SLERP）, to the orientation control of the 6-axis articulated robot （RS2） using LabVIEW and RecurDyn. For the comparison of SLERP with linear Euler interpolation in the view of smooth movement （profile） of joint angles （torques）, the two methods are dynamically simulated on RS2 by using both LabVIEW and RecurDyn. Finally, our original work, specifically the implementation of SLERP and linear Euler interpolation on the actual robot, i.e. RS2, is done using LabVIEW motion control tool kit. The SLERP orientation control is shown to be effective in terms of smooth joint motion and torque when compared to a conventional （linear） Euler interpolation.
文摘Underwater pipeline inspection plays a vital role in the proactive maintenance and management of critical marine infrastructure and subaquatic systems.However,the inspection of underwater pipelines presents a challenge due to factors such as light scattering,absorption,restricted visibility,and ambient noise.The advancement of deep learning has introduced powerful techniques for processing large amounts of unstructured and imperfect data collected from underwater environments.This study evaluated the efficacy of the You Only Look Once(YOLO)algorithm,a real-time object detection and localization model based on convolutional neural networks,in identifying and classifying various types of pipeline defects in underwater settings.YOLOv8,the latest evolution in the YOLO family,integrates advanced capabilities,such as anchor-free detection,a cross-stage partial network backbone for efficient feature extraction,and a feature pyramid network+path aggregation network neck for robust multi-scale object detection,which make it particularly well-suited for complex underwater environments.Due to the lack of suitable open-access datasets for underwater pipeline defects,a custom dataset was captured using a remotely operated vehicle in a controlled environment.This application has the following assets available for use.Extensive experimentation demonstrated that YOLOv8 X-Large consistently outperformed other models in terms of pipe defect detection and classification and achieved a strong balance between precision and recall in identifying pipeline cracks,rust,corners,defective welds,flanges,tapes,and holes.This research establishes the baseline performance of YOLOv8 for underwater defect detection and showcases its potential to enhance the reliability and efficiency of pipeline inspection tasks in challenging underwater environments.
基金supported by Natural Science Foundation of Guangdong Province(Grant No.2025A1515011771)Guangzhou Science and Technology Plan Project(Grant No.2024E04J1242,2023B01J0046)+2 种基金Guangdong Provincial Department of Science and Technology(Grant No.2023A0505050130)Key Projects of Guangzhou Science and Technology Program(Grant No.2024B03J1357)Natural Science Foundation of China(Grant No.61863011,32071912).
文摘In the unstructured litchi orchard,precise identification and localization of litchi fruits and picking points are crucial for litchi-picking robots.Most studies adopt multi-step methods to detect fruit and locate picking points,which are slow and struggle to cope with complex environments.This study proposes a YOLOv8-iGR model based on YOLOv8n-pose improvement,integrating end-to-end network for both object detection and key point detection.Specifically,this study considers the influence of auxiliary points on picking point and designs four litchi key point strategies.Secondly,the architecture named iSaE is proposed,which combines the capabilities of CNN and attention mechanism.Subsequently,C2f is replaced by Generalized Efficient Layer Aggregation Network(GELAN)to reduce model redundancy and improve detection accuracy.Finally,based on RFAConv,RFAPoseHead is designed to address the issue of parameter sharing in large convolutional kernels,thereby more effectively extracting feature information.Experimental results demonstrate that YOLOv8-iGR achieves an AP of 95.7%in litchi fruit detection,and the Euclidean distance error of picking points is less than 8 pixels across different scenes,meeting the requirements of litchi picking.Additionally,the GFLOPs of the model are reduced by 10.71%.The accuracy of the model’s localization for picking points was tested through field picking experiments.In conclusion,YOLOv8-iGR exhibits outstanding detection performance along with lower model complexity,making it more feasible for implementation on robots.This will provide technical support for the vision system of the litchi-picking robot.
基金supported by State Key Laboratory of Precision Blasting and Hubei Key Laboratory of Blasting Engineering,Jianghan University(Grant No.BL2021-16)the Natural Science Foundation of Hubei Province(Grant No.2025AFB176).
文摘To address the challenges of harsh harvesting environments,high labor intensity,and low picking efficiency in tomato harvesting,this study investigates the key technologies related to the end-effector design,detection and recognition,and spatial localization of tomato-picking robots.A non-contact cavity-type end-effector is designed,which effectively prevents tomato damage caused by compression during picking while preserving the peduncle.Additionally,the motion of the robotic arm is simulated for performance analysis.Subsequently,tomato images are captured and annotated for training deep neural network models.Both the original YOLO v8n and the improved YOLO v8n models are used for tomato image detection,with a focus on the impact of varying light intensities and different tomato maturities on recognition and localization accuracy.Experimental results demonstrate that the robot’s vision system achieves optimal recognition and localization performance under light intensities ranging from 20000 to 30000 lx,with an accuracy of 91.5%,an average image detection speed of 15.1 ms per image,and an absolute localization error of 1.55 cm.Furthermore,the prototype tomato-picking robot’s end-effector successfully performed stable grasping of individual tomatoes without damaging the skin,achieving a picking success rate of 83.3%,with an average picking time of approximately 9.5 s per fruit.This study provides a technical support for the automated harvesting of tomato-picking robots.
基金funded by the National Science Foundation and United States Department of Agriculture,National Institute of Food and Agriculture through the“AI Institute for Agriculture”Program(Award No.AWD003473).
文摘Instance segmentation,an important image processing operation for automation in agriculture,is used to precisely delineate individual objects of interestwithin images,which provides foundational information for various automated or robotic tasks such as selective harvesting and precision pruning.This study compares the one-stage YOLOv8 and the two-stage Mask R-CNN machine learning models for instance segmentation under varying orchard conditions across two datasets.Dataset 1,collected in dormant season,includes images of dormant apple trees,which were used to train multi-object segmentation models delineating tree branches and trunks.Dataset 2,collected in the early growing season,includes images of apple tree canopies with green foliage and immature(green)apples(also called fruitlet),which were used to train single-object segmentation models delineating only immature green apples.The results showed that YOLOv8 performed better than Mask R-CNN,achieving good precision and near-perfect recall across both datasets at a confidence threshold of 0.5.Specifically,for Dataset 1,YOLOv8 achieved a precision of 0.90 and a recall of 0.95 for all classes.In comparison,Mask R-CNN demonstrated a precision of 0.81 and a recall of 0.81 for the samedataset.With Dataset 2,YOLOv8 achieved a precision of 0.93 and a recall of 0.97.Mask R-CNN,in this single-class scenario,achieved a precision of 0.85 and a recall of 0.88.Additionally,the inference times for YOLOv8 were 10.9 ms for multi-class segmentation(Dataset 1)and 7.8 ms for single-class segmentation(Dataset 2),compared to 15.6 ms and 12.8 ms achieved by Mask R-CNN's,respectively.These findings showYOLOv8's superior accuracy and efficiency in machine learning applications compared to two-stage models,specifically Mask-R-CNN,which suggests its suitability in developing smart and automated orchard operations,particularly when real-time applications are necessary in such cases as robotic harvesting and robotic immature green fruit thinning.
