To support the process of grasping objects on a tabletop for the blind or robotic arm,it is necessary to address fundamental computer vision tasks,such as detecting,recognizing,and locating objects in space,and determ...To support the process of grasping objects on a tabletop for the blind or robotic arm,it is necessary to address fundamental computer vision tasks,such as detecting,recognizing,and locating objects in space,and determining the position of the grasping information.These results can then be used to guide the visually impaired or to execute grasping tasks with a robotic arm.In this paper,we collected,annotated,and published the benchmark TQUGraspingObject dataset for testing,validation,and evaluation of deep learning(DL)models for detecting,recognizing,and localizing grasping objects in 2D and 3D space,especially 3D point cloud data.Our dataset is collected in a shared room,with common everyday objects placed on the tabletop in jumbled positions by Intel RealSense D435(IR-D435).This dataset includes more than 63k RGB-D pairs and related data such as normalized 3D object point cloud,3D object point cloud segmented,coordinate system normalizationmatrix,3D object point cloud normalized,and hand pose for grasping each object.At the same time,we also conducted experiments on fourDL networks with the best performance:SSD-MobileNetV3,ResNet50-Transformer,ResNet101-Transformer,and YOLOv12.The results present that YOLOv12 has the most suitable results in detecting and recognizing objects in images.All data,annotations,toolkit,source code,point cloud data,and results are publicly available on our project website:https://github.com/HuaTThanhIT2327Tqu/datasetv2.展开更多
Deep learning has become integral to robotics,particularly in tasks such as robotic grasping,where objects often exhibit diverse shapes,textures,and physical properties.In robotic grasping tasks,due to the diverse cha...Deep learning has become integral to robotics,particularly in tasks such as robotic grasping,where objects often exhibit diverse shapes,textures,and physical properties.In robotic grasping tasks,due to the diverse characteristics of the targets,frequent adjustments to the network architecture and parameters are required to avoid a decrease in model accuracy,which presents a significant challenge for non-experts.Neural Architecture Search(NAS)provides a compelling method through the automated generation of network architectures,enabling the discovery of models that achieve high accuracy through efficient search algorithms.Compared to manually designed networks,NAS methods can significantly reduce design costs,time expenditure,and improve model performance.However,such methods often involve complex topological connections,and these redundant structures can severely reduce computational efficiency.To overcome this challenge,this work puts forward a robotic grasp detection framework founded on NAS.The method automatically designs a lightweight network with high accuracy and low topological complexity,effectively adapting to the target object to generate the optimal grasp pose,thereby significantly improving the success rate of robotic grasping.Additionally,we use Class Activation Mapping(CAM)as an interpretability tool,which captures sensitive information during the perception process through visualized results.The searched model achieved competitive,and in some cases superior,performance on the Cornell and Jacquard public datasets,achieving accuracies of 98.3%and 96.8%,respectively,while sustaining a detection speed of 89 frames per second with only 0.41 million parameters.To further validate its effectiveness beyond benchmark evaluations,we conducted real-world grasping experiments on a UR5 robotic arm,where the model demonstrated reliable performance across diverse objects and high grasp success rates,thereby confirming its practical applicability in robotic manipulation tasks.展开更多
Soft grippers research is gaining increasing attention for their flexibility.However,the conventional soft gripper primar-ily focuses on soft fingers,without considering the palm.This makes grasping forces concentrate...Soft grippers research is gaining increasing attention for their flexibility.However,the conventional soft gripper primar-ily focuses on soft fingers,without considering the palm.This makes grasping forces concentrated in the fingertip areas,resulting in objects being prone to damage and instability during handling,especially for delicate items.Additionally,pre-transportation classification faces challenges:tactile methods are complex,visual methods are environment-sensitive,and both struggle with similar objects.To address these problems,inspired by the human hand's transition between finger grasp and palm support and the lotus's hierarchical structure,this paper proposes a dual-layer gripper,named IOSGrip-per.It features four pneumatic soft fingers and a rotational soft-rigid palm.Through coordinated control of the fingers and palm,it transitions concentrated fingertip squeeze force to distributed palm support force,reducing squeeze force and squeeze duration.Moreover,it integrates a range sensor and four load cells,enabling stable and accurate measurements of the object's height and weight.Furthermore,a classifier is developed based on K-nearest neighbor algorithm,allowing real-time object classification.Experiments demonstrate that compared to only using soft fingers,the IOSGripper signifi-cantly reduces the squeeze force on the objects(with 0 N squeeze force during palm support)and damage on the delicate object,while improving grasping stability.Its height and weight measurement errors are within 2 mm and 1 g,respectively.And it achieves high accuracy in three test scenarios,including classifying similar objects.This study provides useful insights for the design of soft grippers capable of human-like grasping and sorting tasks.展开更多
The word“spatial”fundamentally relates to human existence,evolution,and activity in terrestrial and even celestial spaces.After reviewing the spatial features of many areas,the paper describes basics of high level m...The word“spatial”fundamentally relates to human existence,evolution,and activity in terrestrial and even celestial spaces.After reviewing the spatial features of many areas,the paper describes basics of high level model and technology called Spatial Grasp for dealing with large distributed systems,which can provide spatial vision,awareness,management,control,and even consciousness.The technology description includes its key Spatial Grasp Language(SGL),self-evolution of recursive SGL scenarios,and implementation of SGL interpreter converting distributed networked systems into powerful spatial engines.Examples of typical spatial scenarios in SGL include finding shortest path tree and shortest path between network nodes,collecting proper information throughout the whole world,elimination of multiple targets by intelligent teams of chasers,and withstanding cyber attacks in distributed networked systems.Also this paper compares Spatial Grasp model with traditional algorithms,confirming universality of the former for any spatial systems,while the latter just tools for concrete applications.展开更多
“Multidimensional international world”refers to understanding the world through multiple dimensions beyond traditional economic or political measures,fostering cross-cultural collaboration,and creating systems that ...“Multidimensional international world”refers to understanding the world through multiple dimensions beyond traditional economic or political measures,fostering cross-cultural collaboration,and creating systems that balance global integration with local needs.This also includes management of global business operations across diverse cultures in a multipolar international landscape.The paper briefs the developed and already tested in numerous applications high-level Spatial Grasp Model and Technology(SGT),which can help investigate and manage complex systems with a holistic spatial approach effectively covering various physical and virtual dimensions,their interrelations,and integration as a whole.Different areas will be investigated with examples of practical solutions in them and their combinations in a high-level Spatial Grasp Language(SGL),the key element of SGT.This allows for the creation and distributed management of very large spatial networks with different orientation which can be self-spreading,self-analyzing,self-modifying,and self-recovering in complex terrestrial and celestial environments,and also organize dynamic multi-networking solutions supporting global evolution and integrity.展开更多
With the rapid development of robotics,grasp prediction has become fundamental to achieving intelligent physical interactions.To enhance grasp detection accuracy in unstructured environments,we propose a novel Cross-M...With the rapid development of robotics,grasp prediction has become fundamental to achieving intelligent physical interactions.To enhance grasp detection accuracy in unstructured environments,we propose a novel Cross-Multiscale Adaptive Collaborative and Fusion Grasp Detection Network(CMACF-Net).Addressing the limitations of conventional methods in capturing multi-scale spatial features,CMACF-Net introduces the Quantized Multi-scale Global Attention Module(QMGAM),which enables precise multi-scale spatial calibration and adaptive spatial-channel interaction,ultimately yielding a more robust and discriminative feature representation.To reduce the degradation of local features and the loss of high-frequency information,the Cross-scale Context Integration Module(CCI)is employed to facilitate the effective fusion and alignment of global context and local details.Furthermore,an Efficient Up-Convolution Block(EUCB)is integrated into a U-Net architecture to effectively restore spatial details lost during the downsampling process,while simultaneously preserving computational efficiency.Extensive evaluations demonstrate that CMACF-Net achieves state-of-the-art detection accuracies of 98.9% and 95.9% on the Cornell and Jacquard datasets,respectively.Additionally,real-time grasping experiments on the RM65-B robotic platform validate the framework’s robustness and generalization capability,underscoring its applicability to real-world robotic manipulation scenarios.展开更多
Robots are key to expanding the scope of space applications.The end-to-end training for robot vision-based detection and precision operations is challenging owing to constraints such as extreme environments and high c...Robots are key to expanding the scope of space applications.The end-to-end training for robot vision-based detection and precision operations is challenging owing to constraints such as extreme environments and high computational overhead.This study proposes a lightweight integrated framework for grasp detection and imitation learning,named GD-IL;it comprises a grasp detection algorithm based on manipulability and Gaussian mixture model(manipulability-GMM),and a grasp trajectory generation algorithm based on a two-stage robot imitation learning algorithm(TS-RIL).In the manipulability-GMM algorithm,we apply GMM clustering and ellipse regression to the object point cloud,propose two judgment criteria to generate multiple candidate grasp bounding boxes for the robot,and use manipulability as a metric for selecting the optimal grasp bounding box.The stages of the TS-RIL algorithm are grasp trajectory learning and robot pose optimization.In the first stage,the robot grasp trajectory is characterized using a second-order dynamic movement primitive model and Gaussian mixture regression(GMM).By adjusting the function form of the forcing term,the robot closely approximates the target-grasping trajectory.In the second stage,a robot pose optimization model is built based on the derived pose error formula and manipulability metric.This model allows the robot to adjust its configuration in real time while grasping,thereby effectively avoiding singularities.Finally,an algorithm verification platform is developed based on a Robot Operating System and a series of comparative experiments are conducted in real-world scenarios.The experimental results demonstrate that GD-IL significantly improves the effectiveness and robustness of grasp detection and trajectory imitation learning,outperforming existing state-of-the-art methods in execution efficiency,manipulability,and success rate.展开更多
When the space robot captures a floating target, contact impact occurs inevitably and frequently between the manipulator hand and the target, which seriously impacts the position and attitude of the robot and grasping...When the space robot captures a floating target, contact impact occurs inevitably and frequently between the manipulator hand and the target, which seriously impacts the position and attitude of the robot and grasping security. ＂Dynamic grasping area＂ is introduced to describe the collision process of manipulator grasping target, and grasping area control equation is established. By analyzing the impact of grasping control parameters, base and target mass on the grasping process and combining the life experience, it is found that if the product of speed control parameter and dB adjustment parameter is close to but smaller than the minimum grasping speed, collision impact in the grasping process could be reduced greatly, and then an ideal grasping strategy is proposed. Simulation results indicate that during the same period, the strategy grasping is superior to the accelerating grasping, in that the amplitude of impact force is reduced to 20%, and the attitude control torque is reduced to 15%, and the impact on the robot is eliminated significantly. The results would have important academic value and engineering significance.展开更多
In recent years,robotic arm grasping has become a pivotal task in the field of robotics,with applications spanning from industrial automation to healthcare.The optimization of grasping strategies plays a crucial role ...In recent years,robotic arm grasping has become a pivotal task in the field of robotics,with applications spanning from industrial automation to healthcare.The optimization of grasping strategies plays a crucial role in enhancing the effectiveness,efficiency,and reliability of robotic systems.This paper presents a novel approach to optimizing robotic arm grasping strategies based on deep reinforcement learning(DRL).Through the utilization of advanced DRL algorithms,such as Q-Learning,Deep Q-Networks(DQN),Policy Gradient Methods,and Proximal Policy Optimization(PPO),the study aims to improve the performance of robotic arms in grasping objects with varying shapes,sizes,and environmental conditions.The paper provides a detailed analysis of the various deep reinforcement learning methods used for grasping strategy optimization,emphasizing the strengths and weaknesses of each algorithm.It also presents a comprehensive framework for training the DRL models,including simulation environment setup,the optimization process,and the evaluation metrics for grasping success.The results demonstrate that the proposed approach significantly enhances the accuracy and stability of the robotic arm in performing grasping tasks.The study further explores the challenges in training deep reinforcement learning models for real-time robotic applications and offers solutions for improving the efficiency and reliability of grasping strategies.展开更多
Currently,numerous biomimetic robots inspired by natural biological systems have been developed.However,creating soft robots with versatile locomotion modes remains a significant challenge.Snakes,as invertebrate repti...Currently,numerous biomimetic robots inspired by natural biological systems have been developed.However,creating soft robots with versatile locomotion modes remains a significant challenge.Snakes,as invertebrate reptiles,exhibit diverse and powerful locomotion abilities,including prey constriction,sidewinding,accordion locomotion,and winding climbing,making them a focus of robotics research.In this study,we present a snake-inspired soft robot with an initial coiling structure,fabricated using MXene-cellulose nanofiber ink printed on pre-expanded polyethylene film through direct ink writing technology.The controllable fabrication of initial coiling structure soft robot(ICSBot)has been achieved through theoretical calculations and finite element analysis to predict and analyze the initial structure of ICSBot,and programmable ICSBot has been designed and fabricated.This robot functions as a coiling gripper capable of grasping objects with complex shapes under near infrared light stimulation.Additionally,it demonstrates multi-modal crawling locomotion in various environments,including confined spaces,unstructured terrains,and both inside and outside tubes.These results offer a novel strategy for designing and fabricating coiling-structured soft robots and highlight their potential applications in smart and multifunctional robotics.展开更多
Recent years have witnessed unprecedented development in humanoid robotics,with dexterous hand grasping emerging as a focal research area across industrial and academic sectors.To track the state-of-the-art dexterous ...Recent years have witnessed unprecedented development in humanoid robotics,with dexterous hand grasping emerging as a focal research area across industrial and academic sectors.To track the state-of-the-art dexterous hand grasp,a review of dexterous hand grasp based on bibliometric analysis is executed.The related studies on dexterous hand grasp are collected from the Web of Science for analysis,where the publication details and cooperation situations from the perspectives of country,institute,etc.are discussed.The keywords cluster is adopted to find the hot research topic of dexterous hand grasp.The development trend of dexterous hand grasp is explored based on the top 25 keywords with the strongest citation bursts.The review findings indicate that precision control via multimodal fusion,autonomous task understanding and intelligent decision,and in-hand dexterous manipulation are top three hotspots in future.展开更多
The“visual perception+hand-eye transformation+motion planning”paradigm of robotic coordination grasping has demonstrated feasibility in unstructured scenes such as logistics.However,further developments in handling ...The“visual perception+hand-eye transformation+motion planning”paradigm of robotic coordination grasping has demonstrated feasibility in unstructured scenes such as logistics.However,further developments in handling complex and dynamic environments remain challenging.To address the issue of unknown targets requiring immediate deployment for grasping tasks,this paper proposes a novel hand-eye coordinated method for progressive grasping guided by the texture keypoints of the target.First,we develop an efficient system for acquiring texture-matching templates and an estimation algorithm for the feature region that filters the precisely registered texture feature points of the target.Then,we integrate optical flow estimation to update and track the robust texture region in real time,and design a feature-based servo grasping controller to map the optical flow points of the high-registration texture region to the robot joint velocities for precise tracking.Finally,we impose spatiotemporal constraints on the planned trajectory and decouple the target motion,to achieve progressive approach and rotationally invariant grasping for both dynamic and static targets.Comprehensive experiments demonstrate that this tracking grasping method exhibits a low latency,a high precision,and robustness in complex scenarios and dynamic disturbances,with an average position accuracy of approximately 5 mm,a rotation accuracy of approximately 0.02,and an overall grasping success rate of approximately 90%.展开更多
Robot grasp detection is a fundamental vision task for robots.Deep learning-based methods have shown excellent results in enhancing the grasp detection capabilities for model-free objects in unstructured scenes.Most p...Robot grasp detection is a fundamental vision task for robots.Deep learning-based methods have shown excellent results in enhancing the grasp detection capabilities for model-free objects in unstructured scenes.Most popular approaches explore deep network models and exploit RGB-D images combining colour and depth data to acquire enriched feature expressions.However,current work struggles to achieve a satisfactory balance between the accuracy and real-time performance;the variability of RGB and depth feature distributions receives inadequate attention.The treatment of predicted failure cases is also lacking.We propose an efficient fully convolutional network to predict the pixel-level antipodal grasp parameters in RGB-D images.A structure with hierarchical feature fusion is established using multiple lightweight feature extraction blocks.The feature fusion module with 3D global attention is used to select the complementary information in RGB and depth images suficiently.Additionally,a grasp configuration optimization method based on local grasp path is proposed to cope with the possible failures predicted by the model.Extensive experiments on two public grasping datasets,Cornell and Jacquard,demonstrate that the approach can improve the performance of grasping unknown objects.展开更多
Grasping is one of the most fundamental operations in modern robotics applications.While deep rein-forcement learning(DRL)has demonstrated strong potential in robotics,there is too much emphasis on maximizing the cumu...Grasping is one of the most fundamental operations in modern robotics applications.While deep rein-forcement learning(DRL)has demonstrated strong potential in robotics,there is too much emphasis on maximizing the cumulative reward in executing tasks,and the potential safety risks are often ignored.In this paper,an optimization method based on safe reinforcement learning(Safe RL)is proposed to address the robotic grasping problem under safety constraints.Specifically,considering the obstacle avoidance constraints of the system,the grasping problem of the manipulator is modeled as a Constrained Markov Decision Process(CMDP).The Lagrange multiplier and a dynamic weighted mechanism are introduced into the Proximal Policy Optimization(PPO)framework,leading to the development of the dynamic weighted Lagrange PPO(DWL-PPO)algorithm.The behavior of violating safety constraints is punished while the policy is optimized in this proposed method.In addition,the orientation control of the end-effector is included in the reward function,and a compound reward function adapted to changes in pose is designed.Ultimately,the efficacy and advantages of the suggested method are proved by extensive training and testing in the Pybullet simulator.The results of grasping experiments reveal that the recommended approach provides superior safety and efficiency compared with other advanced RL methods and achieves a good trade-off between model learning and risk aversion.展开更多
文摘To support the process of grasping objects on a tabletop for the blind or robotic arm,it is necessary to address fundamental computer vision tasks,such as detecting,recognizing,and locating objects in space,and determining the position of the grasping information.These results can then be used to guide the visually impaired or to execute grasping tasks with a robotic arm.In this paper,we collected,annotated,and published the benchmark TQUGraspingObject dataset for testing,validation,and evaluation of deep learning(DL)models for detecting,recognizing,and localizing grasping objects in 2D and 3D space,especially 3D point cloud data.Our dataset is collected in a shared room,with common everyday objects placed on the tabletop in jumbled positions by Intel RealSense D435(IR-D435).This dataset includes more than 63k RGB-D pairs and related data such as normalized 3D object point cloud,3D object point cloud segmented,coordinate system normalizationmatrix,3D object point cloud normalized,and hand pose for grasping each object.At the same time,we also conducted experiments on fourDL networks with the best performance:SSD-MobileNetV3,ResNet50-Transformer,ResNet101-Transformer,and YOLOv12.The results present that YOLOv12 has the most suitable results in detecting and recognizing objects in images.All data,annotations,toolkit,source code,point cloud data,and results are publicly available on our project website:https://github.com/HuaTThanhIT2327Tqu/datasetv2.
基金funded by Guangdong Basic and Applied Basic Research Foundation(2023B1515120064)National Natural Science Foundation of China(62273097).
文摘Deep learning has become integral to robotics,particularly in tasks such as robotic grasping,where objects often exhibit diverse shapes,textures,and physical properties.In robotic grasping tasks,due to the diverse characteristics of the targets,frequent adjustments to the network architecture and parameters are required to avoid a decrease in model accuracy,which presents a significant challenge for non-experts.Neural Architecture Search(NAS)provides a compelling method through the automated generation of network architectures,enabling the discovery of models that achieve high accuracy through efficient search algorithms.Compared to manually designed networks,NAS methods can significantly reduce design costs,time expenditure,and improve model performance.However,such methods often involve complex topological connections,and these redundant structures can severely reduce computational efficiency.To overcome this challenge,this work puts forward a robotic grasp detection framework founded on NAS.The method automatically designs a lightweight network with high accuracy and low topological complexity,effectively adapting to the target object to generate the optimal grasp pose,thereby significantly improving the success rate of robotic grasping.Additionally,we use Class Activation Mapping(CAM)as an interpretability tool,which captures sensitive information during the perception process through visualized results.The searched model achieved competitive,and in some cases superior,performance on the Cornell and Jacquard public datasets,achieving accuracies of 98.3%and 96.8%,respectively,while sustaining a detection speed of 89 frames per second with only 0.41 million parameters.To further validate its effectiveness beyond benchmark evaluations,we conducted real-world grasping experiments on a UR5 robotic arm,where the model demonstrated reliable performance across diverse objects and high grasp success rates,thereby confirming its practical applicability in robotic manipulation tasks.
基金the Major research program of national natural science foundation of China(91848206).
文摘Soft grippers research is gaining increasing attention for their flexibility.However,the conventional soft gripper primar-ily focuses on soft fingers,without considering the palm.This makes grasping forces concentrated in the fingertip areas,resulting in objects being prone to damage and instability during handling,especially for delicate items.Additionally,pre-transportation classification faces challenges:tactile methods are complex,visual methods are environment-sensitive,and both struggle with similar objects.To address these problems,inspired by the human hand's transition between finger grasp and palm support and the lotus's hierarchical structure,this paper proposes a dual-layer gripper,named IOSGrip-per.It features four pneumatic soft fingers and a rotational soft-rigid palm.Through coordinated control of the fingers and palm,it transitions concentrated fingertip squeeze force to distributed palm support force,reducing squeeze force and squeeze duration.Moreover,it integrates a range sensor and four load cells,enabling stable and accurate measurements of the object's height and weight.Furthermore,a classifier is developed based on K-nearest neighbor algorithm,allowing real-time object classification.Experiments demonstrate that compared to only using soft fingers,the IOSGripper signifi-cantly reduces the squeeze force on the objects(with 0 N squeeze force during palm support)and damage on the delicate object,while improving grasping stability.Its height and weight measurement errors are within 2 mm and 1 g,respectively.And it achieves high accuracy in three test scenarios,including classifying similar objects.This study provides useful insights for the design of soft grippers capable of human-like grasping and sorting tasks.
文摘The word“spatial”fundamentally relates to human existence,evolution,and activity in terrestrial and even celestial spaces.After reviewing the spatial features of many areas,the paper describes basics of high level model and technology called Spatial Grasp for dealing with large distributed systems,which can provide spatial vision,awareness,management,control,and even consciousness.The technology description includes its key Spatial Grasp Language(SGL),self-evolution of recursive SGL scenarios,and implementation of SGL interpreter converting distributed networked systems into powerful spatial engines.Examples of typical spatial scenarios in SGL include finding shortest path tree and shortest path between network nodes,collecting proper information throughout the whole world,elimination of multiple targets by intelligent teams of chasers,and withstanding cyber attacks in distributed networked systems.Also this paper compares Spatial Grasp model with traditional algorithms,confirming universality of the former for any spatial systems,while the latter just tools for concrete applications.
文摘“Multidimensional international world”refers to understanding the world through multiple dimensions beyond traditional economic or political measures,fostering cross-cultural collaboration,and creating systems that balance global integration with local needs.This also includes management of global business operations across diverse cultures in a multipolar international landscape.The paper briefs the developed and already tested in numerous applications high-level Spatial Grasp Model and Technology(SGT),which can help investigate and manage complex systems with a holistic spatial approach effectively covering various physical and virtual dimensions,their interrelations,and integration as a whole.Different areas will be investigated with examples of practical solutions in them and their combinations in a high-level Spatial Grasp Language(SGL),the key element of SGT.This allows for the creation and distributed management of very large spatial networks with different orientation which can be self-spreading,self-analyzing,self-modifying,and self-recovering in complex terrestrial and celestial environments,and also organize dynamic multi-networking solutions supporting global evolution and integrity.
基金supported by the Jiangxi Provincial Natural Science Foundation(No.20232BAB202027)the National Natural Science Foundation of China(No.62367006)the Natural Science Foundation of Hubei Province of China(No.2022CFB536).
文摘With the rapid development of robotics,grasp prediction has become fundamental to achieving intelligent physical interactions.To enhance grasp detection accuracy in unstructured environments,we propose a novel Cross-Multiscale Adaptive Collaborative and Fusion Grasp Detection Network(CMACF-Net).Addressing the limitations of conventional methods in capturing multi-scale spatial features,CMACF-Net introduces the Quantized Multi-scale Global Attention Module(QMGAM),which enables precise multi-scale spatial calibration and adaptive spatial-channel interaction,ultimately yielding a more robust and discriminative feature representation.To reduce the degradation of local features and the loss of high-frequency information,the Cross-scale Context Integration Module(CCI)is employed to facilitate the effective fusion and alignment of global context and local details.Furthermore,an Efficient Up-Convolution Block(EUCB)is integrated into a U-Net architecture to effectively restore spatial details lost during the downsampling process,while simultaneously preserving computational efficiency.Extensive evaluations demonstrate that CMACF-Net achieves state-of-the-art detection accuracies of 98.9% and 95.9% on the Cornell and Jacquard datasets,respectively.Additionally,real-time grasping experiments on the RM65-B robotic platform validate the framework’s robustness and generalization capability,underscoring its applicability to real-world robotic manipulation scenarios.
基金Supported by National Natural Science Foundation of China(Grant No.52475280)Shaanxi Provincial Natural Science Basic Research Program(Grant No.2025SYSSYSZD-105).
文摘Robots are key to expanding the scope of space applications.The end-to-end training for robot vision-based detection and precision operations is challenging owing to constraints such as extreme environments and high computational overhead.This study proposes a lightweight integrated framework for grasp detection and imitation learning,named GD-IL;it comprises a grasp detection algorithm based on manipulability and Gaussian mixture model(manipulability-GMM),and a grasp trajectory generation algorithm based on a two-stage robot imitation learning algorithm(TS-RIL).In the manipulability-GMM algorithm,we apply GMM clustering and ellipse regression to the object point cloud,propose two judgment criteria to generate multiple candidate grasp bounding boxes for the robot,and use manipulability as a metric for selecting the optimal grasp bounding box.The stages of the TS-RIL algorithm are grasp trajectory learning and robot pose optimization.In the first stage,the robot grasp trajectory is characterized using a second-order dynamic movement primitive model and Gaussian mixture regression(GMM).By adjusting the function form of the forcing term,the robot closely approximates the target-grasping trajectory.In the second stage,a robot pose optimization model is built based on the derived pose error formula and manipulability metric.This model allows the robot to adjust its configuration in real time while grasping,thereby effectively avoiding singularities.Finally,an algorithm verification platform is developed based on a Robot Operating System and a series of comparative experiments are conducted in real-world scenarios.The experimental results demonstrate that GD-IL significantly improves the effectiveness and robustness of grasp detection and trajectory imitation learning,outperforming existing state-of-the-art methods in execution efficiency,manipulability,and success rate.
基金Program for Innovative Research Team in University(IRT520)CAST of China (20090703)
文摘When the space robot captures a floating target, contact impact occurs inevitably and frequently between the manipulator hand and the target, which seriously impacts the position and attitude of the robot and grasping security. ＂Dynamic grasping area＂ is introduced to describe the collision process of manipulator grasping target, and grasping area control equation is established. By analyzing the impact of grasping control parameters, base and target mass on the grasping process and combining the life experience, it is found that if the product of speed control parameter and dB adjustment parameter is close to but smaller than the minimum grasping speed, collision impact in the grasping process could be reduced greatly, and then an ideal grasping strategy is proposed. Simulation results indicate that during the same period, the strategy grasping is superior to the accelerating grasping, in that the amplitude of impact force is reduced to 20%, and the attitude control torque is reduced to 15%, and the impact on the robot is eliminated significantly. The results would have important academic value and engineering significance.
文摘In recent years,robotic arm grasping has become a pivotal task in the field of robotics,with applications spanning from industrial automation to healthcare.The optimization of grasping strategies plays a crucial role in enhancing the effectiveness,efficiency,and reliability of robotic systems.This paper presents a novel approach to optimizing robotic arm grasping strategies based on deep reinforcement learning(DRL).Through the utilization of advanced DRL algorithms,such as Q-Learning,Deep Q-Networks(DQN),Policy Gradient Methods,and Proximal Policy Optimization(PPO),the study aims to improve the performance of robotic arms in grasping objects with varying shapes,sizes,and environmental conditions.The paper provides a detailed analysis of the various deep reinforcement learning methods used for grasping strategy optimization,emphasizing the strengths and weaknesses of each algorithm.It also presents a comprehensive framework for training the DRL models,including simulation environment setup,the optimization process,and the evaluation metrics for grasping success.The results demonstrate that the proposed approach significantly enhances the accuracy and stability of the robotic arm in performing grasping tasks.The study further explores the challenges in training deep reinforcement learning models for real-time robotic applications and offers solutions for improving the efficiency and reliability of grasping strategies.
基金supported by the National Key R&D Program of China(NO.2024YFB3409900)the China Postdoctoral Science Foundation(NO.2023M730845)the Heilongjiang Postdoctoral Fund(NO.LBH-Z23182)。
文摘Currently,numerous biomimetic robots inspired by natural biological systems have been developed.However,creating soft robots with versatile locomotion modes remains a significant challenge.Snakes,as invertebrate reptiles,exhibit diverse and powerful locomotion abilities,including prey constriction,sidewinding,accordion locomotion,and winding climbing,making them a focus of robotics research.In this study,we present a snake-inspired soft robot with an initial coiling structure,fabricated using MXene-cellulose nanofiber ink printed on pre-expanded polyethylene film through direct ink writing technology.The controllable fabrication of initial coiling structure soft robot(ICSBot)has been achieved through theoretical calculations and finite element analysis to predict and analyze the initial structure of ICSBot,and programmable ICSBot has been designed and fabricated.This robot functions as a coiling gripper capable of grasping objects with complex shapes under near infrared light stimulation.Additionally,it demonstrates multi-modal crawling locomotion in various environments,including confined spaces,unstructured terrains,and both inside and outside tubes.These results offer a novel strategy for designing and fabricating coiling-structured soft robots and highlight their potential applications in smart and multifunctional robotics.
基金Supported by the National Natural Science Foundation of China(Grant No.52405530)the Beijing Natural Science Foundation(Grant No.L243009)the Beijing Institute of Technology Research Fund Program for Young Scholars。
文摘Recent years have witnessed unprecedented development in humanoid robotics,with dexterous hand grasping emerging as a focal research area across industrial and academic sectors.To track the state-of-the-art dexterous hand grasp,a review of dexterous hand grasp based on bibliometric analysis is executed.The related studies on dexterous hand grasp are collected from the Web of Science for analysis,where the publication details and cooperation situations from the perspectives of country,institute,etc.are discussed.The keywords cluster is adopted to find the hot research topic of dexterous hand grasp.The development trend of dexterous hand grasp is explored based on the top 25 keywords with the strongest citation bursts.The review findings indicate that precision control via multimodal fusion,autonomous task understanding and intelligent decision,and in-hand dexterous manipulation are top three hotspots in future.
基金Supported by National Key R&D Program of China(Grant No.2024YFB4709800)Fundamental Research Funds for the Central Universities。
文摘The“visual perception+hand-eye transformation+motion planning”paradigm of robotic coordination grasping has demonstrated feasibility in unstructured scenes such as logistics.However,further developments in handling complex and dynamic environments remain challenging.To address the issue of unknown targets requiring immediate deployment for grasping tasks,this paper proposes a novel hand-eye coordinated method for progressive grasping guided by the texture keypoints of the target.First,we develop an efficient system for acquiring texture-matching templates and an estimation algorithm for the feature region that filters the precisely registered texture feature points of the target.Then,we integrate optical flow estimation to update and track the robust texture region in real time,and design a feature-based servo grasping controller to map the optical flow points of the high-registration texture region to the robot joint velocities for precise tracking.Finally,we impose spatiotemporal constraints on the planned trajectory and decouple the target motion,to achieve progressive approach and rotationally invariant grasping for both dynamic and static targets.Comprehensive experiments demonstrate that this tracking grasping method exhibits a low latency,a high precision,and robustness in complex scenarios and dynamic disturbances,with an average position accuracy of approximately 5 mm,a rotation accuracy of approximately 0.02,and an overall grasping success rate of approximately 90%.
基金the National Natural Science Foundation of China(No.62173230)the Program of Science and Technology Commission of Shanghai Municipality(No.22511101400)。
文摘Robot grasp detection is a fundamental vision task for robots.Deep learning-based methods have shown excellent results in enhancing the grasp detection capabilities for model-free objects in unstructured scenes.Most popular approaches explore deep network models and exploit RGB-D images combining colour and depth data to acquire enriched feature expressions.However,current work struggles to achieve a satisfactory balance between the accuracy and real-time performance;the variability of RGB and depth feature distributions receives inadequate attention.The treatment of predicted failure cases is also lacking.We propose an efficient fully convolutional network to predict the pixel-level antipodal grasp parameters in RGB-D images.A structure with hierarchical feature fusion is established using multiple lightweight feature extraction blocks.The feature fusion module with 3D global attention is used to select the complementary information in RGB and depth images suficiently.Additionally,a grasp configuration optimization method based on local grasp path is proposed to cope with the possible failures predicted by the model.Extensive experiments on two public grasping datasets,Cornell and Jacquard,demonstrate that the approach can improve the performance of grasping unknown objects.
文摘Grasping is one of the most fundamental operations in modern robotics applications.While deep rein-forcement learning(DRL)has demonstrated strong potential in robotics,there is too much emphasis on maximizing the cumulative reward in executing tasks,and the potential safety risks are often ignored.In this paper,an optimization method based on safe reinforcement learning(Safe RL)is proposed to address the robotic grasping problem under safety constraints.Specifically,considering the obstacle avoidance constraints of the system,the grasping problem of the manipulator is modeled as a Constrained Markov Decision Process(CMDP).The Lagrange multiplier and a dynamic weighted mechanism are introduced into the Proximal Policy Optimization(PPO)framework,leading to the development of the dynamic weighted Lagrange PPO(DWL-PPO)algorithm.The behavior of violating safety constraints is punished while the policy is optimized in this proposed method.In addition,the orientation control of the end-effector is included in the reward function,and a compound reward function adapted to changes in pose is designed.Ultimately,the efficacy and advantages of the suggested method are proved by extensive training and testing in the Pybullet simulator.The results of grasping experiments reveal that the recommended approach provides superior safety and efficiency compared with other advanced RL methods and achieves a good trade-off between model learning and risk aversion.
