Self-localization is a fundamental requirement for the mobile robot. Robot usually contains a large number of dif- ferent sensors, which provide the information of robot localization, and all the sensor information sh...Self-localization is a fundamental requirement for the mobile robot. Robot usually contains a large number of dif- ferent sensors, which provide the information of robot localization, and all the sensor information should be considered for the optimal location. Kalman filter is efficient to realize the information fusion. Used as an efficient sensor fusion algorithm, Kalman filter is an advanced filtering technique which can reduce errors of the position and orientation of the sensors. Kalman filter has been paied much attention to robot automation and solutions to solve uncertainties such as robot localization, navigation, following, tracking, motion control, estimation and prediction. The paper briefly describes Kalman filter theory, and establishes a simple mathematical model based on muti-sensor mobile robot. Meanwhile, Kalman filter is used in robot self-localization by simulations, and it is demonstrated by simulations that Kalman filter is effective.展开更多
A practical serf-localization scheme for mobile robots is proposed and implemented by utilizing sonar sensors. Specifically, the localization problem is solved by employing Monte Carlo method with a new mechanism prop...A practical serf-localization scheme for mobile robots is proposed and implemented by utilizing sonar sensors. Specifically, the localization problem is solved by employing Monte Carlo method with a new mechanism proposed to calculate the samples＇ weights; the convergence and veracity of the sample set are guaranteed by the designed resampling and scattering process. The proposed serf-localization algorithm is fully implemented on a specific mobile robot system, and experimental results illustrate that it provides an efficient solution for the kidnapped problem.展开更多
Self-localization is one of the most important aspects for using mobile robots in unstructured environments. In this paper, the authors introduce a new approach for a self-localization and navigation unit for mobile p...Self-localization is one of the most important aspects for using mobile robots in unstructured environments. In this paper, the authors introduce a new approach for a self-localization and navigation unit for mobile platforms in extraterrestrial environments, based on the authors＂ successful results in self-localization of forestry machines on earth. The presented approach is developed from a highly modular concept, which allows a simple but efficient adaption to specific applications by just substituting some scenario dependent components. In this paper, the authors will explain the general concept and the terrestrial implementation so far. On this basis, the authors will demonstrate and discuss the necessary adaptions to the general concept in order to handle the different conditions on extraterrestrial surfaces.展开更多
Single-cell biomechanics and electrophysiology measuring tools have transformed biological research over the last few decades,which enabling a comprehensive and nuanced understanding of cellular behavior and function....Single-cell biomechanics and electrophysiology measuring tools have transformed biological research over the last few decades,which enabling a comprehensive and nuanced understanding of cellular behavior and function.Despite their high-quality information content,these single-cell measuring techniques suffer from laborious manual processing by highly skilled workers and extremely low throughput(tens of cells per day).Recently,numerous researchers have automated the measurement of cell mechanical and electrical signals through robotic localization and control processes.While these efforts have demonstrated promising progress,critical challenges persist,including human dependency,learning complexity,in-situ measurement,and multidimensional signal acquisition.To identify key limitations and highlight emerging opportunities for innovation,in this review,we comprehensively summarize the key steps of robotic technologies in single-cell biomechanics and electrophysiology.We also discussed the prospects and challenges of robotics and automation in biological research.By bridging gaps between engineering,biology,and data science,this work aims to stimulate interdisciplinary research and accelerate the translation of robotic single-cell technologies into practical applications in the life sciences and medical fields.展开更多
Underwater pipeline inspection plays a vital role in the proactive maintenance and management of critical marine infrastructure and subaquatic systems.However,the inspection of underwater pipelines presents a challeng...Underwater pipeline inspection plays a vital role in the proactive maintenance and management of critical marine infrastructure and subaquatic systems.However,the inspection of underwater pipelines presents a challenge due to factors such as light scattering,absorption,restricted visibility,and ambient noise.The advancement of deep learning has introduced powerful techniques for processing large amounts of unstructured and imperfect data collected from underwater environments.This study evaluated the efficacy of the You Only Look Once(YOLO)algorithm,a real-time object detection and localization model based on convolutional neural networks,in identifying and classifying various types of pipeline defects in underwater settings.YOLOv8,the latest evolution in the YOLO family,integrates advanced capabilities,such as anchor-free detection,a cross-stage partial network backbone for efficient feature extraction,and a feature pyramid network+path aggregation network neck for robust multi-scale object detection,which make it particularly well-suited for complex underwater environments.Due to the lack of suitable open-access datasets for underwater pipeline defects,a custom dataset was captured using a remotely operated vehicle in a controlled environment.This application has the following assets available for use.Extensive experimentation demonstrated that YOLOv8 X-Large consistently outperformed other models in terms of pipe defect detection and classification and achieved a strong balance between precision and recall in identifying pipeline cracks,rust,corners,defective welds,flanges,tapes,and holes.This research establishes the baseline performance of YOLOv8 for underwater defect detection and showcases its potential to enhance the reliability and efficiency of pipeline inspection tasks in challenging underwater environments.展开更多
A unilateral self-locking mechanism(USM) was proposed to increase the tractive ability of the inchworm in-pipe robots for pipeline inspection.The USM was basically composed of a cam,a torsional spring and an axis.The ...A unilateral self-locking mechanism(USM) was proposed to increase the tractive ability of the inchworm in-pipe robots for pipeline inspection.The USM was basically composed of a cam,a torsional spring and an axis.The self-locking and virtual work principles were applied to studying the basic self-locking condition of the USM.In order to make the cooperation between the crutch and telescopic mechanism more harmonical,the unlocking time of the USM was calculated.A set of parameters were selected to build a virtual model and fabricate a prototype.Both the simulation and performance experiments were carried out in a pipe with a nominal inside diameter of 160 mm.The results show that USM enables the robot to move quickly in one way,and in the other way it helps the robot get self-locking with the pipe wall.The traction of the inchworm robot can rise to 1.2 kN,beyond the limitation of friction of 0.497 kN.展开更多
The follow-up application of underwater wireless sensor network is influenced by accuracy of self-localization of nodes. The self-localization of nodes is discussed in this paper. First of all, nodes of underwater wir...The follow-up application of underwater wireless sensor network is influenced by accuracy of self-localization of nodes. The self-localization of nodes is discussed in this paper. First of all, nodes of underwater wireless sensor network are classified into several levels according to the accuracy of position of nodes and the levels are from the first to the fifth in accordance with accuracy of nodes from high to low respectively. Secondly, the level of anchor nodes can be known by those unknown nodes from the information given by the anchor nodes themselves, At the same time the unknown nodes are able to be located in the area controlled by the first level of anchor nodes that are as the aggregation. Then the positioning algorithm is designed correspondingly in accordance with the accuracy level of nodes. Finally, the positioning algorithm is simulated and analyzed. The result shows that the unknown nodes can be located effectively by hierarchical control.展开更多
When a group of mobile agents track a target,they can locate themselves and the target in a cooperative manner.To maximize the group advantage,a parallel integration strategy of cooperative target-localization(CTL)and...When a group of mobile agents track a target,they can locate themselves and the target in a cooperative manner.To maximize the group advantage,a parallel integration strategy of cooperative target-localization(CTL)and cooperative self-localization(CSL)is designed.Firstly,a global cost function containing the agents’positions and the target’s position is established.Secondly,along with the agents’positions being re-estimated during CTL,the Utransform is employed to propagate the error covariance of the position estimations among the agents.The simulation results show that,the proposal exploits more information for locating the target and the agents than the cases where CTL and CSL run separately,and the global optimal position estimations of the agents and the target are obtained.展开更多
This paper develops a new algorithm for sensor network self-localization, which is an enhanced version of the existing Crocco’s method in [11]. The algorithm explores the noisy time of flight (TOF) measurements that ...This paper develops a new algorithm for sensor network self-localization, which is an enhanced version of the existing Crocco’s method in [11]. The algorithm explores the noisy time of flight (TOF) measurements that quantify the distances between sensor nodes to be localized and sources also at unknown positions. The newly proposed technique first obtains rough estimates of the sensor node and source positions, and then it refines the estimates via a least squares estimator (LSE). The LSE takes into account the geometrical constraints introduced by the desired global coordinate system to improve performance. Simulations show that the new technique offers superior localization accuracy over the original Crocco’s algorithm under small measurement noise condition.展开更多
Large language models(LLMs)have undergone significant expansion and have been increasingly integrated across various domains.Notably,in the realm of robot task planning,LLMs harness their advanced reasoning and langua...Large language models(LLMs)have undergone significant expansion and have been increasingly integrated across various domains.Notably,in the realm of robot task planning,LLMs harness their advanced reasoning and language comprehension capabilities to formulate precise and efficient action plans based on natural language instructions.However,for embodied tasks,where robots interact with complex environments,textonly LLMs often face challenges due to a lack of compatibility with robotic visual perception.This study provides a comprehensive overview of the emerging integration of LLMs and multimodal LLMs into various robotic tasks.Additionally,we propose a framework that utilizes multimodal GPT-4V to enhance embodied task planning through the combination of natural language instructions and robot visual perceptions.Our results,based on diverse datasets,indicate that GPT-4V effectively enhances robot performance in embodied tasks.This extensive survey and evaluation of LLMs and multimodal LLMs across a variety of robotic tasks enriches the understanding of LLM-centric embodied intelligence and provides forward-looking insights towards bridging the gap in Human-Robot-Environment interaction.展开更多
OBJECTIVE:To propose an automatic acupuncture robot system for performing acupuncture operations.METHODS:The acupuncture robot system consists of three components:automatic acupoint localization,acupuncture manipulati...OBJECTIVE:To propose an automatic acupuncture robot system for performing acupuncture operations.METHODS:The acupuncture robot system consists of three components:automatic acupoint localization,acupuncture manipulations,and De Qi sensation detection.The OptiTrack motion capture system is used to locate acupoints,which are then translated into coordinates in the robot control system.A flexible collaborative robot with an intelligent gripper is then used to perform acupuncture manipulations with high precision.In addition,a De Qi sensation detection system is proposed to evaluate the effect of acupuncture.To verify the stability of the designed acupuncture robot,acupoints'coordinates localized by the acupuncture robot are compared with the Gold Standard labeled by a professional acupuncturist using significant level tests.RESULTS:Through repeated experiments for eight acupoints,the acupuncture robot achieved a positioning error within 3.3 mm,which is within the allowable range of needle extraction and acupoint insertion.During needle insertion,the robot arm followed the prescribed trajectory with a mean deviation distance of 0.02 mm and a deviation angle of less than 0.15°.The results of the lifting thrusting operation in the Xingzhen process show that the mean acupuncture depth error of the designed acupuncture robot is approximately 2 mm,which is within the recommended depth range for the Xingzhen operation.In addition,the average detection accuracy of the De Qi keywords is 94.52%,which meets the requirements of acupuncture effect testing for different dialects.CONCLUSION:The proposed acupuncture robot system streamlines the acupuncture process,increases efficiency,and reduces practitioner fatigue,while also allowing for the quantification of acupuncture manipulations and evaluation of therapeutic effects.The development of an acupuncture robot system has the potential to revolutionize low back pain treatment and improve patient outcomes.展开更多
Objective:This study aimed to assess the feasibility and safety of the SHURUI single-port robotic surgical system for a range of major urological surgeries.Methods:In this prospective,multicenter clinical trial,we exa...Objective:This study aimed to assess the feasibility and safety of the SHURUI single-port robotic surgical system for a range of major urological surgeries.Methods:In this prospective,multicenter clinical trial,we examined the effectiveness of the SHURUI single-port robotic surgical system in urological interventions.The first 50 patients from four centers in China underwent single-port surgeries including partial nephrectomy,radical prostatectomy,partial adrenalectomy,and pyeloureteroplasty,exclusively by the SHURUI single-port robotic surgical system.The study's primary endpoints focused on the success of surgeries,defined as no deviations from planned procedures,no need for more than one port,and no re-operations within 24 h after surgery.Secondary endpoints encompassed a range of surgical metrics,functional outcomes,and patient demographic data.Clinical assessments were conducted before surgery,before discharge,and 1 month after discharge.Results:The surgical procedures were executed successfully without requiring intraoperative conversions or transfusions.Both estimated blood loss and operation durations were maintained within satisfactory limits.For each type of surgery,the mean console times and estimated blood loss were 179.8(standard deviation[SD]39.4)min and 125.6(SD 126.0)mL for radical prostatectomy,126.7(SD 47.8)min and 39.2(SD 54.4)mL for partial nephrectomy,112.6(SD 37.4)min and 20.0(SD 13.2)mL for partial adrenalectomy,and 148.0(SD 18.2)min and 18.0(SD 17.9)mL for pyeloureteroplasty,respectively.Across the cohort,17 patients experienced a total of 25 adverse events,while 10 postoperative complications,all rated as Clavien-Dindo grade I,were encountered by eight patients.All patients had shown recovery or improvement from these events before the end of this trial.Conclusion:The SHURUI single-port robotic surgical system demonstrated feasibility and safety in the performance of major urological surgeries.These initial findings highlight the system's potential,though further research and longer follow-up are required to assess long-term outcomes.展开更多
The advent of parametric design has resulted in a marked increase in the complexity of building.Unfortunately,traditional construction methods make it difficult to meet the needs.Therefore,construction robots have bec...The advent of parametric design has resulted in a marked increase in the complexity of building.Unfortunately,traditional construction methods make it difficult to meet the needs.Therefore,construction robots have become a pivotal production tool in this context.Since the arm span of a single robot usually does not exceed 3 meters,it is not competent for producing large-scale building components.Accordingly,the extension of the robot,s working range is often achieved by external axes.Nevertheless,the coupling control of external axes and robots and their kinematic solution have become key challenges.The primary technical difficulties include customized construction robots,automatic solutions for external axes,fixed axis joints,and specific motion mode control.This paper proposes solutions to these difficulties,introduces the relevant basic concepts and algorithms in detail,and encapsulates these robotics principles and algorithm processes into the Grasshopper plug-in commonly used by architects to form the FURobot software platform.This platform effectively solves the above problems,lowers the threshold for architects,and improves production efficiency.The effectiveness of the algorithm and software in this paper is verified through simulation experiments.展开更多
This paper endeavours to bridge the existing gap in muscular actuator design for ligament-skeletal-inspired robots,thereby fostering the evolution of these robotic systems.We introduce two novel compliant actuators,na...This paper endeavours to bridge the existing gap in muscular actuator design for ligament-skeletal-inspired robots,thereby fostering the evolution of these robotic systems.We introduce two novel compliant actuators,namely the Internal Torsion Spring Compliant Actuator(ICA)and the External Spring Compliant Actuator(ECA),and present a comparative analysis against the previously conceived Magnet Integrated Soft Actuator(MISA)through computational and experimental results.These actuators,employing a motor-tendon system,emulate biological muscle-like forms,enhancing artificial muscle technology.Then,applications of the proposed actuators in a robotic arm inspired by the human musculoskeletal system are presented.Experiments demonstrate satisfactory power in tasks like lifting dumbbells(peak power:36 W),playing table tennis(end-effector speed:3.2 m/s),and door opening,without compromising biomimetic aesthetics.Compared to other linear stiffness serial elastic actuators(SEAs),ECA and ICA exhibit high power-to-volume(361×10^(3)W/m^(3))and power-to-mass(111.6 W/kg)ratios respectively,endorsing the biomimetic design’s promise in robotic development.展开更多
Fault detection in industrial robot drive systems is a critical aspect of ensuring operational reliability and efficiency.To address the challenge of balancing accuracy and robustness in existing fault detection metho...Fault detection in industrial robot drive systems is a critical aspect of ensuring operational reliability and efficiency.To address the challenge of balancing accuracy and robustness in existing fault detection methods,this paper proposes an enhanced fault detection method based on the unscented Kalman filter(UKF).A comprehensive mathematical model of the brushless DC motor drive system is developed to provide a theoretical foundation for the design of subsequent fault detection methods.The conventional UKF estimation process is detailed,and its limitations in balancing estimation accuracy and robustness are addressed by introducing a dynamic,time-varying boundary layer.To further enhance detection performance,the method incorporates residual analysis using improved z-score and signal-tonoise ratio(SNR)metrics.Numerical simulations under both fault-free and faulty conditions demonstrate that the proposed approach achieves lower root mean square error(RMSE)in fault-free scenarios and provides reliable fault detection.These results highlight the potential of the proposed method to enhance the reliability and robustness of fault detection in industrial robot drive systems.展开更多
To improve the efficiency and accuracy of path planning for fan inspection tasks in thermal power plants,this paper proposes an intelligent inspection robot path planning scheme based on an improved A^(*)algorithm.The...To improve the efficiency and accuracy of path planning for fan inspection tasks in thermal power plants,this paper proposes an intelligent inspection robot path planning scheme based on an improved A^(*)algorithm.The inspection robot utilizes multiple sensors to monitor key parameters of the fans,such as vibration,noise,and bearing temperature,and upload the data to the monitoring center.The robot’s inspection path employs the improved A^(*)algorithm,incorporating obstacle penalty terms,path reconstruction,and smoothing optimization techniques,thereby achieving optimal path planning for the inspection robot in complex environments.Simulation results demonstrate that the improved A^(*)algorithm significantly outperforms the traditional A^(*)algorithm in terms of total path distance,smoothness,and detour rate,effectively improving the execution efficiency of inspection tasks.展开更多
基金Research Fund for the Doctoral Program of Higher Education of China(No.20123718120007)
文摘Self-localization is a fundamental requirement for the mobile robot. Robot usually contains a large number of dif- ferent sensors, which provide the information of robot localization, and all the sensor information should be considered for the optimal location. Kalman filter is efficient to realize the information fusion. Used as an efficient sensor fusion algorithm, Kalman filter is an advanced filtering technique which can reduce errors of the position and orientation of the sensors. Kalman filter has been paied much attention to robot automation and solutions to solve uncertainties such as robot localization, navigation, following, tracking, motion control, estimation and prediction. The paper briefly describes Kalman filter theory, and establishes a simple mathematical model based on muti-sensor mobile robot. Meanwhile, Kalman filter is used in robot self-localization by simulations, and it is demonstrated by simulations that Kalman filter is effective.
基金Supported by the National Natural Science Foundation of China (No. 60875055)Natural Science Foundation of Tianjin (No. 07JCY-BJC05400)Program for New Century Excellent Talents in University (No. NCET-06-0210)
文摘A practical serf-localization scheme for mobile robots is proposed and implemented by utilizing sonar sensors. Specifically, the localization problem is solved by employing Monte Carlo method with a new mechanism proposed to calculate the samples＇ weights; the convergence and veracity of the sample set are guaranteed by the designed resampling and scattering process. The proposed serf-localization algorithm is fully implemented on a specific mobile robot system, and experimental results illustrate that it provides an efficient solution for the kidnapped problem.
文摘Self-localization is one of the most important aspects for using mobile robots in unstructured environments. In this paper, the authors introduce a new approach for a self-localization and navigation unit for mobile platforms in extraterrestrial environments, based on the authors＂ successful results in self-localization of forestry machines on earth. The presented approach is developed from a highly modular concept, which allows a simple but efficient adaption to specific applications by just substituting some scenario dependent components. In this paper, the authors will explain the general concept and the terrestrial implementation so far. On this basis, the authors will demonstrate and discuss the necessary adaptions to the general concept in order to handle the different conditions on extraterrestrial surfaces.
基金the National Natural Science Foundation of China[62525301,62127811,62433019]the New Cornerstone Science Foundation through the XPLORER PRIZEthe financial support by the China Postdoctoral Science Foundation[GZB20240797].
文摘Single-cell biomechanics and electrophysiology measuring tools have transformed biological research over the last few decades,which enabling a comprehensive and nuanced understanding of cellular behavior and function.Despite their high-quality information content,these single-cell measuring techniques suffer from laborious manual processing by highly skilled workers and extremely low throughput(tens of cells per day).Recently,numerous researchers have automated the measurement of cell mechanical and electrical signals through robotic localization and control processes.While these efforts have demonstrated promising progress,critical challenges persist,including human dependency,learning complexity,in-situ measurement,and multidimensional signal acquisition.To identify key limitations and highlight emerging opportunities for innovation,in this review,we comprehensively summarize the key steps of robotic technologies in single-cell biomechanics and electrophysiology.We also discussed the prospects and challenges of robotics and automation in biological research.By bridging gaps between engineering,biology,and data science,this work aims to stimulate interdisciplinary research and accelerate the translation of robotic single-cell technologies into practical applications in the life sciences and medical fields.
文摘Underwater pipeline inspection plays a vital role in the proactive maintenance and management of critical marine infrastructure and subaquatic systems.However,the inspection of underwater pipelines presents a challenge due to factors such as light scattering,absorption,restricted visibility,and ambient noise.The advancement of deep learning has introduced powerful techniques for processing large amounts of unstructured and imperfect data collected from underwater environments.This study evaluated the efficacy of the You Only Look Once(YOLO)algorithm,a real-time object detection and localization model based on convolutional neural networks,in identifying and classifying various types of pipeline defects in underwater settings.YOLOv8,the latest evolution in the YOLO family,integrates advanced capabilities,such as anchor-free detection,a cross-stage partial network backbone for efficient feature extraction,and a feature pyramid network+path aggregation network neck for robust multi-scale object detection,which make it particularly well-suited for complex underwater environments.Due to the lack of suitable open-access datasets for underwater pipeline defects,a custom dataset was captured using a remotely operated vehicle in a controlled environment.This application has the following assets available for use.Extensive experimentation demonstrated that YOLOv8 X-Large consistently outperformed other models in terms of pipe defect detection and classification and achieved a strong balance between precision and recall in identifying pipeline cracks,rust,corners,defective welds,flanges,tapes,and holes.This research establishes the baseline performance of YOLOv8 for underwater defect detection and showcases its potential to enhance the reliability and efficiency of pipeline inspection tasks in challenging underwater environments.
基金Project(2007AA04Z256) supported by the National High-Tech Research and Development Program of China
文摘A unilateral self-locking mechanism(USM) was proposed to increase the tractive ability of the inchworm in-pipe robots for pipeline inspection.The USM was basically composed of a cam,a torsional spring and an axis.The self-locking and virtual work principles were applied to studying the basic self-locking condition of the USM.In order to make the cooperation between the crutch and telescopic mechanism more harmonical,the unlocking time of the USM was calculated.A set of parameters were selected to build a virtual model and fabricate a prototype.Both the simulation and performance experiments were carried out in a pipe with a nominal inside diameter of 160 mm.The results show that USM enables the robot to move quickly in one way,and in the other way it helps the robot get self-locking with the pipe wall.The traction of the inchworm robot can rise to 1.2 kN,beyond the limitation of friction of 0.497 kN.
基金Funded by Department of Education of Zhejiang Province (No.Y201119307)
文摘The follow-up application of underwater wireless sensor network is influenced by accuracy of self-localization of nodes. The self-localization of nodes is discussed in this paper. First of all, nodes of underwater wireless sensor network are classified into several levels according to the accuracy of position of nodes and the levels are from the first to the fifth in accordance with accuracy of nodes from high to low respectively. Secondly, the level of anchor nodes can be known by those unknown nodes from the information given by the anchor nodes themselves, At the same time the unknown nodes are able to be located in the area controlled by the first level of anchor nodes that are as the aggregation. Then the positioning algorithm is designed correspondingly in accordance with the accuracy level of nodes. Finally, the positioning algorithm is simulated and analyzed. The result shows that the unknown nodes can be located effectively by hierarchical control.
文摘When a group of mobile agents track a target,they can locate themselves and the target in a cooperative manner.To maximize the group advantage,a parallel integration strategy of cooperative target-localization(CTL)and cooperative self-localization(CSL)is designed.Firstly,a global cost function containing the agents’positions and the target’s position is established.Secondly,along with the agents’positions being re-estimated during CTL,the Utransform is employed to propagate the error covariance of the position estimations among the agents.The simulation results show that,the proposal exploits more information for locating the target and the agents than the cases where CTL and CSL run separately,and the global optimal position estimations of the agents and the target are obtained.
文摘This paper develops a new algorithm for sensor network self-localization, which is an enhanced version of the existing Crocco’s method in [11]. The algorithm explores the noisy time of flight (TOF) measurements that quantify the distances between sensor nodes to be localized and sources also at unknown positions. The newly proposed technique first obtains rough estimates of the sensor node and source positions, and then it refines the estimates via a least squares estimator (LSE). The LSE takes into account the geometrical constraints introduced by the desired global coordinate system to improve performance. Simulations show that the new technique offers superior localization accuracy over the original Crocco’s algorithm under small measurement noise condition.
基金supported by National Natural Science Foundation of China(62376219 and 62006194)Foundational Research Project in Specialized Discipline(Grant No.G2024WD0146)Faculty Construction Project(Grant No.24GH0201148).
文摘Large language models(LLMs)have undergone significant expansion and have been increasingly integrated across various domains.Notably,in the realm of robot task planning,LLMs harness their advanced reasoning and language comprehension capabilities to formulate precise and efficient action plans based on natural language instructions.However,for embodied tasks,where robots interact with complex environments,textonly LLMs often face challenges due to a lack of compatibility with robotic visual perception.This study provides a comprehensive overview of the emerging integration of LLMs and multimodal LLMs into various robotic tasks.Additionally,we propose a framework that utilizes multimodal GPT-4V to enhance embodied task planning through the combination of natural language instructions and robot visual perceptions.Our results,based on diverse datasets,indicate that GPT-4V effectively enhances robot performance in embodied tasks.This extensive survey and evaluation of LLMs and multimodal LLMs across a variety of robotic tasks enriches the understanding of LLM-centric embodied intelligence and provides forward-looking insights towards bridging the gap in Human-Robot-Environment interaction.
基金Modernization of Traditional Chinese Medicine Project of National Key R&D Program of China:The construction of the theoretical system of Traditional Chinese Medicine nonpharmacological therapy based on body surface stimulation(2023YFC3502704)Sichuan Provincial Science and Technology Program Project:Research and Development of Chinese Medicine Intelligent Tongue Diagnosis Equipment for Digestive System Chinese Medicine Advantageous Diseases(2023YFS0327)+2 种基金Research and Development of Chinese Medicine Intelligent Detection System for Intestinal Functions(2024YFFK0044)Research and Application of Chinese Medicine Diagnosis and Treatment Program for Herpes Zoster Treated by Shu Pai Fire Acupuncture(2024YFFK0089)Major Research and Development Project of The China Academy of Chinese Medical Sciences Innovation:Construction and application of the theoretical research mode of Traditional Chinese Medicine diagnosis and treatment of modern diseases(CI2021A00104)。
文摘OBJECTIVE:To propose an automatic acupuncture robot system for performing acupuncture operations.METHODS:The acupuncture robot system consists of three components:automatic acupoint localization,acupuncture manipulations,and De Qi sensation detection.The OptiTrack motion capture system is used to locate acupoints,which are then translated into coordinates in the robot control system.A flexible collaborative robot with an intelligent gripper is then used to perform acupuncture manipulations with high precision.In addition,a De Qi sensation detection system is proposed to evaluate the effect of acupuncture.To verify the stability of the designed acupuncture robot,acupoints'coordinates localized by the acupuncture robot are compared with the Gold Standard labeled by a professional acupuncturist using significant level tests.RESULTS:Through repeated experiments for eight acupoints,the acupuncture robot achieved a positioning error within 3.3 mm,which is within the allowable range of needle extraction and acupoint insertion.During needle insertion,the robot arm followed the prescribed trajectory with a mean deviation distance of 0.02 mm and a deviation angle of less than 0.15°.The results of the lifting thrusting operation in the Xingzhen process show that the mean acupuncture depth error of the designed acupuncture robot is approximately 2 mm,which is within the recommended depth range for the Xingzhen operation.In addition,the average detection accuracy of the De Qi keywords is 94.52%,which meets the requirements of acupuncture effect testing for different dialects.CONCLUSION:The proposed acupuncture robot system streamlines the acupuncture process,increases efficiency,and reduces practitioner fatigue,while also allowing for the quantification of acupuncture manipulations and evaluation of therapeutic effects.The development of an acupuncture robot system has the potential to revolutionize low back pain treatment and improve patient outcomes.
基金funded by the National Key Research and Development Program of China(Grant No.2022YFB4700904 to Wang L)the Shanghai Shenkang Hospital Development Center's project for the Promotion of Clinical Skills and Clinical Innovation Three-Year Action Plan(Project No.SHDC2022CRT006 to Wang L and SHDC2022CRS010B to Tang S).
文摘Objective:This study aimed to assess the feasibility and safety of the SHURUI single-port robotic surgical system for a range of major urological surgeries.Methods:In this prospective,multicenter clinical trial,we examined the effectiveness of the SHURUI single-port robotic surgical system in urological interventions.The first 50 patients from four centers in China underwent single-port surgeries including partial nephrectomy,radical prostatectomy,partial adrenalectomy,and pyeloureteroplasty,exclusively by the SHURUI single-port robotic surgical system.The study's primary endpoints focused on the success of surgeries,defined as no deviations from planned procedures,no need for more than one port,and no re-operations within 24 h after surgery.Secondary endpoints encompassed a range of surgical metrics,functional outcomes,and patient demographic data.Clinical assessments were conducted before surgery,before discharge,and 1 month after discharge.Results:The surgical procedures were executed successfully without requiring intraoperative conversions or transfusions.Both estimated blood loss and operation durations were maintained within satisfactory limits.For each type of surgery,the mean console times and estimated blood loss were 179.8(standard deviation[SD]39.4)min and 125.6(SD 126.0)mL for radical prostatectomy,126.7(SD 47.8)min and 39.2(SD 54.4)mL for partial nephrectomy,112.6(SD 37.4)min and 20.0(SD 13.2)mL for partial adrenalectomy,and 148.0(SD 18.2)min and 18.0(SD 17.9)mL for pyeloureteroplasty,respectively.Across the cohort,17 patients experienced a total of 25 adverse events,while 10 postoperative complications,all rated as Clavien-Dindo grade I,were encountered by eight patients.All patients had shown recovery or improvement from these events before the end of this trial.Conclusion:The SHURUI single-port robotic surgical system demonstrated feasibility and safety in the performance of major urological surgeries.These initial findings highlight the system's potential,though further research and longer follow-up are required to assess long-term outcomes.
基金National Key R&D Program of China(Nos.2023YFC3806900,2022YFE0141400)。
文摘The advent of parametric design has resulted in a marked increase in the complexity of building.Unfortunately,traditional construction methods make it difficult to meet the needs.Therefore,construction robots have become a pivotal production tool in this context.Since the arm span of a single robot usually does not exceed 3 meters,it is not competent for producing large-scale building components.Accordingly,the extension of the robot,s working range is often achieved by external axes.Nevertheless,the coupling control of external axes and robots and their kinematic solution have become key challenges.The primary technical difficulties include customized construction robots,automatic solutions for external axes,fixed axis joints,and specific motion mode control.This paper proposes solutions to these difficulties,introduces the relevant basic concepts and algorithms in detail,and encapsulates these robotics principles and algorithm processes into the Grasshopper plug-in commonly used by architects to form the FURobot software platform.This platform effectively solves the above problems,lowers the threshold for architects,and improves production efficiency.The effectiveness of the algorithm and software in this paper is verified through simulation experiments.
基金research project funded by the National Natural Science Foundation of China(NSFC)under Grant 91948302 and Grant 52021003Research England fund at NERIC.
文摘This paper endeavours to bridge the existing gap in muscular actuator design for ligament-skeletal-inspired robots,thereby fostering the evolution of these robotic systems.We introduce two novel compliant actuators,namely the Internal Torsion Spring Compliant Actuator(ICA)and the External Spring Compliant Actuator(ECA),and present a comparative analysis against the previously conceived Magnet Integrated Soft Actuator(MISA)through computational and experimental results.These actuators,employing a motor-tendon system,emulate biological muscle-like forms,enhancing artificial muscle technology.Then,applications of the proposed actuators in a robotic arm inspired by the human musculoskeletal system are presented.Experiments demonstrate satisfactory power in tasks like lifting dumbbells(peak power:36 W),playing table tennis(end-effector speed:3.2 m/s),and door opening,without compromising biomimetic aesthetics.Compared to other linear stiffness serial elastic actuators(SEAs),ECA and ICA exhibit high power-to-volume(361×10^(3)W/m^(3))and power-to-mass(111.6 W/kg)ratios respectively,endorsing the biomimetic design’s promise in robotic development.
基金Supported by the Natural Science Foundation of the Higher Education Institutions of Jiangsu Province(22KJB520012)the Research Project on Higher Education Reform in Jiangsu Province(2023JSJG781)the College Student Innovation and Entrepreneurship Training Program Project(202313571008Z)。
文摘Fault detection in industrial robot drive systems is a critical aspect of ensuring operational reliability and efficiency.To address the challenge of balancing accuracy and robustness in existing fault detection methods,this paper proposes an enhanced fault detection method based on the unscented Kalman filter(UKF).A comprehensive mathematical model of the brushless DC motor drive system is developed to provide a theoretical foundation for the design of subsequent fault detection methods.The conventional UKF estimation process is detailed,and its limitations in balancing estimation accuracy and robustness are addressed by introducing a dynamic,time-varying boundary layer.To further enhance detection performance,the method incorporates residual analysis using improved z-score and signal-tonoise ratio(SNR)metrics.Numerical simulations under both fault-free and faulty conditions demonstrate that the proposed approach achieves lower root mean square error(RMSE)in fault-free scenarios and provides reliable fault detection.These results highlight the potential of the proposed method to enhance the reliability and robustness of fault detection in industrial robot drive systems.
文摘To improve the efficiency and accuracy of path planning for fan inspection tasks in thermal power plants,this paper proposes an intelligent inspection robot path planning scheme based on an improved A^(*)algorithm.The inspection robot utilizes multiple sensors to monitor key parameters of the fans,such as vibration,noise,and bearing temperature,and upload the data to the monitoring center.The robot’s inspection path employs the improved A^(*)algorithm,incorporating obstacle penalty terms,path reconstruction,and smoothing optimization techniques,thereby achieving optimal path planning for the inspection robot in complex environments.Simulation results demonstrate that the improved A^(*)algorithm significantly outperforms the traditional A^(*)algorithm in terms of total path distance,smoothness,and detour rate,effectively improving the execution efficiency of inspection tasks.
