Radiofrequency ablation(RFA)guided by X-ray images aims to relieve herniated disc pain with mini-mal invasiveness and fast recovery.It requires an accurate and fast positioning of the puncture needle.We propose a tele...Radiofrequency ablation(RFA)guided by X-ray images aims to relieve herniated disc pain with mini-mal invasiveness and fast recovery.It requires an accurate and fast positioning of the puncture needle.We propose a teleoperated robotic system for percutaneous puncture to support RFA.We report the kinematics modelling and workspace analysis of the proposed system,which comprises preliminary and accurate positioning mechanisms.Preliminary positioning mechanism automatically drives the needle to the puncture area,and accurate positioning is then achieved by teleoperation under the guidance of X-ray images.We calculate the teleoperation workspace of the robot system using a spatial search algorithm and quantitatively analyze the optimal structural parameters aiming to maximize the workspace.The workspace of the proposed robot system complies with clinical requirements to support RFA.展开更多
To automate heavy-duty hydraulic manipulators in construction applications,trajectory learning from demonstration is increasingly in demand.However,it faces difficulties in motion noise owing to factors such as size s...To automate heavy-duty hydraulic manipulators in construction applications,trajectory learning from demonstration is increasingly in demand.However,it faces difficulties in motion noise owing to factors such as size scaling and oscillation tendency.A smooth trajectory learning method is established to overcome this problem by segmenting the demonstration and extracting the subgoals for motion noise cancellation.The imperfect demonstration trajectory is segmented by clustering the end-effector’s velocity in the task space with locally weighted noise cancellation to reduce the impact of velocity fluctuations.A sequentially hierarchical Dirichlet process algorithm with temporal encoding is designed to extract the intended subgoals and filter inefficient operations.Then,the learned trajectory is reconstructed,combined with dynamic motion primitives(DMP).The comparison test results indicate that the proposed method can learn a relevant trajectory that reflects the real intention of the user from an imperfect demonstration.Taking DMP and Sparse Sampling as comparisons,two cases of automatic trajectory tracking tasks are performed,which shows that the average position error with respect to the reference can be reduced because inefficient operations or movements are effectively filtered.展开更多
Digital simulation of the full operation of a remotely operated vehicle(ROV)is an economically feasible way for algorithm pretesting and operator training prior to the actual underwater tasks,due to the huge diffculti...Digital simulation of the full operation of a remotely operated vehicle(ROV)is an economically feasible way for algorithm pretesting and operator training prior to the actual underwater tasks,due to the huge diffculties encountered during the underwater test,high equipment cost,and the time-consuming nature of the process.In this paper,a human-interactive digital simulation platform is established for the navigation,motion,and teleoperated manipulation of work-class ROVs,and provides the human operator with the visualized full operation process.Specially,two mechanisms are presented in this platform:one provides the virtual simulation platform for operator training;the other provides real-time visual and force feedback when implementing the actual tasks.Moreover,an open data interface is designed for researchers for pretesting various algorithms before implementing the actual underwater tasks.Additionally,typical underwater scenarios of the ROV,including underwater sediment sampling and pipeline docking tasks,are selected as the case studies for hydrodynamics-based simulation.Human operator can operate the manipulator installed on the ROV via the master manipulator with the visual and force feedback after the ROV is navigated to the desired position.During the full operation,the dynamic windows approach(DWA)-based local navigation algorithm,sliding mode control(SMC)controller,and the teleoperation control framework are implemented to show the effectiveness of the designed platform.Finally,a user study on the ROV operation mode is carried out,and several metrics are designed to evaluate the superiority and accuracy of the digital simulation platform for immersive underwater teleoperation.展开更多
A 3D augmented reality navigation system using stereoscopic images is developed for teleoperated robot systems. The accurate matching between the simulated model and the video image of the actual robot can be realized...A 3D augmented reality navigation system using stereoscopic images is developed for teleoperated robot systems. The accurate matching between the simulated model and the video image of the actual robot can be realized, which helps the operator to accomplish the remote control task correctly and reliably. The system introduces the disparity map translation transformation method to take parallax images for stereoscopic displays, providing the operator an immersive 3D experience. Meanwhile, a fast and accurate registration method of dynamic stereo video is proposed, and effective integration of a virtual robot and the real stereo scene can be achieved. Preliminary experiments show that operation error of the system is maintained at less than 2.2 mm and the average error is 0.854 7, 0.909 3 and 0.697 2 mm at x, y, z direction respectively. Lots of experiments such as pressing the button, pulling the drawer and so on are also conducted to evaluate the performance of the system. The feasibility studies show that the depth information of structure can be rapidly and recognized in remote environment site. The augmented reality of the image overlay system could increase the operating accuracy and reduce the procedure time as a result of intuitive 3D viewing.展开更多
Robust teleoperation in image-guided interventions faces critical challenges from latency,deformation,and the quasi-periodic nature of physiological motion.This paper presents a fully integrated,latency-aware visual s...Robust teleoperation in image-guided interventions faces critical challenges from latency,deformation,and the quasi-periodic nature of physiological motion.This paper presents a fully integrated,latency-aware visual servoing system leveraging stereo vision,hand–eye calibration,and learning-based prediction for motion-compensated teleoperation.The system combines a calibrated binocular camera setup,dual robotic arms,and a predictive control loop incorporating Long Short-Term Memory(LSTM)and Temporal Convolutional Network(TCN)models.Through experiments using both in vivo and phantom datasets,we quantitatively assess the prediction accuracy and motion-compensation performance of both models.Results show that TCNs deliver more stable and precise tracking,especially on regular trajectories,while LSTMs exhibit robustness under quasi-periodic dynamics.By matching prediction horizons to system latency,the approach significantly reduces peak and steady-state tracking errors,demonstrating practical feasibility for deploying prediction-augmented servoing in teleoperated surgical.展开更多
Force feedback bilateral teleoperation represents a pivotal advancement in control technology,finding widespread application in hazardous material transportation,perilous environments,space and deep-sea exploration,an...Force feedback bilateral teleoperation represents a pivotal advancement in control technology,finding widespread application in hazardous material transportation,perilous environments,space and deep-sea exploration,and healthcare domains.This paper traces the evolutionary trajectory of force feedback bilateral teleoperation from its conceptual inception to its current complexity.It elucidates the fundamental principles underpinning interaction forces and tactile exchanges,with a specific emphasis on the crucial role of tactile devices.In this review,a quantitative analysis of force feedback bilateral teleoperation development trends from 2011 to 2024 has been conducted,utilizing published journal article data as the primary source of information.The review accentuates classical control frameworks and algorithms,while also delving into existing research advancements and prospec-tive breakthrough directions.Moreover,it explores specific practical scenarios ranging from intricate surgeries to hazardous environment exploration,underscoring the technology’s potential to revolutionize industries by augmenting human manipulation of remote systems.This underscores the pivotal role of force feedback bilateral teleoperation as a transformative human-machine interface,capable of shaping flexible control strategies and addressing technological bottlenecks.Future research endeavors in force feedback bilateral teleoperation are expected to prioritize the creation of more immersive experiences,overcoming technical hurdles,fortifying human-machine collaboration,and broadening application domains,particularly within the realms of medical intervention and hazardous environments.With the continuous progression of technology,the integration of human intelligence and robotic capabilities is expected to produce more innovations and breakthroughs in the field of automatic control.展开更多
Haptic teleoperation in nuclear and aerospace applications faces challenges such as a limited workspace,high payload demands,and the need for both coarse positioning and fine manipulation.Existing commercial systems o...Haptic teleoperation in nuclear and aerospace applications faces challenges such as a limited workspace,high payload demands,and the need for both coarse positioning and fine manipulation.Existing commercial systems often lack structural consistency between master and slave devices,which leads to complex motion mapping and limited adaptability.This paper presents a modular isomorphic haptic master device and dual-mode control strategy tailored for these environments.Two reconfigurable versions(5-DOF and 6-DOF)were developed to match the task-specific slave arms.The system supports autonomous-to-manual switching,joint and end-effector mapping,and real-time haptic rendering.Simulations and experiments verified their performance in representative scenarios.The proposed solution addresses structural mismatches and control inflexibility through a scalable task-driven design for high-risk remote operations.展开更多
Master robots are integral components of teleoperated robot-assisted minimally invasive surgery systems.Among them,parallel mechanism-based 6-degree-of-freedom master robots are distinguished by low inertia and high-f...Master robots are integral components of teleoperated robot-assisted minimally invasive surgery systems.Among them,parallel mechanism-based 6-degree-of-freedom master robots are distinguished by low inertia and high-force feedback.However,complex kinematics and singularities are the main barriers limiting its usage.This study converts the Hexa-type 6-RUS mechanism into a master robot to construct master-slave teleoperation system.The clinical background is briefly introduced and a representative surgical robot is employed to analyze the master-slave mapping relationship.The inverse/forward kinematics,the Jacobian matrix,and the translation and orientation workspace are derived as the bases of master robot’s application.The architecture parameters are optimized by the global transmission index to achieve better motion/force transmissibility.Based on the optimal result,the prototype and the master-slave control loop are constructed.Finally,the corresponding master-slave teleoperation experiment and model experiment demonstrate that the proposed master robot satisfies the basic need for medical application.展开更多
This article investigates virtual reality (VR)-based teleoperation with robustness against modeling errors. VR technology is an effective way to overcome the large time delay during space robot teleoperation. However,...This article investigates virtual reality (VR)-based teleoperation with robustness against modeling errors. VR technology is an effective way to overcome the large time delay during space robot teleoperation. However, it depends highly on the accuracy of model. Model errors between the virtual and real environment exist inevitably. The existing way to deal with the problem is by means of either model matching or robot compliance control. As distinct from the existing methods, this article tries to combine m...展开更多
The worldwide research status of head tracking is introduced and the works made in the research of the predictive algorithm and in the exploration of the rule of the head tracking are set forth. A time delay model f...The worldwide research status of head tracking is introduced and the works made in the research of the predictive algorithm and in the exploration of the rule of the head tracking are set forth. A time delay model for the telerobotic scout system is built. In respect of eliminating error caused by time delay and making reasonable prediction to the data stream, many methods are experimented in order to realize the aim of real time tracking. The application of extrapolation algorithm and auto recursive algorithm in the orientation tracking is described in detail. These two algorithms are realized in Matlab environment. Through analysis of the curves generated by using these two predictive algorithms, an appropriate method was applied in the telerobotic scout system. The effect is satisfying.展开更多
This paper presents a new solution to haptic based teleoperation to control a large-sized slave robot for space exploration, which includes two specially designed haptic joysticks, a hybrid master-slave motion mapping...This paper presents a new solution to haptic based teleoperation to control a large-sized slave robot for space exploration, which includes two specially designed haptic joysticks, a hybrid master-slave motion mapping method, and a haptic feedback model rendering the operating resistance and the interactive feedback on the slave side. Two devices using the 3 R and DELTA mechanisms respectively are developed to be manipulated to control the position and orientation of a large-sized slave robot by using both of a user's two hands respectively. The hybrid motion mapping method combines rate control and variable scaled position mapping to realize accurate and efficient master-slave control. Haptic feedback for these two mapping modes is designed with emphasis on ergonomics to improve the immersion of haptic based teleoperation. A stiffness estimation method is used to calculate the contact stiffness on the slave side and play the contact force rendered by using a traditional spring-damping model to a user on the master side stably. Experiments by using virtual environments to simulate the slave side are conducted to validate the effectiveness and efficiency of the proposed solution.展开更多
This paper presents a new composite nonlinear bilateral control method based on the nonlinear disturbance observer(NDOB) for teleoperation systems with external disturbances. By introducing the estimations of NDOB and...This paper presents a new composite nonlinear bilateral control method based on the nonlinear disturbance observer(NDOB) for teleoperation systems with external disturbances. By introducing the estimations of NDOB and systems' nominal nonlinear dynamics into controller design, a NDOB based composite nonlinear bilateral controller is constructed to attenuate the influence of disturbance and uncertain nonlinearities. As compared with the existing bilateral control methods which usually achieve force haptic(i.e., contact force tracking)through a passive way, the newly proposed method has two major merits: 1) asymptotical convergence of both position and force tracking errors is guaranteed;2) disturbance influence on force tracking error dynamics is rejected through the direct feedforward compensation of disturbance estimation. Simulations on a nonlinear teleoperation system are carried out and the results validate the effectiveness of the proposed controller.展开更多
A particular emphasis is put on a novel wearable exoskeleton arm, ZJUESA, with 6 degrees of freedom, which is used for the robot teleoperation with the force-feedback in the unknown environment. In this external struc...A particular emphasis is put on a novel wearable exoskeleton arm, ZJUESA, with 6 degrees of freedom, which is used for the robot teleoperation with the force-feedback in the unknown environment. In this external structure mechanism, the 3-revolution-prismatic-spherical （3RPS） parallel mechanism is devised from the concept of the human upper-limb anatomy and applied for the shoulder 3-DOF joint. Meanwhile, the orthogonal experiment design method is introduced for its optimal design. Aiming at enhancing the performance of teleoperation, the force feedback is employed by the pneumatic system on ZJUESA to produce the vivid feeling in addition to the soft control interface. Due to the compressibility and nonlinearity of the pneumatic force feedback system, a novel hybrid fuzzy controller for the precise force control is proposed and realized based on the Mega8 microcontroller units as the units of the distributed control system on ZJUESA. With the results of several experiments for master-slave control with force feedback, the feasibility of ZJUESA system and the effect of its hybrid fuzzy controller are verified.展开更多
Most target grabbing problems have been dealt with by computer vision system,however,computer vision method is not always enough when it comes to the precision contact grabbing problems during the teleoperation proces...Most target grabbing problems have been dealt with by computer vision system,however,computer vision method is not always enough when it comes to the precision contact grabbing problems during the teleoperation process,and need to be combined with the stiffness display to provide more effective information to the operator on the remote side.Therefore,in this paper a more portable stiffness display device with a small volume and extended function is developed based on our previous work.A new static load calibration of the improved stiffness display device is performed to detect its accuracy,and the relationship between the stiffness and the position is given.An effective target grabbing strategy is presented to help operator on the remote side to judge and control and the target is classified by multi-class SVM(supporter vector machine).The teleoperation system is established to test and verify the feasibility.A special experiment is designed and the results demonstrate that the improved stiffness display device could greatly help operator on the remote side control the telerobot to grab target and the target grabbing strategy is effective.展开更多
Teleoperation can assist people to complete various complex tasks in inaccessible or high-risk environments,in which a wearable hand exoskeleton is one of the key devices.Adequate adaptability would be available to en...Teleoperation can assist people to complete various complex tasks in inaccessible or high-risk environments,in which a wearable hand exoskeleton is one of the key devices.Adequate adaptability would be available to enable the master hand exoskeleton to capture the motion of human fingers and reproduce the contact force between the slave hand and its object.This paper presents a novel finger exoskeleton based on the cascading four-link closed-loop kinematic chain.Each finger has an independent closed-loop kinematic chain,and the angle sensors are used to obtain the finger motion including the flexion/extension and the adduction/abduction.The cable tension is changed by the servo motor to transmit the contact force to the fingers in real time.Based on the finger exoskeleton,an adaptive hand exoskeleton is consequently developed.In addition,the hand exoskeleton is tested in a master-slave system.The experiment results show that the adaptive hand exoskeleton can be worn without any mechanical constraints,and the slave hand can follow the motions of each human finger.The accuracy and the real-time capability of the force reproduction are validated.The proposed adaptive hand exoskeleton can be employed as the master hand to remotely control the humanoid five-fingered dexterous slave hand,thus,enabling the teleoperation system to complete complex dexterous manipulation tasks.展开更多
Multi-scale casting parts are important components of high-end equipment used in the aerospace,automobile manufacturing,shipbuilding,and other industries.Residual features such as parting lines and pouring risers that...Multi-scale casting parts are important components of high-end equipment used in the aerospace,automobile manufacturing,shipbuilding,and other industries.Residual features such as parting lines and pouring risers that inevitably appear during the casting process are random in size,morphology,and distribution.The traditional manual processing method has disadvantages such as low efficiency,high labor intensity,and harsh working environment.Existing machine tool and serial robot grinding/cutting equipment do not easily achieve high-quality and high-efficiency removal of residual features due to poor dexterity and low stiffness,respectively.To address these problems,a five-degree-of-freedom(5-DoF)hybrid grinding/cutting robot with high dexterity and high stiffness is proposed.Based on it,three types of grinding/cutting equipment combined with offline programming,master-slave control,and other technologies are developed to remove the residual features of small,medium,and large casting parts.Finally,the advantages of teleoperation processing and other solutions are elaborated,and the difficulties and challenges are discussed.This paper reviews the grinding/cutting technology and equipment of casting parts and provides a reference for the research on the processing of multi-scale casting parts.展开更多
State convergence is a novel control algorithm for bilateral teleoperation of robotic systems. First, it models the teleoperation system on state space and considers all the possible interactions between the master an...State convergence is a novel control algorithm for bilateral teleoperation of robotic systems. First, it models the teleoperation system on state space and considers all the possible interactions between the master and slave systems. Second, it presents an elegant design procedure which requires a set of equations to be solved in order to compute the control gains of the bilateral loop. These design conditions are obtained by turning the master-slave error into an autonomous system and imposing the desired dynamic behavior of the teleoperation system. Resultantly, the convergence of master and slave states is achieved in a well-defined manner. The present study aims at achieving a similar convergence behavior offered by state convergence controller while reducing the number of variables sent across the communication channel. The proposal suggests transmitting composite master and slave variables instead of full master and slave states while keeping the operator's force channel intact. We show that,with these composite and force variables;it is indeed possible to achieve the convergence of states in a desired way by strictly following the method of state convergence. The proposal leads to a reduced complexity state convergence algorithm which is termed as composite state convergence controller. In order to validate the proposed scheme in the absence and presence of communication time delays, MATLAB simulations and semi-real time experiments are performed on a single degree-of-freedom teleoperation system.展开更多
Based on a robotic telesurgery system whose function is to liberate doctor from X-ray radiation, a robotic tele-drill system is constructed. The system is in client/server structure. Client part includes main control ...Based on a robotic telesurgery system whose function is to liberate doctor from X-ray radiation, a robotic tele-drill system is constructed. The system is in client/server structure. Client part includes main control interface, video-audio interface and predictive display interface. Server part includes robot control server and video, audio server. For applying to teleoperation, a virtual reality environment of the system developed by using Java, Java 3D, Pro/E, etc. is established. The geometry and kinematics model of serial robot MOTOMAN sv3x, parallel robot, C-type arm and X-ray machine, surgery bed and its work environment are fulfilled in it. Simulation engine and its simulation syntax are finished, which made the environment controllable. This environment is used as predictive display interface in the telerobotics in order to tackling the problem in visualization feedback as ambiguous or time delay. Experiments that verified feasibility of the system have been done.展开更多
The teleoperation of a 6 degrees-of-freedom(DOF)manipulator is one of the basic methods to extend people’s capabilities in the wide variety of applications.The master interface based on the force/torque(FT)sensor cou...The teleoperation of a 6 degrees-of-freedom(DOF)manipulator is one of the basic methods to extend people’s capabilities in the wide variety of applications.The master interface based on the force/torque(FT)sensor could provide the full-dimension intuitive teleoperation of a 6-DOF robot since it has the ability to trigger 6-DOF command input.However,due to the force coupling,noise disturbance and unlimited input signals of the FT sensor,this force-sensed interface could not be widely used in practice.In this paper,we present an intuitive teleoperation method based on the FT sensor to overcome these challenges.In this method,the input signals from the force-sensed joystick were filtered and then processed to the force commands by force limit algorithm,with the merits of anti-interference,output limitation,and online velocity adjustment.Furthermore,based on the admittance control and position controller,the manipulator could be teleoperated by the force commands.Three experiments were conducted on our self-designed robotic system.The result of the first experiment shows that the interfered force from the force coupling could be effectively suppressed with the limitation of the input force through force limit algorithm.Then,a parameter was introduced in the other two experiments to adjust the velocity online practically with force limit algorithm.The proposed method could give a practical solution to the intuitive teleoperation based on the FT sensor.展开更多
Currently,most teleoperation work is focusing on scenarios where slave robots interact with unknown environments.However,in some fields such as medical robots or rescue robots,the other typical teleoperation applicati...Currently,most teleoperation work is focusing on scenarios where slave robots interact with unknown environments.However,in some fields such as medical robots or rescue robots,the other typical teleoperation application is precise object transportation.Generally,the object’s weight is unknown yet essential for both accurate control of the slave robot and intuitive perception of the human operator.However,due to high cost and limited installation space,it is unreliable to employ a force sensor to directly measure the weight.Therefore,in this paper,a control scheme free of force sensor is proposed for teleoperation robots to transfer a weight-unknown object accurately.In this scheme,the workspace mapping between master and slave robot is firstly established,based on which,the operator can generate command trajectory on-line by operating the master robot.Then,a slave controller is designed to follow the master command closely and estimate the object’s weight rapidly,accurately and robust to unmodeled uncertainties.Finally,for the sake of telepresence,a master controller is designed to generate force feedback to reproduce the estimated weight of the object.In the end,comparative experiments show that the proposed scheme can achieve better control accuracy and telepresence,with accurate force feedback generated in only 500 ms.展开更多
基金the National Natural Science Foundation of China(No.51575343)。
文摘Radiofrequency ablation(RFA)guided by X-ray images aims to relieve herniated disc pain with mini-mal invasiveness and fast recovery.It requires an accurate and fast positioning of the puncture needle.We propose a teleoperated robotic system for percutaneous puncture to support RFA.We report the kinematics modelling and workspace analysis of the proposed system,which comprises preliminary and accurate positioning mechanisms.Preliminary positioning mechanism automatically drives the needle to the puncture area,and accurate positioning is then achieved by teleoperation under the guidance of X-ray images.We calculate the teleoperation workspace of the robot system using a spatial search algorithm and quantitatively analyze the optimal structural parameters aiming to maximize the workspace.The workspace of the proposed robot system complies with clinical requirements to support RFA.
基金supported in part by the National Natural Science Foundation of China(Grant Nos.52322503 and 52075055)in part by the Natural Science Foundation of Chongqing,China(Grant No.2024NSCQ-JQX0123)+1 种基金in part by the Major Scientific and Technological Research and Development Project of Jiangxi Province,China(Grant No.20233AAE02001)in part by the Independent Research Project of State Key Laboratory of Mechanical Transmission for Advanced Equipment,China(Grant No.SKLMT-ZZKT-2024R02).
文摘To automate heavy-duty hydraulic manipulators in construction applications,trajectory learning from demonstration is increasingly in demand.However,it faces difficulties in motion noise owing to factors such as size scaling and oscillation tendency.A smooth trajectory learning method is established to overcome this problem by segmenting the demonstration and extracting the subgoals for motion noise cancellation.The imperfect demonstration trajectory is segmented by clustering the end-effector’s velocity in the task space with locally weighted noise cancellation to reduce the impact of velocity fluctuations.A sequentially hierarchical Dirichlet process algorithm with temporal encoding is designed to extract the intended subgoals and filter inefficient operations.Then,the learned trajectory is reconstructed,combined with dynamic motion primitives(DMP).The comparison test results indicate that the proposed method can learn a relevant trajectory that reflects the real intention of the user from an imperfect demonstration.Taking DMP and Sparse Sampling as comparisons,two cases of automatic trajectory tracking tasks are performed,which shows that the average position error with respect to the reference can be reduced because inefficient operations or movements are effectively filtered.
基金Project supported by the National Natural Science Foundation of China(Nos.52075476 and 52301404)the Zhejiang Provincial Natural Science Foundation of China(No.LR23E050001)。
文摘Digital simulation of the full operation of a remotely operated vehicle(ROV)is an economically feasible way for algorithm pretesting and operator training prior to the actual underwater tasks,due to the huge diffculties encountered during the underwater test,high equipment cost,and the time-consuming nature of the process.In this paper,a human-interactive digital simulation platform is established for the navigation,motion,and teleoperated manipulation of work-class ROVs,and provides the human operator with the visualized full operation process.Specially,two mechanisms are presented in this platform:one provides the virtual simulation platform for operator training;the other provides real-time visual and force feedback when implementing the actual tasks.Moreover,an open data interface is designed for researchers for pretesting various algorithms before implementing the actual underwater tasks.Additionally,typical underwater scenarios of the ROV,including underwater sediment sampling and pipeline docking tasks,are selected as the case studies for hydrodynamics-based simulation.Human operator can operate the manipulator installed on the ROV via the master manipulator with the visual and force feedback after the ROV is navigated to the desired position.During the full operation,the dynamic windows approach(DWA)-based local navigation algorithm,sliding mode control(SMC)controller,and the teleoperation control framework are implemented to show the effectiveness of the designed platform.Finally,a user study on the ROV operation mode is carried out,and several metrics are designed to evaluate the superiority and accuracy of the digital simulation platform for immersive underwater teleoperation.
基金supported by the Fundamental Research Funds for the Central Universities (2009RC0603)the Hi-Tech Research and Development Program of China (2009AA7034550)the National Natural Science Foundation of China (60803103)
文摘A 3D augmented reality navigation system using stereoscopic images is developed for teleoperated robot systems. The accurate matching between the simulated model and the video image of the actual robot can be realized, which helps the operator to accomplish the remote control task correctly and reliably. The system introduces the disparity map translation transformation method to take parallax images for stereoscopic displays, providing the operator an immersive 3D experience. Meanwhile, a fast and accurate registration method of dynamic stereo video is proposed, and effective integration of a virtual robot and the real stereo scene can be achieved. Preliminary experiments show that operation error of the system is maintained at less than 2.2 mm and the average error is 0.854 7, 0.909 3 and 0.697 2 mm at x, y, z direction respectively. Lots of experiments such as pressing the button, pulling the drawer and so on are also conducted to evaluate the performance of the system. The feasibility studies show that the depth information of structure can be rapidly and recognized in remote environment site. The augmented reality of the image overlay system could increase the operating accuracy and reduce the procedure time as a result of intuitive 3D viewing.
基金Support by Sichuan Science and Technology Program[2023YFSY0026,2023YFH0004]Guangzhou Huashang University[2024HSZD01,HS2023JYSZH01].
文摘Robust teleoperation in image-guided interventions faces critical challenges from latency,deformation,and the quasi-periodic nature of physiological motion.This paper presents a fully integrated,latency-aware visual servoing system leveraging stereo vision,hand–eye calibration,and learning-based prediction for motion-compensated teleoperation.The system combines a calibrated binocular camera setup,dual robotic arms,and a predictive control loop incorporating Long Short-Term Memory(LSTM)and Temporal Convolutional Network(TCN)models.Through experiments using both in vivo and phantom datasets,we quantitatively assess the prediction accuracy and motion-compensation performance of both models.Results show that TCNs deliver more stable and precise tracking,especially on regular trajectories,while LSTMs exhibit robustness under quasi-periodic dynamics.By matching prediction horizons to system latency,the approach significantly reduces peak and steady-state tracking errors,demonstrating practical feasibility for deploying prediction-augmented servoing in teleoperated surgical.
基金supported by the MSIT(Ministry of Science and ICT),Republic of Korea,under the Convergence Security Core Talent Training Business Support Program(IITP-2024-RS-2024-00423071)supervised by the IITP(Institute of Information&Communications Technology Planning&Evaluation)supported by Sichuan Science and Technology Program(2023YFSY0026,2023YFH0004).
文摘Force feedback bilateral teleoperation represents a pivotal advancement in control technology,finding widespread application in hazardous material transportation,perilous environments,space and deep-sea exploration,and healthcare domains.This paper traces the evolutionary trajectory of force feedback bilateral teleoperation from its conceptual inception to its current complexity.It elucidates the fundamental principles underpinning interaction forces and tactile exchanges,with a specific emphasis on the crucial role of tactile devices.In this review,a quantitative analysis of force feedback bilateral teleoperation development trends from 2011 to 2024 has been conducted,utilizing published journal article data as the primary source of information.The review accentuates classical control frameworks and algorithms,while also delving into existing research advancements and prospec-tive breakthrough directions.Moreover,it explores specific practical scenarios ranging from intricate surgeries to hazardous environment exploration,underscoring the technology’s potential to revolutionize industries by augmenting human manipulation of remote systems.This underscores the pivotal role of force feedback bilateral teleoperation as a transformative human-machine interface,capable of shaping flexible control strategies and addressing technological bottlenecks.Future research endeavors in force feedback bilateral teleoperation are expected to prioritize the creation of more immersive experiences,overcoming technical hurdles,fortifying human-machine collaboration,and broadening application domains,particularly within the realms of medical intervention and hazardous environments.With the continuous progression of technology,the integration of human intelligence and robotic capabilities is expected to produce more innovations and breakthroughs in the field of automatic control.
基金Supported by National Key Research and Development Program of China(Grant No.2022YFE0112500).
文摘Haptic teleoperation in nuclear and aerospace applications faces challenges such as a limited workspace,high payload demands,and the need for both coarse positioning and fine manipulation.Existing commercial systems often lack structural consistency between master and slave devices,which leads to complex motion mapping and limited adaptability.This paper presents a modular isomorphic haptic master device and dual-mode control strategy tailored for these environments.Two reconfigurable versions(5-DOF and 6-DOF)were developed to match the task-specific slave arms.The system supports autonomous-to-manual switching,joint and end-effector mapping,and real-time haptic rendering.Simulations and experiments verified their performance in representative scenarios.The proposed solution addresses structural mismatches and control inflexibility through a scalable task-driven design for high-risk remote operations.
基金the National Natural Science Foundation of China(No.62211540723)the Interdisciplinary Program of Shanghai Jiao Tong University(Nos.YG2023ZD05 and YG2023ZD14)+1 种基金the Quanzhou High-Level Talent Innovation and Entrepreneurship Project(No.2021C003R)the Research Project of Institute of Medical Robotics of Shanghai Jiao Tong University。
文摘Master robots are integral components of teleoperated robot-assisted minimally invasive surgery systems.Among them,parallel mechanism-based 6-degree-of-freedom master robots are distinguished by low inertia and high-force feedback.However,complex kinematics and singularities are the main barriers limiting its usage.This study converts the Hexa-type 6-RUS mechanism into a master robot to construct master-slave teleoperation system.The clinical background is briefly introduced and a representative surgical robot is employed to analyze the master-slave mapping relationship.The inverse/forward kinematics,the Jacobian matrix,and the translation and orientation workspace are derived as the bases of master robot’s application.The architecture parameters are optimized by the global transmission index to achieve better motion/force transmissibility.Based on the optimal result,the prototype and the master-slave control loop are constructed.Finally,the corresponding master-slave teleoperation experiment and model experiment demonstrate that the proposed master robot satisfies the basic need for medical application.
基金National Natural Science Foundation of China (60675054)National High-Tech Research and Development Program (2006AA04Z228)"111" Project (B07018)
文摘This article investigates virtual reality (VR)-based teleoperation with robustness against modeling errors. VR technology is an effective way to overcome the large time delay during space robot teleoperation. However, it depends highly on the accuracy of model. Model errors between the virtual and real environment exist inevitably. The existing way to deal with the problem is by means of either model matching or robot compliance control. As distinct from the existing methods, this article tries to combine m...
文摘The worldwide research status of head tracking is introduced and the works made in the research of the predictive algorithm and in the exploration of the rule of the head tracking are set forth. A time delay model for the telerobotic scout system is built. In respect of eliminating error caused by time delay and making reasonable prediction to the data stream, many methods are experimented in order to realize the aim of real time tracking. The application of extrapolation algorithm and auto recursive algorithm in the orientation tracking is described in detail. These two algorithms are realized in Matlab environment. Through analysis of the curves generated by using these two predictive algorithms, an appropriate method was applied in the telerobotic scout system. The effect is satisfying.
基金supported by the Open Research Fund of Key Laboratory of Space Utilization,Chinese Academy of Sciences(No.LSU-YKZX-2017-02)
文摘This paper presents a new solution to haptic based teleoperation to control a large-sized slave robot for space exploration, which includes two specially designed haptic joysticks, a hybrid master-slave motion mapping method, and a haptic feedback model rendering the operating resistance and the interactive feedback on the slave side. Two devices using the 3 R and DELTA mechanisms respectively are developed to be manipulated to control the position and orientation of a large-sized slave robot by using both of a user's two hands respectively. The hybrid motion mapping method combines rate control and variable scaled position mapping to realize accurate and efficient master-slave control. Haptic feedback for these two mapping modes is designed with emphasis on ergonomics to improve the immersion of haptic based teleoperation. A stiffness estimation method is used to calculate the contact stiffness on the slave side and play the contact force rendered by using a traditional spring-damping model to a user on the master side stably. Experiments by using virtual environments to simulate the slave side are conducted to validate the effectiveness and efficiency of the proposed solution.
基金supported in part by the National Natural Science Foundation of China(61573099,61633003,61750110525,61903192)Fundamental Research Funds for the Central Universities(2242016R30011)+5 种基金Graduate Innovation Program of Jiangsu Province(KYLX15-0114)Scientific Research Foundation of Graduate School of Southeast University(YBJJ1561)Open Project Program of Ministry of EducationKey Laboratory of Measurement and Control of School of Computer Science and Engineering(CSE) MCCSE2017A01,MCCSE2019A01)Chinese Scholarship CouncilNewton Fund by the British Council
文摘This paper presents a new composite nonlinear bilateral control method based on the nonlinear disturbance observer(NDOB) for teleoperation systems with external disturbances. By introducing the estimations of NDOB and systems' nominal nonlinear dynamics into controller design, a NDOB based composite nonlinear bilateral controller is constructed to attenuate the influence of disturbance and uncertain nonlinearities. As compared with the existing bilateral control methods which usually achieve force haptic(i.e., contact force tracking)through a passive way, the newly proposed method has two major merits: 1) asymptotical convergence of both position and force tracking errors is guaranteed;2) disturbance influence on force tracking error dynamics is rejected through the direct feedforward compensation of disturbance estimation. Simulations on a nonlinear teleoperation system are carried out and the results validate the effectiveness of the proposed controller.
基金National Natural Science Foundation of China(No.50305035)
文摘A particular emphasis is put on a novel wearable exoskeleton arm, ZJUESA, with 6 degrees of freedom, which is used for the robot teleoperation with the force-feedback in the unknown environment. In this external structure mechanism, the 3-revolution-prismatic-spherical （3RPS） parallel mechanism is devised from the concept of the human upper-limb anatomy and applied for the shoulder 3-DOF joint. Meanwhile, the orthogonal experiment design method is introduced for its optimal design. Aiming at enhancing the performance of teleoperation, the force feedback is employed by the pneumatic system on ZJUESA to produce the vivid feeling in addition to the soft control interface. Due to the compressibility and nonlinearity of the pneumatic force feedback system, a novel hybrid fuzzy controller for the precise force control is proposed and realized based on the Mega8 microcontroller units as the units of the distributed control system on ZJUESA. With the results of several experiments for master-slave control with force feedback, the feasibility of ZJUESA system and the effect of its hybrid fuzzy controller are verified.
基金partially supported by the National Natural Science Foundation of China(61663027,61325018,81501560)the Science and Technology Department of Jiangxi Province of China(20151BAB207050)
文摘Most target grabbing problems have been dealt with by computer vision system,however,computer vision method is not always enough when it comes to the precision contact grabbing problems during the teleoperation process,and need to be combined with the stiffness display to provide more effective information to the operator on the remote side.Therefore,in this paper a more portable stiffness display device with a small volume and extended function is developed based on our previous work.A new static load calibration of the improved stiffness display device is performed to detect its accuracy,and the relationship between the stiffness and the position is given.An effective target grabbing strategy is presented to help operator on the remote side to judge and control and the target is classified by multi-class SVM(supporter vector machine).The teleoperation system is established to test and verify the feasibility.A special experiment is designed and the results demonstrate that the improved stiffness display device could greatly help operator on the remote side control the telerobot to grab target and the target grabbing strategy is effective.
基金Supported by National Key Research and Development Program of China(Grant No.2018YFE0125600)Zhejiang Provincial Key Research,Develop-ment Program(Grant No.2021C04015)Natural Science Foundation of Zhejiang(Grant No.LZ23E050005).
文摘Teleoperation can assist people to complete various complex tasks in inaccessible or high-risk environments,in which a wearable hand exoskeleton is one of the key devices.Adequate adaptability would be available to enable the master hand exoskeleton to capture the motion of human fingers and reproduce the contact force between the slave hand and its object.This paper presents a novel finger exoskeleton based on the cascading four-link closed-loop kinematic chain.Each finger has an independent closed-loop kinematic chain,and the angle sensors are used to obtain the finger motion including the flexion/extension and the adduction/abduction.The cable tension is changed by the servo motor to transmit the contact force to the fingers in real time.Based on the finger exoskeleton,an adaptive hand exoskeleton is consequently developed.In addition,the hand exoskeleton is tested in a master-slave system.The experiment results show that the adaptive hand exoskeleton can be worn without any mechanical constraints,and the slave hand can follow the motions of each human finger.The accuracy and the real-time capability of the force reproduction are validated.The proposed adaptive hand exoskeleton can be employed as the master hand to remotely control the humanoid five-fingered dexterous slave hand,thus,enabling the teleoperation system to complete complex dexterous manipulation tasks.
基金National Natural Science Foundation of China(Grant Nos.51875391,51875392)Tianjin Science and Technology Planning Project(Grant Nos.18PTLCSY00080,20YDLZGX00290)State Key Laboratory of Digital Manufacturing Equipment and Technology(Grant No.DMETKF2022007).
文摘Multi-scale casting parts are important components of high-end equipment used in the aerospace,automobile manufacturing,shipbuilding,and other industries.Residual features such as parting lines and pouring risers that inevitably appear during the casting process are random in size,morphology,and distribution.The traditional manual processing method has disadvantages such as low efficiency,high labor intensity,and harsh working environment.Existing machine tool and serial robot grinding/cutting equipment do not easily achieve high-quality and high-efficiency removal of residual features due to poor dexterity and low stiffness,respectively.To address these problems,a five-degree-of-freedom(5-DoF)hybrid grinding/cutting robot with high dexterity and high stiffness is proposed.Based on it,three types of grinding/cutting equipment combined with offline programming,master-slave control,and other technologies are developed to remove the residual features of small,medium,and large casting parts.Finally,the advantages of teleoperation processing and other solutions are elaborated,and the difficulties and challenges are discussed.This paper reviews the grinding/cutting technology and equipment of casting parts and provides a reference for the research on the processing of multi-scale casting parts.
基金supported by the Natural Sciences and Engineering Research Council of Canada(NSERC)
文摘State convergence is a novel control algorithm for bilateral teleoperation of robotic systems. First, it models the teleoperation system on state space and considers all the possible interactions between the master and slave systems. Second, it presents an elegant design procedure which requires a set of equations to be solved in order to compute the control gains of the bilateral loop. These design conditions are obtained by turning the master-slave error into an autonomous system and imposing the desired dynamic behavior of the teleoperation system. Resultantly, the convergence of master and slave states is achieved in a well-defined manner. The present study aims at achieving a similar convergence behavior offered by state convergence controller while reducing the number of variables sent across the communication channel. The proposal suggests transmitting composite master and slave variables instead of full master and slave states while keeping the operator's force channel intact. We show that,with these composite and force variables;it is indeed possible to achieve the convergence of states in a desired way by strictly following the method of state convergence. The proposal leads to a reduced complexity state convergence algorithm which is termed as composite state convergence controller. In order to validate the proposed scheme in the absence and presence of communication time delays, MATLAB simulations and semi-real time experiments are performed on a single degree-of-freedom teleoperation system.
文摘Based on a robotic telesurgery system whose function is to liberate doctor from X-ray radiation, a robotic tele-drill system is constructed. The system is in client/server structure. Client part includes main control interface, video-audio interface and predictive display interface. Server part includes robot control server and video, audio server. For applying to teleoperation, a virtual reality environment of the system developed by using Java, Java 3D, Pro/E, etc. is established. The geometry and kinematics model of serial robot MOTOMAN sv3x, parallel robot, C-type arm and X-ray machine, surgery bed and its work environment are fulfilled in it. Simulation engine and its simulation syntax are finished, which made the environment controllable. This environment is used as predictive display interface in the telerobotics in order to tackling the problem in visualization feedback as ambiguous or time delay. Experiments that verified feasibility of the system have been done.
基金National Key Research and Development Program of China(Grant No.2019YFB1309900)Shandong Provincial Key Research and Development Program of China(Grant No.2019JZZY010432)Institute for Guo Qiang,Tsinghua University,China(Grant No.2019GQG0007).
文摘The teleoperation of a 6 degrees-of-freedom(DOF)manipulator is one of the basic methods to extend people’s capabilities in the wide variety of applications.The master interface based on the force/torque(FT)sensor could provide the full-dimension intuitive teleoperation of a 6-DOF robot since it has the ability to trigger 6-DOF command input.However,due to the force coupling,noise disturbance and unlimited input signals of the FT sensor,this force-sensed interface could not be widely used in practice.In this paper,we present an intuitive teleoperation method based on the FT sensor to overcome these challenges.In this method,the input signals from the force-sensed joystick were filtered and then processed to the force commands by force limit algorithm,with the merits of anti-interference,output limitation,and online velocity adjustment.Furthermore,based on the admittance control and position controller,the manipulator could be teleoperated by the force commands.Three experiments were conducted on our self-designed robotic system.The result of the first experiment shows that the interfered force from the force coupling could be effectively suppressed with the limitation of the input force through force limit algorithm.Then,a parameter was introduced in the other two experiments to adjust the velocity online practically with force limit algorithm.The proposed method could give a practical solution to the intuitive teleoperation based on the FT sensor.
基金supported in part by the National Natural Science Foundation of China(52075476,92048302)Key R&D Program of Zhejiang Province(2021C03013)。
文摘Currently,most teleoperation work is focusing on scenarios where slave robots interact with unknown environments.However,in some fields such as medical robots or rescue robots,the other typical teleoperation application is precise object transportation.Generally,the object’s weight is unknown yet essential for both accurate control of the slave robot and intuitive perception of the human operator.However,due to high cost and limited installation space,it is unreliable to employ a force sensor to directly measure the weight.Therefore,in this paper,a control scheme free of force sensor is proposed for teleoperation robots to transfer a weight-unknown object accurately.In this scheme,the workspace mapping between master and slave robot is firstly established,based on which,the operator can generate command trajectory on-line by operating the master robot.Then,a slave controller is designed to follow the master command closely and estimate the object’s weight rapidly,accurately and robust to unmodeled uncertainties.Finally,for the sake of telepresence,a master controller is designed to generate force feedback to reproduce the estimated weight of the object.In the end,comparative experiments show that the proposed scheme can achieve better control accuracy and telepresence,with accurate force feedback generated in only 500 ms.
