This paper presents a method of multicopter intercep-tion control based on visual servo and virtual tube in a cluttered environment.The proposed hybrid heuristic function improves the efficiency of the A*algorithm.The...This paper presents a method of multicopter intercep-tion control based on visual servo and virtual tube in a cluttered environment.The proposed hybrid heuristic function improves the efficiency of the A*algorithm.The revised objective function makes the virtual tube generating curve not only smooth but also close to the path points generated by the A*algorithm.In six dif-ferent simulation scenarios,the efficiency of the modified A*algorithm is 6.2%higher than that of the traditional A*algorithm.The efficiency of path planning and virtual tube planning is veri-fied by simulations.The effectiveness of interception control is verified by a software-in-loop(SIL)simulation.展开更多
In this paper,a visual servoing approach is developed to capture the docking rings of tumbling non-cooperative satellites with a space manipulator.The primary challenge addressed is the potential for the docking ring ...In this paper,a visual servoing approach is developed to capture the docking rings of tumbling non-cooperative satellites with a space manipulator.The primary challenge addressed is the potential for the docking ring to leave the monocular camera’s field-of-view as the manipulator approaches the target,due to the ring’s large size.To solve this issue,a two-phase visual servoing scheme combining a monocular camera and a three-line structured light vision system is proposed.In an effort to augment the success rate and safety of capture operations,several constraints are formulated,encompassing manipulator’s kinematics,monocular camera’s field-of-view,obstacle avoidance,structured light’s breakpoints and smooth capture.Subsequently,a nonlinear model predictive controller is proposed to manage these constraints in real-time and regulate the system.System models are established based on image moments and pose for each phase,selecting these features as visual feedback to simplify the formulation of servo constraints and avoid the complex circle-based pose measurement.Furthermore,to ensure unbiased predictions,the model disturbances arising from the imprecise estimation of target motion parameter are observed using an extended Kalman filter,which are then incorporated into the predictive control framework.The simulation results demonstrate the effectiveness of this scheme.展开更多
This paper addresses a coordinated control problem for Spacecraft Formation Flying(SFF). The distributed followers are required to track and synchronize with the leader spacecraft.By using the feature points in the tw...This paper addresses a coordinated control problem for Spacecraft Formation Flying(SFF). The distributed followers are required to track and synchronize with the leader spacecraft.By using the feature points in the two-dimensional image space, an integrated 6-degree-of-freedom dynamic model is formulated for spacecraft relative motion. Without sophisticated threedimensional reconstruction, image features are directly utilized for the controller design. The proposed image-based controller can drive the follower spacecraft in the desired configuration with respect to the leader when the real-time captured images match their reference counterparts. To improve the precision of the formation configuration, the proposed controller employs a coordinated term to reduce the relative distance errors between followers. The uncertainties in the system dynamics are handled by integrating the adaptive technique into the controller, which increases the robustness of the SFF system. The closed-loop system stability is analyzed using the Lyapunov method and algebraic graph theory. A numerical simulation for a given SFF scenario is performed to evaluate the performance of the controller.展开更多
A visual servoing tracking controller is proposed based on the sliding mode control theory in order to achieve strong robustness against parameter variations and external disturbances. A sliding plane with time delay ...A visual servoing tracking controller is proposed based on the sliding mode control theory in order to achieve strong robustness against parameter variations and external disturbances. A sliding plane with time delay compensation is presented by the pre-estimate of states. To reduce the chattering of the sliding mode controller, a modified exponential reaching law and hyperbolic tangent function are applied to the design of visual controller and robot joint controller. Simulation results show that the visual servoing control scheme is robust and has good tracking performance.展开更多
There are two main trends in the development of unmanned aerial vehicle(UAV)technologies:miniaturization and intellectualization,in which realizing object tracking capabilities for a nano-scale UAV is one of the most ...There are two main trends in the development of unmanned aerial vehicle(UAV)technologies:miniaturization and intellectualization,in which realizing object tracking capabilities for a nano-scale UAV is one of the most challenging problems.In this paper,we present a visual object tracking and servoing control system utilizing a tailor-made 38 g nano-scale quadrotor.A lightweight visual module is integrated to enable object tracking capabilities,and a micro positioning deck is mounted to provide accurate pose estimation.In order to be robust against object appearance variations,a novel object tracking algorithm,denoted by RMCTer,is proposed,which integrates a powerful short-term tracking module and an efficient long-term processing module.In particular,the long-term processing module can provide additional object information and modify the short-term tracking model in a timely manner.Furthermore,a positionbased visual servoing control method is proposed for the quadrotor,where an adaptive tracking controller is designed by leveraging backstepping and adaptive techniques.Stable and accurate object tracking is achieved even under disturbances.Experimental results are presented to demonstrate the high accuracy and stability of the whole tracking system.展开更多
A new visual servo control scheme for a robotic manipulator is presented in this paper, where a back propagation （BP） neural network is used to make a direct transition from image feature to joint angles without req...A new visual servo control scheme for a robotic manipulator is presented in this paper, where a back propagation （BP） neural network is used to make a direct transition from image feature to joint angles without requiring robot kinematics and camera calibration. To speed up the convergence and avoid local minimum of the neural network, this paper uses a genetic algorithm to find the optimal initial weights and thresholds and then uses the BP Mgorithm to train the neural network according to the data given. The proposed method can effectively combine the good global searching ability of genetic algorithms with the accurate local searching feature of BP neural network. The Simulink model for PUMA560 robot visual servo system based on the improved BP neural network is built with the Robotics Toolbox of Matlab. The simulation results indicate that the proposed method can accelerate convergence of the image errors and provide a simple and effective way of robot control.展开更多
This paper presents an improved support vector machine （SVM） algorithm, which employs invariant moments-based edge extraction to obtain feature attribute. A heuristic attribute reduction algorithm based on rough set...This paper presents an improved support vector machine （SVM） algorithm, which employs invariant moments-based edge extraction to obtain feature attribute. A heuristic attribute reduction algorithm based on rough set＇s discernible matrix is proposed to identify and classify micro-targets. To avoid the complicated calibration for intrinsic parameters of camera, an improved Broyden＇s method is proposed to estimate the image Jacobian matrix which employs Chebyshev polynomial to construct a cost function to approximate the optimization value. Finally, a visual controller is designed for a robotic micromanipulation system. The experiment results of micro-parts assembly show that the proposed methods and algorithms are effective and feasible.展开更多
In this paper, we propose a method to select the observation position in visual servoing with an eye-in-vehicle configuration for the manipulator. In traditional visual servoing, the images taken by the camera may hav...In this paper, we propose a method to select the observation position in visual servoing with an eye-in-vehicle configuration for the manipulator. In traditional visual servoing, the images taken by the camera may have various problems, including being out of view, large perspective aberrance, improper projection area of object in images and so on. In this paper, we propose a method to determine the observation position to solve these problems. A mobile robot system with pan-tilt camera is designed, which calculates the observation position based on an observation and then moves there. Both simulation and experimental results are provided to validate the effectiveness of the proposed method.展开更多
This paper describes a system for grasping known objects with unmanned aerial vehicles (UAVs) provided with dual manipulators using an RGB-D camera. Aerial manipulation remains a very challenging task. This paper cove...This paper describes a system for grasping known objects with unmanned aerial vehicles (UAVs) provided with dual manipulators using an RGB-D camera. Aerial manipulation remains a very challenging task. This paper covers three principal aspects for this task: object detection and pose estimation, grasp planning, and in-flight grasp execution. First, an artificial neural network (ANN) is used to obtain clues regarding the object’s position. Next, an alignment algorithm is used to obtain the object’s sixdimensional (6D) pose, which is filtered with an extended Kalman filter. A three-dimensional (3D) model of the object is then used to estimate an arranged list of good grasps for the aerial manipulator. The results from the detection algorithm—that is, the object’s pose—are used to update the trajectories of the arms toward the object. If the target poses are not reachable due to the UAV’s oscillations, the algorithm switches to the next feasible grasp. This paper introduces the overall methodology, and provides the experimental results of both simulation and real experiments for each module, in addition to a video showing the results.展开更多
In order to solve the visual guiding task of initial welding position for arc welding robot, this paper presents a practice prone image based visual servo control strategy without calibration, and we perform validat...In order to solve the visual guiding task of initial welding position for arc welding robot, this paper presents a practice prone image based visual servo control strategy without calibration, and we perform validating experiments on a nine DOF arc welding robot system. Experimental results illustrate presented method has the function to fulfill the task of welding robot initial positioning with certain anti jamming ability. This method provides a basis for guiding welding gun to initial welding pose with real typical seam’s image properties to replace flag block properties, and is a significant exploit to realize visual guiding of initial welding position and seam tracing in robot welding system.展开更多
An improved self-calibrating algorithm for visual servo based on adaptive genetic algorithm is proposed in this paper. Our approach introduces an extension of Mendonca-Cipolla and G. Chesi＇s self-calibration for the ...An improved self-calibrating algorithm for visual servo based on adaptive genetic algorithm is proposed in this paper. Our approach introduces an extension of Mendonca-Cipolla and G. Chesi＇s self-calibration for the positionbased visual servo technique which exploits the singular value property of the essential matrix. Specifically, a suitable dynamic online cost function is generated according to the property of the three singular values. The visual servo process is carried out simultaneous to the dynamic self-calibration, and then the cost function is minimized using the adaptive genetic algorithm instead of the gradient descent method in G. Chesi＇s approach. Moreover, this method overcomes the limitation that the initial parameters must be selected close to the true value, which is not constant in many cases. It is not necessary to know exactly the camera intrinsic parameters when using our approach, instead, coarse coding bounds of the five parameters are enough for the algorithm, which can be done once and for all off-line. Besides, this algorithm does not require knowledge of the 3D model of the object. Simulation experiments are carried out and the results demonstrate that the proposed approach provides a fast convergence speed and robustness against unpredictable perturbations of camera parameters, and it is an effective and efficient visual servo algorithm.展开更多
A robust visual servoing system is investigated on a humanoid robot which grasps a brush in Chinese calligraphy task.The system is implemented based on uncalibrated visual servoing controller utilizing Kalman-Bucy fil...A robust visual servoing system is investigated on a humanoid robot which grasps a brush in Chinese calligraphy task.The system is implemented based on uncalibrated visual servoing controller utilizing Kalman-Bucy filter,with the help of an object detector by continuously adaptive MeanShift(CAMShift) algorithm.Under this control scheme,a humanoid robot can satisfactorily grasp a brush without system modeling.The proposed method is shown to be robust and effective through a Chinese calligraphy task on a NAO robot.展开更多
A dual operational modes mobile robot system based on visual guiding and visual servo control is presented. This system consists of a mobile robot with a two-axis manipulator and a tele-operation station. In the visua...A dual operational modes mobile robot system based on visual guiding and visual servo control is presented. This system consists of a mobile robot with a two-axis manipulator and a tele-operation station. In the visual guiding mode, for the robot works in an open loop visual servo control mode, the manipulating burden of the operator is reduced largely. In the visual servo mode the robot can locate the position of the target assigned by the operator and pick it up by its manipulator. With the help of the operator, the diffieuh problems of finding and handling a target in a complicated environment by the robot can be solved easily.展开更多
In IBVS (image based visual servoing), the error signal in image space should be transformed into the control signal in the input space quickly. To avoid the iterative adjustment and complicated inverse solution of im...In IBVS (image based visual servoing), the error signal in image space should be transformed into the control signal in the input space quickly. To avoid the iterative adjustment and complicated inverse solution of image Jacobian, CMAC (cerebellar model articulation controller) neural network is inserted into visual servo control loop to implement the nonlinear mapping. Two control schemes are used. Simulation results on two schemes are provided, which show a better tracking precision and stability can be achieved using scheme 2.展开更多
Welding path planning can substitute for the manual teaching process of the robot and can promote the autonomous level of the robotic welding. A path planning method by visual servoing was presented, in which the opti...Welding path planning can substitute for the manual teaching process of the robot and can promote the autonomous level of the robotic welding. A path planning method by visual servoing was presented, in which the optimal angle of charge-coupled device (CCD) camera was also planned. Aiming at planning two forms of kinked line seams, obtuse angle seam and right angle seam, a practicable solution was put forward. In this solution, the intersection of two adjacent straight segments is detected in each local seam image, and if intersection is found, the seam errors are calculated using the next straight segment. The experimental results show that kinked line seam can be well planned using this solution.展开更多
基金supported by the National Natural Science Foundation of China(62303350).
文摘This paper presents a method of multicopter intercep-tion control based on visual servo and virtual tube in a cluttered environment.The proposed hybrid heuristic function improves the efficiency of the A*algorithm.The revised objective function makes the virtual tube generating curve not only smooth but also close to the path points generated by the A*algorithm.In six dif-ferent simulation scenarios,the efficiency of the modified A*algorithm is 6.2%higher than that of the traditional A*algorithm.The efficiency of path planning and virtual tube planning is veri-fied by simulations.The effectiveness of interception control is verified by a software-in-loop(SIL)simulation.
基金supported by the China Postdoctoral Science Foundation(No.2022M710956).
文摘In this paper,a visual servoing approach is developed to capture the docking rings of tumbling non-cooperative satellites with a space manipulator.The primary challenge addressed is the potential for the docking ring to leave the monocular camera’s field-of-view as the manipulator approaches the target,due to the ring’s large size.To solve this issue,a two-phase visual servoing scheme combining a monocular camera and a three-line structured light vision system is proposed.In an effort to augment the success rate and safety of capture operations,several constraints are formulated,encompassing manipulator’s kinematics,monocular camera’s field-of-view,obstacle avoidance,structured light’s breakpoints and smooth capture.Subsequently,a nonlinear model predictive controller is proposed to manage these constraints in real-time and regulate the system.System models are established based on image moments and pose for each phase,selecting these features as visual feedback to simplify the formulation of servo constraints and avoid the complex circle-based pose measurement.Furthermore,to ensure unbiased predictions,the model disturbances arising from the imprecise estimation of target motion parameter are observed using an extended Kalman filter,which are then incorporated into the predictive control framework.The simulation results demonstrate the effectiveness of this scheme.
文摘This paper addresses a coordinated control problem for Spacecraft Formation Flying(SFF). The distributed followers are required to track and synchronize with the leader spacecraft.By using the feature points in the two-dimensional image space, an integrated 6-degree-of-freedom dynamic model is formulated for spacecraft relative motion. Without sophisticated threedimensional reconstruction, image features are directly utilized for the controller design. The proposed image-based controller can drive the follower spacecraft in the desired configuration with respect to the leader when the real-time captured images match their reference counterparts. To improve the precision of the formation configuration, the proposed controller employs a coordinated term to reduce the relative distance errors between followers. The uncertainties in the system dynamics are handled by integrating the adaptive technique into the controller, which increases the robustness of the SFF system. The closed-loop system stability is analyzed using the Lyapunov method and algebraic graph theory. A numerical simulation for a given SFF scenario is performed to evaluate the performance of the controller.
基金supported by China Postdoctoral Science Founda-tion (No. 20080441093)Key Laboratory Foundation of Liaoning Province (No. 2008S088).
文摘A visual servoing tracking controller is proposed based on the sliding mode control theory in order to achieve strong robustness against parameter variations and external disturbances. A sliding plane with time delay compensation is presented by the pre-estimate of states. To reduce the chattering of the sliding mode controller, a modified exponential reaching law and hyperbolic tangent function are applied to the design of visual controller and robot joint controller. Simulation results show that the visual servoing control scheme is robust and has good tracking performance.
基金supported in part by the Institute for Guo Qiang of Tsinghua University(2019GQG1023)in part by Graduate Education and Teaching Reform Project of Tsinghua University(202007J007)+1 种基金in part by National Natural Science Foundation of China(U19B2029,62073028,61803222)in part by the Independent Research Program of Tsinghua University(2018Z05JDX002)。
文摘There are two main trends in the development of unmanned aerial vehicle(UAV)technologies:miniaturization and intellectualization,in which realizing object tracking capabilities for a nano-scale UAV is one of the most challenging problems.In this paper,we present a visual object tracking and servoing control system utilizing a tailor-made 38 g nano-scale quadrotor.A lightweight visual module is integrated to enable object tracking capabilities,and a micro positioning deck is mounted to provide accurate pose estimation.In order to be robust against object appearance variations,a novel object tracking algorithm,denoted by RMCTer,is proposed,which integrates a powerful short-term tracking module and an efficient long-term processing module.In particular,the long-term processing module can provide additional object information and modify the short-term tracking model in a timely manner.Furthermore,a positionbased visual servoing control method is proposed for the quadrotor,where an adaptive tracking controller is designed by leveraging backstepping and adaptive techniques.Stable and accurate object tracking is achieved even under disturbances.Experimental results are presented to demonstrate the high accuracy and stability of the whole tracking system.
文摘A new visual servo control scheme for a robotic manipulator is presented in this paper, where a back propagation （BP） neural network is used to make a direct transition from image feature to joint angles without requiring robot kinematics and camera calibration. To speed up the convergence and avoid local minimum of the neural network, this paper uses a genetic algorithm to find the optimal initial weights and thresholds and then uses the BP Mgorithm to train the neural network according to the data given. The proposed method can effectively combine the good global searching ability of genetic algorithms with the accurate local searching feature of BP neural network. The Simulink model for PUMA560 robot visual servo system based on the improved BP neural network is built with the Robotics Toolbox of Matlab. The simulation results indicate that the proposed method can accelerate convergence of the image errors and provide a simple and effective way of robot control.
基金supported by National Natural Science Foundation of China (No.60873032)National High Technology Research and Development Program of China (863 Program) (No.2008AA8041302)
文摘This paper presents an improved support vector machine （SVM） algorithm, which employs invariant moments-based edge extraction to obtain feature attribute. A heuristic attribute reduction algorithm based on rough set＇s discernible matrix is proposed to identify and classify micro-targets. To avoid the complicated calibration for intrinsic parameters of camera, an improved Broyden＇s method is proposed to estimate the image Jacobian matrix which employs Chebyshev polynomial to construct a cost function to approximate the optimization value. Finally, a visual controller is designed for a robotic micromanipulation system. The experiment results of micro-parts assembly show that the proposed methods and algorithms are effective and feasible.
基金supported by Natural Science Foundation of China (No. 61773374)Key Research and Development Program of China (No. 2017YFB1300104)
文摘In this paper, we propose a method to select the observation position in visual servoing with an eye-in-vehicle configuration for the manipulator. In traditional visual servoing, the images taken by the camera may have various problems, including being out of view, large perspective aberrance, improper projection area of object in images and so on. In this paper, we propose a method to determine the observation position to solve these problems. A mobile robot system with pan-tilt camera is designed, which calculates the observation position based on an observation and then moves there. Both simulation and experimental results are provided to validate the effectiveness of the proposed method.
文摘This paper describes a system for grasping known objects with unmanned aerial vehicles (UAVs) provided with dual manipulators using an RGB-D camera. Aerial manipulation remains a very challenging task. This paper covers three principal aspects for this task: object detection and pose estimation, grasp planning, and in-flight grasp execution. First, an artificial neural network (ANN) is used to obtain clues regarding the object’s position. Next, an alignment algorithm is used to obtain the object’s sixdimensional (6D) pose, which is filtered with an extended Kalman filter. A three-dimensional (3D) model of the object is then used to estimate an arranged list of good grasps for the aerial manipulator. The results from the detection algorithm—that is, the object’s pose—are used to update the trajectories of the arms toward the object. If the target poses are not reachable due to the UAV’s oscillations, the algorithm switches to the next feasible grasp. This paper introduces the overall methodology, and provides the experimental results of both simulation and real experiments for each module, in addition to a video showing the results.
基金NationalNatureScienceFoundation (No .5 963 5 160 )
文摘In order to solve the visual guiding task of initial welding position for arc welding robot, this paper presents a practice prone image based visual servo control strategy without calibration, and we perform validating experiments on a nine DOF arc welding robot system. Experimental results illustrate presented method has the function to fulfill the task of welding robot initial positioning with certain anti jamming ability. This method provides a basis for guiding welding gun to initial welding pose with real typical seam’s image properties to replace flag block properties, and is a significant exploit to realize visual guiding of initial welding position and seam tracing in robot welding system.
基金the National Natural Science Foundation of China (No.60675048)Science and Technology Research Project of the Ministry of Education (No.204181).
文摘An improved self-calibrating algorithm for visual servo based on adaptive genetic algorithm is proposed in this paper. Our approach introduces an extension of Mendonca-Cipolla and G. Chesi＇s self-calibration for the positionbased visual servo technique which exploits the singular value property of the essential matrix. Specifically, a suitable dynamic online cost function is generated according to the property of the three singular values. The visual servo process is carried out simultaneous to the dynamic self-calibration, and then the cost function is minimized using the adaptive genetic algorithm instead of the gradient descent method in G. Chesi＇s approach. Moreover, this method overcomes the limitation that the initial parameters must be selected close to the true value, which is not constant in many cases. It is not necessary to know exactly the camera intrinsic parameters when using our approach, instead, coarse coding bounds of the five parameters are enough for the algorithm, which can be done once and for all off-line. Besides, this algorithm does not require knowledge of the 3D model of the object. Simulation experiments are carried out and the results demonstrate that the proposed approach provides a fast convergence speed and robustness against unpredictable perturbations of camera parameters, and it is an effective and efficient visual servo algorithm.
基金Supported by National Natural Science Foundation of China（60874002） Key Project of Shanghai Education Committee （09ZZ158） Leading Academic Discipline Project of Shanghai Municipal Government （S30501）
基金Supported by the National Natural Science Foundation of China(No.61221003)
文摘A robust visual servoing system is investigated on a humanoid robot which grasps a brush in Chinese calligraphy task.The system is implemented based on uncalibrated visual servoing controller utilizing Kalman-Bucy filter,with the help of an object detector by continuously adaptive MeanShift(CAMShift) algorithm.Under this control scheme,a humanoid robot can satisfactorily grasp a brush without system modeling.The proposed method is shown to be robust and effective through a Chinese calligraphy task on a NAO robot.
基金Supported by the National High Technology Research and Development Program of China （No. 2003AA421030） and the National Science Foundation of China （No. 60375026）.
文摘A dual operational modes mobile robot system based on visual guiding and visual servo control is presented. This system consists of a mobile robot with a two-axis manipulator and a tele-operation station. In the visual guiding mode, for the robot works in an open loop visual servo control mode, the manipulating burden of the operator is reduced largely. In the visual servo mode the robot can locate the position of the target assigned by the operator and pick it up by its manipulator. With the help of the operator, the diffieuh problems of finding and handling a target in a complicated environment by the robot can be solved easily.
基金This project is supported by National Natural Science Foundation of China (No.59990470).
文摘In IBVS (image based visual servoing), the error signal in image space should be transformed into the control signal in the input space quickly. To avoid the iterative adjustment and complicated inverse solution of image Jacobian, CMAC (cerebellar model articulation controller) neural network is inserted into visual servo control loop to implement the nonlinear mapping. Two control schemes are used. Simulation results on two schemes are provided, which show a better tracking precision and stability can be achieved using scheme 2.
文摘Welding path planning can substitute for the manual teaching process of the robot and can promote the autonomous level of the robotic welding. A path planning method by visual servoing was presented, in which the optimal angle of charge-coupled device (CCD) camera was also planned. Aiming at planning two forms of kinked line seams, obtuse angle seam and right angle seam, a practicable solution was put forward. In this solution, the intersection of two adjacent straight segments is detected in each local seam image, and if intersection is found, the seam errors are calculated using the next straight segment. The experimental results show that kinked line seam can be well planned using this solution.
