In this paper,we present a robot vision based system for coordinate measurement of feature points on large scale automobile parts.Our system consists of an industrial 6-DOF robot mounted with a CCD camera and a PC.The...In this paper,we present a robot vision based system for coordinate measurement of feature points on large scale automobile parts.Our system consists of an industrial 6-DOF robot mounted with a CCD camera and a PC.The system controls the robot into the area of feature points.The images of measuring feature points are acquired by the camera mounted on the robot.3D positions of the feature points are obtained from a model based pose estimation that applies to the images.The measured positions of all feature points are then transformed to the reference coordinate of feature points whose positions are obtained from the coordinate measuring machine（CMM）.Finally,the point-to-point distances between the measured feature points and the reference feature points are calculated and reported.The results show that the root mean square error（RMSE） of measure values obtained by our system is less than 0.5 mm.Our system is adequate for automobile assembly and can perform faster than conventional methods.展开更多
A method is put forward to realize the recognition and guiding of initial welding position. The weld seams are marked with black lines, which simplify the computational complexity of image processing greatly. A two-ti...A method is put forward to realize the recognition and guiding of initial welding position. The weld seams are marked with black lines, which simplify the computational complexity of image processing greatly. A two-time template matching method has been advanced to search for the target point, which is simple and has higher calculation speed. According to the depth computing principle with the special point matching using binocular stereovision, the initial welding position can be confirmed by calculating the middle point of the perpendicular line of two radials in the space. Taking the welding of propellant fuel container for example, good results are obtained with the algorithms. Finally, similar method for terminating welding position is also advanced.展开更多
Accurate stereo vision calibration is a preliminary step towards high-precision visual posi- tioning of robot. Combining with the characteristics of genetic algorithm （GA） and particle swarm optimization （PSO）, a ...Accurate stereo vision calibration is a preliminary step towards high-precision visual posi- tioning of robot. Combining with the characteristics of genetic algorithm （GA） and particle swarm optimization （PSO）, a three-stage calibration method based on hybrid intelligent optimization is pro- posed for nonlinear camera models in this paper. The motivation is to improve the accuracy of the calibration process. In this approach, the stereo vision calibration is considered as an optimization problem that can be solved by the GA and PSO. The initial linear values can be obtained in the frost stage. Then in the second stage, two cameras＇ parameters are optimized separately. Finally, the in- tegrated optimized calibration of two models is obtained in the third stage. Direct linear transforma- tion （DLT）, GA and PSO are individually used in three stages. It is shown that the results of every stage can correctly find near-optimal solution and it can be used to initialize the next stage. Simula- tion analysis and actual experimental results indicate that this calibration method works more accu- rate and robust in noisy environment compared with traditional calibration methods. The proposed method can fulfill the requirements of robot sophisticated visual operation.展开更多
A new design of vision based soccer robot using the type TMS320F240 of DSPs for MiroSot series is presented. The DSP used enables cost effective control of DC motor, and features fewer external components, lower syste...A new design of vision based soccer robot using the type TMS320F240 of DSPs for MiroSot series is presented. The DSP used enables cost effective control of DC motor, and features fewer external components, lower system cost and better performances than traditional microcontroller. The hardware architecture of robot is firstly presented in detail, and then the software design is briefly discussed. The control structure of decision making subsystem is illuminated also in this paper. The conclusion and prospect are given at last.展开更多
Modeling of welding process by robotic vision is basically a theoretical problem, means mainly on physical problem, and also technological problem. To obtain a good model of welding process by robotic vision, theoreti...Modeling of welding process by robotic vision is basically a theoretical problem, means mainly on physical problem, and also technological problem. To obtain a good model of welding process by robotic vision, theoretical researches are required but also constant experimental researches of several welding processes. Until today researches of welding processes has been based on empirical and detailed experimentation. This paper presents welding process by robotic and automation points of view with application of new technologies. Welding robotic system has been designed with possibility to control and inspect this process. Parameters that should be controlled during the process have been identified to reach desired quality. Figure of control system of welding process by robotic vision is given in this paper.展开更多
The fast paced nature of robotic soccer necessitates real time sensing coupled with quick behaving and decision making. In the field with real robots, it is important to well perceive the location of ball, team robots...The fast paced nature of robotic soccer necessitates real time sensing coupled with quick behaving and decision making. In the field with real robots, it is important to well perceive the location of ball, team robots and opponent robots through the vision system in real time. In this paper the architecture of global vision system of our small size robotic team and the process of object recognition is described. According to the study on color distribution in different color space and quantitative investigation, a method which uses H (Hue) thresholds as the major thresholds to feature exact and recognize object in real time is presented.展开更多
This paper presents some human-inspired strategies for lighting control in a robot system for best scene interpretation,where the main intention is to avoid possible glares or highlights occurring in images. It firstl...This paper presents some human-inspired strategies for lighting control in a robot system for best scene interpretation,where the main intention is to avoid possible glares or highlights occurring in images. It firstly compares the characteristics of human eyes and robot eyes. Then some evaluation criteria are addressed to assess the lighting conditions. A bio-inspired method is adopted to avoid the visual glare which is caused by either direct illumination from large light sources or indirect illumination reflected by smooth surfaces. Appropriate methods are proposed to optimize the pose and optical parameters of the light source and the vision camera.展开更多
One being developed automatic sweep robot, need to estimate if anyone is on a certain range of road ahead then automatically adjust running speed, in order to ensure work efficiency and operation safety. This paper pr...One being developed automatic sweep robot, need to estimate if anyone is on a certain range of road ahead then automatically adjust running speed, in order to ensure work efficiency and operation safety. This paper proposed a method using face detection to predict the data of image sensor. The experimental results show that, the proposed algorithm is practical and reliable, and good outcome have been achieved in the application of instruction robot.展开更多
The task of an intelligent control system design applying soft and quantum computational intelligence technologies discussed.An example of a control object as a mobile robot with redundant robotic manipulator and ster...The task of an intelligent control system design applying soft and quantum computational intelligence technologies discussed.An example of a control object as a mobile robot with redundant robotic manipulator and stereovision introduced.Design of robust knowledge bases is performed using a developed computational intelligence-quantum/soft computing toolkit(QC/SCOptKBTM).The knowledge base self-organization process of fuzzy homogeneous regulators through the application of end-to-end IT of quantum computing described.The coordination control between the mobile robot and redundant manipulator with stereovision based on soft computing described.The general design methodology of a generalizing control unit based on the physical laws of quantum computing(quantum information-thermodynamic trade-off of control quality distribution and knowledge base self-organization goal)is considered.The modernization of the pattern recognition system based on stereo vision technology presented.The effectiveness of the proposed methodology is demonstrated in comparison with the structures of control systems based on soft computing for unforeseen control situations with sensor system.The main objective of this article is to demonstrate the advantages of the approach based on quantum/soft computing.展开更多
A nonlinear visual mapping model is presented to replace the image Jacobian relation for uncalibrated hand/eye coordination. A new visual tracking controller based on artificial neural network is designed. Simulation ...A nonlinear visual mapping model is presented to replace the image Jacobian relation for uncalibrated hand/eye coordination. A new visual tracking controller based on artificial neural network is designed. Simulation results show that this method can drive the static tracking error to zero quickly and keep good robustness and adaptability at the same time. In addition, the algorithm is very easy to be implemented with low computational complexity.展开更多
In this paper, we present a vision guided robotic ball-beam balancing control system, consisting of a robot manipulator (actuator), a ball-beam system (plant) and a machine vision system (feedback). The machine vision...In this paper, we present a vision guided robotic ball-beam balancing control system, consisting of a robot manipulator (actuator), a ball-beam system (plant) and a machine vision system (feedback). The machine vision system feedbacks real-time beam angle and ball position data at a speed of 50 frames per second. Based on feedback data, the end-effector of a robot manipulator is driven to control the ball position by maneuvering of the inclination angle of the ball-beam system. The overall control system is implemented with two FPGA chips, one for machine vision processing, and one for robot joints servo PID controllers as well as ball position PD controller. Experiments are performed on a 5-axes robot manipulator to validate the proposed ball beam balancing control system.展开更多
Robot grasp detection is a fundamental vision task for robots.Deep learning-based methods have shown excellent results in enhancing the grasp detection capabilities for model-free objects in unstructured scenes.Most p...Robot grasp detection is a fundamental vision task for robots.Deep learning-based methods have shown excellent results in enhancing the grasp detection capabilities for model-free objects in unstructured scenes.Most popular approaches explore deep network models and exploit RGB-D images combining colour and depth data to acquire enriched feature expressions.However,current work struggles to achieve a satisfactory balance between the accuracy and real-time performance;the variability of RGB and depth feature distributions receives inadequate attention.The treatment of predicted failure cases is also lacking.We propose an efficient fully convolutional network to predict the pixel-level antipodal grasp parameters in RGB-D images.A structure with hierarchical feature fusion is established using multiple lightweight feature extraction blocks.The feature fusion module with 3D global attention is used to select the complementary information in RGB and depth images suficiently.Additionally,a grasp configuration optimization method based on local grasp path is proposed to cope with the possible failures predicted by the model.Extensive experiments on two public grasping datasets,Cornell and Jacquard,demonstrate that the approach can improve the performance of grasping unknown objects.展开更多
基金wsupported by the Thailand Research Fund and Solimac Automation Co.,Ltd.under the Research and Researchers for Industry Program(RRI)under Grant No.MSD56I0098Office of the Higher Education Commission under the National Research University Project of Thailand
文摘In this paper,we present a robot vision based system for coordinate measurement of feature points on large scale automobile parts.Our system consists of an industrial 6-DOF robot mounted with a CCD camera and a PC.The system controls the robot into the area of feature points.The images of measuring feature points are acquired by the camera mounted on the robot.3D positions of the feature points are obtained from a model based pose estimation that applies to the images.The measured positions of all feature points are then transformed to the reference coordinate of feature points whose positions are obtained from the coordinate measuring machine（CMM）.Finally,the point-to-point distances between the measured feature points and the reference feature points are calculated and reported.The results show that the root mean square error（RMSE） of measure values obtained by our system is less than 0.5 mm.Our system is adequate for automobile assembly and can perform faster than conventional methods.
基金This project is supported by National Natural Science Foundation of China(No.60474036) and Shanghai Municipal Science and Technology CommitteeFoundation, China (No.021111116).
文摘A method is put forward to realize the recognition and guiding of initial welding position. The weld seams are marked with black lines, which simplify the computational complexity of image processing greatly. A two-time template matching method has been advanced to search for the target point, which is simple and has higher calculation speed. According to the depth computing principle with the special point matching using binocular stereovision, the initial welding position can be confirmed by calculating the middle point of the perpendicular line of two radials in the space. Taking the welding of propellant fuel container for example, good results are obtained with the algorithms. Finally, similar method for terminating welding position is also advanced.
文摘Accurate stereo vision calibration is a preliminary step towards high-precision visual posi- tioning of robot. Combining with the characteristics of genetic algorithm （GA） and particle swarm optimization （PSO）, a three-stage calibration method based on hybrid intelligent optimization is pro- posed for nonlinear camera models in this paper. The motivation is to improve the accuracy of the calibration process. In this approach, the stereo vision calibration is considered as an optimization problem that can be solved by the GA and PSO. The initial linear values can be obtained in the frost stage. Then in the second stage, two cameras＇ parameters are optimized separately. Finally, the in- tegrated optimized calibration of two models is obtained in the third stage. Direct linear transforma- tion （DLT）, GA and PSO are individually used in three stages. It is shown that the results of every stage can correctly find near-optimal solution and it can be used to initialize the next stage. Simula- tion analysis and actual experimental results indicate that this calibration method works more accu- rate and robust in noisy environment compared with traditional calibration methods. The proposed method can fulfill the requirements of robot sophisticated visual operation.
文摘A new design of vision based soccer robot using the type TMS320F240 of DSPs for MiroSot series is presented. The DSP used enables cost effective control of DC motor, and features fewer external components, lower system cost and better performances than traditional microcontroller. The hardware architecture of robot is firstly presented in detail, and then the software design is briefly discussed. The control structure of decision making subsystem is illuminated also in this paper. The conclusion and prospect are given at last.
文摘Modeling of welding process by robotic vision is basically a theoretical problem, means mainly on physical problem, and also technological problem. To obtain a good model of welding process by robotic vision, theoretical researches are required but also constant experimental researches of several welding processes. Until today researches of welding processes has been based on empirical and detailed experimentation. This paper presents welding process by robotic and automation points of view with application of new technologies. Welding robotic system has been designed with possibility to control and inspect this process. Parameters that should be controlled during the process have been identified to reach desired quality. Figure of control system of welding process by robotic vision is given in this paper.
文摘The fast paced nature of robotic soccer necessitates real time sensing coupled with quick behaving and decision making. In the field with real robots, it is important to well perceive the location of ball, team robots and opponent robots through the vision system in real time. In this paper the architecture of global vision system of our small size robotic team and the process of object recognition is described. According to the study on color distribution in different color space and quantitative investigation, a method which uses H (Hue) thresholds as the major thresholds to feature exact and recognize object in real time is presented.
基金supported by the National Natural Science Foundation of China and Microsoft Research Asia ( No.NSFC-61173096 No.61103140),NCET+3 种基金the Science and Technology Department of Zhejiang Province ( No.R1110679 No.2010R10006 No.2010C33095 No.Y1090592)
文摘This paper presents some human-inspired strategies for lighting control in a robot system for best scene interpretation,where the main intention is to avoid possible glares or highlights occurring in images. It firstly compares the characteristics of human eyes and robot eyes. Then some evaluation criteria are addressed to assess the lighting conditions. A bio-inspired method is adopted to avoid the visual glare which is caused by either direct illumination from large light sources or indirect illumination reflected by smooth surfaces. Appropriate methods are proposed to optimize the pose and optical parameters of the light source and the vision camera.
文摘One being developed automatic sweep robot, need to estimate if anyone is on a certain range of road ahead then automatically adjust running speed, in order to ensure work efficiency and operation safety. This paper proposed a method using face detection to predict the data of image sensor. The experimental results show that, the proposed algorithm is practical and reliable, and good outcome have been achieved in the application of instruction robot.
文摘The task of an intelligent control system design applying soft and quantum computational intelligence technologies discussed.An example of a control object as a mobile robot with redundant robotic manipulator and stereovision introduced.Design of robust knowledge bases is performed using a developed computational intelligence-quantum/soft computing toolkit(QC/SCOptKBTM).The knowledge base self-organization process of fuzzy homogeneous regulators through the application of end-to-end IT of quantum computing described.The coordination control between the mobile robot and redundant manipulator with stereovision based on soft computing described.The general design methodology of a generalizing control unit based on the physical laws of quantum computing(quantum information-thermodynamic trade-off of control quality distribution and knowledge base self-organization goal)is considered.The modernization of the pattern recognition system based on stereo vision technology presented.The effectiveness of the proposed methodology is demonstrated in comparison with the structures of control systems based on soft computing for unforeseen control situations with sensor system.The main objective of this article is to demonstrate the advantages of the approach based on quantum/soft computing.
基金This project was supported by the National Natural Science Foundation (No. 69875010).
文摘A nonlinear visual mapping model is presented to replace the image Jacobian relation for uncalibrated hand/eye coordination. A new visual tracking controller based on artificial neural network is designed. Simulation results show that this method can drive the static tracking error to zero quickly and keep good robustness and adaptability at the same time. In addition, the algorithm is very easy to be implemented with low computational complexity.
文摘In this paper, we present a vision guided robotic ball-beam balancing control system, consisting of a robot manipulator (actuator), a ball-beam system (plant) and a machine vision system (feedback). The machine vision system feedbacks real-time beam angle and ball position data at a speed of 50 frames per second. Based on feedback data, the end-effector of a robot manipulator is driven to control the ball position by maneuvering of the inclination angle of the ball-beam system. The overall control system is implemented with two FPGA chips, one for machine vision processing, and one for robot joints servo PID controllers as well as ball position PD controller. Experiments are performed on a 5-axes robot manipulator to validate the proposed ball beam balancing control system.
基金the National Natural Science Foundation of China(No.62173230)the Program of Science and Technology Commission of Shanghai Municipality(No.22511101400)。
文摘Robot grasp detection is a fundamental vision task for robots.Deep learning-based methods have shown excellent results in enhancing the grasp detection capabilities for model-free objects in unstructured scenes.Most popular approaches explore deep network models and exploit RGB-D images combining colour and depth data to acquire enriched feature expressions.However,current work struggles to achieve a satisfactory balance between the accuracy and real-time performance;the variability of RGB and depth feature distributions receives inadequate attention.The treatment of predicted failure cases is also lacking.We propose an efficient fully convolutional network to predict the pixel-level antipodal grasp parameters in RGB-D images.A structure with hierarchical feature fusion is established using multiple lightweight feature extraction blocks.The feature fusion module with 3D global attention is used to select the complementary information in RGB and depth images suficiently.Additionally,a grasp configuration optimization method based on local grasp path is proposed to cope with the possible failures predicted by the model.Extensive experiments on two public grasping datasets,Cornell and Jacquard,demonstrate that the approach can improve the performance of grasping unknown objects.
