Conventional robotic manipulators consist of touch and vision sensors in order to pick and place differently shaped objects.Due to the technology development and degrading sensors over a long period,the stereo vision ...Conventional robotic manipulators consist of touch and vision sensors in order to pick and place differently shaped objects.Due to the technology development and degrading sensors over a long period,the stereo vision technique has become a promising alternative.In this study,a low-cost stereo vision-based system,and a gripper to be placed at the end of the robot arm(Fanuc M10 iA/12)are developed for position and orientation estimation of robotic manipulators to pick and place different shaped objects.The stereo vision system developed in this research is used to estimate the position(X,Y,Z),orientation(P_(y))of the Center of Volume of four standard objects(cube,cuboid,cylinder,and sphere)whereas the robot arm with the gripper is used to mechanically pick and place the objects.The stereo vision system is placed on the movable robot arm,and it consists of two cameras to capture two 2D views of a stationary object to derive 3D depth information in 3D space.Moreover,a graphical user interface is developed to train a linear regression model,live predict the coordinates of the objects,and check the accuracy of the predicted data.The graphical user interface can also send predicted coordinates and angles to the gripper and the robot arm.The project is facilitated with python programming language modules and image processing techniques.Identification of the stationary object and estimation of its coordinates is done using image processing techniques.The final product can be identified as a device that converts conventional robot arms without an image processing vision system into a highly precise and accurate robot arm with an image processing vision system.Experimental studies are performed to test the efficiency and effectiveness of used techniques and the gripper prototype.Necessary actions are taken to minimize the errors in position and orientation estimation.In addition,as a future implementation,an embedded system will be developed with a user-friendly software interface to install the vision system into the Fanuc M10 iA/12 robot arm and will upgrade the system to a device that can be implemented with any kind of customized robot arms available in the industry.展开更多
Accurate stereo vision calibration is a preliminary step towards high-precision visual posi- tioning of robot. Combining with the characteristics of genetic algorithm （GA） and particle swarm optimization （PSO）, a ...Accurate stereo vision calibration is a preliminary step towards high-precision visual posi- tioning of robot. Combining with the characteristics of genetic algorithm （GA） and particle swarm optimization （PSO）, a three-stage calibration method based on hybrid intelligent optimization is pro- posed for nonlinear camera models in this paper. The motivation is to improve the accuracy of the calibration process. In this approach, the stereo vision calibration is considered as an optimization problem that can be solved by the GA and PSO. The initial linear values can be obtained in the frost stage. Then in the second stage, two cameras＇ parameters are optimized separately. Finally, the in- tegrated optimized calibration of two models is obtained in the third stage. Direct linear transforma- tion （DLT）, GA and PSO are individually used in three stages. It is shown that the results of every stage can correctly find near-optimal solution and it can be used to initialize the next stage. Simula- tion analysis and actual experimental results indicate that this calibration method works more accu- rate and robust in noisy environment compared with traditional calibration methods. The proposed method can fulfill the requirements of robot sophisticated visual operation.展开更多
This paper presents a method for structured scene modeling using micro stereo vision system with large field of view. The proposed algorithm includes edge detection with Canny detector, line fitting with principle axi...This paper presents a method for structured scene modeling using micro stereo vision system with large field of view. The proposed algorithm includes edge detection with Canny detector, line fitting with principle axis based approach, finding corresponding lines using feature based matching method, and 3D line depth computation.展开更多
Using stereo vision for autonomous mobile robot path-planning is a hot technology.The environment mapping and path-planning algorithms were introduced,and they were applied in the autonomous mobile robot experiment pl...Using stereo vision for autonomous mobile robot path-planning is a hot technology.The environment mapping and path-planning algorithms were introduced,and they were applied in the autonomous mobile robot experiment platform.Through experiments in the robot platform,the effectiveness of these algorithms was verified.展开更多
A supportive mobile robot for assisting the elderly is an emerging requirement mainly in countries like Japan where population ageing become relevant in near future.Falls related injuries are considered as a critical ...A supportive mobile robot for assisting the elderly is an emerging requirement mainly in countries like Japan where population ageing become relevant in near future.Falls related injuries are considered as a critical issue when taking into account the physical health of older people.A personal assistive robot with the capability of picking up and carrying objects for long/short distances can be used to overcome or lessen this problem.Here,we design and introduce a 3 D dynamic simulation of such an assistive robot to perform pick and place of objects through visual recognition.The robot consists of two major components;a robotic arm or manipulator to do the pick and place,and an omnidirectional wheeled robotic platform to support mobility.Both components are designed and operated according to their kinematics and dynamics and the controllers are integrated for the combined performance.The objective was to improve the accuracy of the robot at a considerably high speed.Designed mobile manipulator has been successfully tested and simulated with a stereo vision system to perform object recognition and tracking in a virtual environment resembling aroom of an elderly care.The tracking accuracy of the mobile manipulator at an average speed of 0.5 m/s is 90%and is well suited for the proposed application.展开更多
BACKGROUND Medical robot is a promising surgical tool,but no specific one has been designed for interventional treatment of chronic pain.We developed a computed tomography-image based navigation robot using a new regi...BACKGROUND Medical robot is a promising surgical tool,but no specific one has been designed for interventional treatment of chronic pain.We developed a computed tomography-image based navigation robot using a new registration method with binocular vision.This kind of robot is appropriate for minimal invasive interventional procedures and easy to operate.The feasibility,accuracy and stability of this new robot need to be tested.AIM To assess quantitatively the feasibility,accuracy and stability of the binocularstereo-vision-based navigation robot for minimally invasive interventional procedures.METHODS A box model was designed for assessing the accuracy for targets at different distances.Nine(three sets)lead spheres were embedded in the model as puncture goals.The entry-to-target distances were set 50 mm(short-distance),100 mm(medium-distance)and 150 mm(long-distance).Puncture procedure was repeated three times for each goal.The Euclidian error of each puncture was calculated and statistically analyzed.Three head phantoms were used to explore the clinical feasibility and stability.Three independent operators conducted foramen ovale placement on head phantoms(both sides)by freehand or under the guidance of robot(18 punctures with each method).The operation time,adjustment time and one-time success rate were recorded,and the two guidancemethods were compared.RESULTS On the box model,the mean puncture errors of navigation robot were 1.7±0.9 mm for the short-distance target,2.4±1.0 mm for the moderate target and 4.4±1.4 mm for the long-distance target.On the head phantom,no obvious differences in operation time and adjustment time were found among the three performers(P>0.05).The median adjustment time was significantly less under the guidance of the robot than under free hand.The one-time success rate was significantly higher with the robot(P<0.05).There was no obvious difference in operation time between the two methods(P>0.05).CONCLUSION In the laboratory environment,accuracy of binocular-stereo-vision-based navigation robot is acceptable for target at 100 mm depth or less.Compared with freehand,foramen ovale placement accuracy can be improved with robot guidance.展开更多
The task of an intelligent control system design applying soft and quantum computational intelligence technologies discussed.An example of a control object as a mobile robot with redundant robotic manipulator and ster...The task of an intelligent control system design applying soft and quantum computational intelligence technologies discussed.An example of a control object as a mobile robot with redundant robotic manipulator and stereovision introduced.Design of robust knowledge bases is performed using a developed computational intelligence-quantum/soft computing toolkit(QC/SCOptKBTM).The knowledge base self-organization process of fuzzy homogeneous regulators through the application of end-to-end IT of quantum computing described.The coordination control between the mobile robot and redundant manipulator with stereovision based on soft computing described.The general design methodology of a generalizing control unit based on the physical laws of quantum computing(quantum information-thermodynamic trade-off of control quality distribution and knowledge base self-organization goal)is considered.The modernization of the pattern recognition system based on stereo vision technology presented.The effectiveness of the proposed methodology is demonstrated in comparison with the structures of control systems based on soft computing for unforeseen control situations with sensor system.The main objective of this article is to demonstrate the advantages of the approach based on quantum/soft computing.展开更多
A nonlinear visual mapping model is presented to replace the image Jacobian relation for uncalibrated hand/eye coordination. A new visual tracking controller based on artificial neural network is designed. Simulation ...A nonlinear visual mapping model is presented to replace the image Jacobian relation for uncalibrated hand/eye coordination. A new visual tracking controller based on artificial neural network is designed. Simulation results show that this method can drive the static tracking error to zero quickly and keep good robustness and adaptability at the same time. In addition, the algorithm is very easy to be implemented with low computational complexity.展开更多
A stereo matching algorithm based on the epipolar line constraint is designed to meet the real-time and the accuracy requirements. The algorithm is applied to photodynamic therapy binocular surveillance system for por...A stereo matching algorithm based on the epipolar line constraint is designed to meet the real-time and the accuracy requirements. The algorithm is applied to photodynamic therapy binocular surveillance system for port wine stain （PWS） when it monitors the position of the treatment region. The corner matching based on Hu moments is used to calculate the fundamental matrix of the binocular vision system. Experimental results are in agreement with the theoretical calculation.展开更多
This study addresses a critical problem in the control of process capability as to the positioning accuracy of vision-guided robot. Depending on the calibration accuracy, the process capability varies widely, which re...This study addresses a critical problem in the control of process capability as to the positioning accuracy of vision-guided robot. Depending on the calibration accuracy, the process capability varies widely, which renders the precise control of assembly tasks difficult. Furthermore, some vision sensors prohibit the programming access to rectify the lens distortion effects, which even complicates the problem. This study proposes a method of circumventing the lack of programming access by implementing the lens optical center alignment. Three different calibration methods are compared as to the process capability, and the proposed method shows a very good accuracy. The method can be easily adopted on the shop floor since it doesn’t require a complex setup and mathematical derivation process. Therefore, the practitioners can benefit from the proposed method, while maintaining a high level of precision in terms of robot positioning accuracy.展开更多
文摘Conventional robotic manipulators consist of touch and vision sensors in order to pick and place differently shaped objects.Due to the technology development and degrading sensors over a long period,the stereo vision technique has become a promising alternative.In this study,a low-cost stereo vision-based system,and a gripper to be placed at the end of the robot arm(Fanuc M10 iA/12)are developed for position and orientation estimation of robotic manipulators to pick and place different shaped objects.The stereo vision system developed in this research is used to estimate the position(X,Y,Z),orientation(P_(y))of the Center of Volume of four standard objects(cube,cuboid,cylinder,and sphere)whereas the robot arm with the gripper is used to mechanically pick and place the objects.The stereo vision system is placed on the movable robot arm,and it consists of two cameras to capture two 2D views of a stationary object to derive 3D depth information in 3D space.Moreover,a graphical user interface is developed to train a linear regression model,live predict the coordinates of the objects,and check the accuracy of the predicted data.The graphical user interface can also send predicted coordinates and angles to the gripper and the robot arm.The project is facilitated with python programming language modules and image processing techniques.Identification of the stationary object and estimation of its coordinates is done using image processing techniques.The final product can be identified as a device that converts conventional robot arms without an image processing vision system into a highly precise and accurate robot arm with an image processing vision system.Experimental studies are performed to test the efficiency and effectiveness of used techniques and the gripper prototype.Necessary actions are taken to minimize the errors in position and orientation estimation.In addition,as a future implementation,an embedded system will be developed with a user-friendly software interface to install the vision system into the Fanuc M10 iA/12 robot arm and will upgrade the system to a device that can be implemented with any kind of customized robot arms available in the industry.
文摘Accurate stereo vision calibration is a preliminary step towards high-precision visual posi- tioning of robot. Combining with the characteristics of genetic algorithm （GA） and particle swarm optimization （PSO）, a three-stage calibration method based on hybrid intelligent optimization is pro- posed for nonlinear camera models in this paper. The motivation is to improve the accuracy of the calibration process. In this approach, the stereo vision calibration is considered as an optimization problem that can be solved by the GA and PSO. The initial linear values can be obtained in the frost stage. Then in the second stage, two cameras＇ parameters are optimized separately. Finally, the in- tegrated optimized calibration of two models is obtained in the third stage. Direct linear transforma- tion （DLT）, GA and PSO are individually used in three stages. It is shown that the results of every stage can correctly find near-optimal solution and it can be used to initialize the next stage. Simula- tion analysis and actual experimental results indicate that this calibration method works more accu- rate and robust in noisy environment compared with traditional calibration methods. The proposed method can fulfill the requirements of robot sophisticated visual operation.
文摘This paper presents a method for structured scene modeling using micro stereo vision system with large field of view. The proposed algorithm includes edge detection with Canny detector, line fitting with principle axis based approach, finding corresponding lines using feature based matching method, and 3D line depth computation.
基金Leading Academic Discipline Project of Shanghai Municipal Education Commission,Project Number,J51301Innovation Program of Shanghai Municipal Education Commission,09YZ343
文摘Using stereo vision for autonomous mobile robot path-planning is a hot technology.The environment mapping and path-planning algorithms were introduced,and they were applied in the autonomous mobile robot experiment platform.Through experiments in the robot platform,the effectiveness of these algorithms was verified.
文摘A supportive mobile robot for assisting the elderly is an emerging requirement mainly in countries like Japan where population ageing become relevant in near future.Falls related injuries are considered as a critical issue when taking into account the physical health of older people.A personal assistive robot with the capability of picking up and carrying objects for long/short distances can be used to overcome or lessen this problem.Here,we design and introduce a 3 D dynamic simulation of such an assistive robot to perform pick and place of objects through visual recognition.The robot consists of two major components;a robotic arm or manipulator to do the pick and place,and an omnidirectional wheeled robotic platform to support mobility.Both components are designed and operated according to their kinematics and dynamics and the controllers are integrated for the combined performance.The objective was to improve the accuracy of the robot at a considerably high speed.Designed mobile manipulator has been successfully tested and simulated with a stereo vision system to perform object recognition and tracking in a virtual environment resembling aroom of an elderly care.The tracking accuracy of the mobile manipulator at an average speed of 0.5 m/s is 90%and is well suited for the proposed application.
基金Supported by Jiangsu Provincial Department of Science and Technology,No.BE2017603 and No.BE2017675。
文摘BACKGROUND Medical robot is a promising surgical tool,but no specific one has been designed for interventional treatment of chronic pain.We developed a computed tomography-image based navigation robot using a new registration method with binocular vision.This kind of robot is appropriate for minimal invasive interventional procedures and easy to operate.The feasibility,accuracy and stability of this new robot need to be tested.AIM To assess quantitatively the feasibility,accuracy and stability of the binocularstereo-vision-based navigation robot for minimally invasive interventional procedures.METHODS A box model was designed for assessing the accuracy for targets at different distances.Nine(three sets)lead spheres were embedded in the model as puncture goals.The entry-to-target distances were set 50 mm(short-distance),100 mm(medium-distance)and 150 mm(long-distance).Puncture procedure was repeated three times for each goal.The Euclidian error of each puncture was calculated and statistically analyzed.Three head phantoms were used to explore the clinical feasibility and stability.Three independent operators conducted foramen ovale placement on head phantoms(both sides)by freehand or under the guidance of robot(18 punctures with each method).The operation time,adjustment time and one-time success rate were recorded,and the two guidancemethods were compared.RESULTS On the box model,the mean puncture errors of navigation robot were 1.7±0.9 mm for the short-distance target,2.4±1.0 mm for the moderate target and 4.4±1.4 mm for the long-distance target.On the head phantom,no obvious differences in operation time and adjustment time were found among the three performers(P>0.05).The median adjustment time was significantly less under the guidance of the robot than under free hand.The one-time success rate was significantly higher with the robot(P<0.05).There was no obvious difference in operation time between the two methods(P>0.05).CONCLUSION In the laboratory environment,accuracy of binocular-stereo-vision-based navigation robot is acceptable for target at 100 mm depth or less.Compared with freehand,foramen ovale placement accuracy can be improved with robot guidance.
文摘The task of an intelligent control system design applying soft and quantum computational intelligence technologies discussed.An example of a control object as a mobile robot with redundant robotic manipulator and stereovision introduced.Design of robust knowledge bases is performed using a developed computational intelligence-quantum/soft computing toolkit(QC/SCOptKBTM).The knowledge base self-organization process of fuzzy homogeneous regulators through the application of end-to-end IT of quantum computing described.The coordination control between the mobile robot and redundant manipulator with stereovision based on soft computing described.The general design methodology of a generalizing control unit based on the physical laws of quantum computing(quantum information-thermodynamic trade-off of control quality distribution and knowledge base self-organization goal)is considered.The modernization of the pattern recognition system based on stereo vision technology presented.The effectiveness of the proposed methodology is demonstrated in comparison with the structures of control systems based on soft computing for unforeseen control situations with sensor system.The main objective of this article is to demonstrate the advantages of the approach based on quantum/soft computing.
基金This project was supported by the National Natural Science Foundation (No. 69875010).
文摘A nonlinear visual mapping model is presented to replace the image Jacobian relation for uncalibrated hand/eye coordination. A new visual tracking controller based on artificial neural network is designed. Simulation results show that this method can drive the static tracking error to zero quickly and keep good robustness and adaptability at the same time. In addition, the algorithm is very easy to be implemented with low computational complexity.
基金Supported by the National High Technology Research and Development Program of China("863"Program)(2007AA04Z231)~~
文摘A stereo matching algorithm based on the epipolar line constraint is designed to meet the real-time and the accuracy requirements. The algorithm is applied to photodynamic therapy binocular surveillance system for port wine stain （PWS） when it monitors the position of the treatment region. The corner matching based on Hu moments is used to calculate the fundamental matrix of the binocular vision system. Experimental results are in agreement with the theoretical calculation.
文摘This study addresses a critical problem in the control of process capability as to the positioning accuracy of vision-guided robot. Depending on the calibration accuracy, the process capability varies widely, which renders the precise control of assembly tasks difficult. Furthermore, some vision sensors prohibit the programming access to rectify the lens distortion effects, which even complicates the problem. This study proposes a method of circumventing the lack of programming access by implementing the lens optical center alignment. Three different calibration methods are compared as to the process capability, and the proposed method shows a very good accuracy. The method can be easily adopted on the shop floor since it doesn’t require a complex setup and mathematical derivation process. Therefore, the practitioners can benefit from the proposed method, while maintaining a high level of precision in terms of robot positioning accuracy.
