In the robotic welding process with thick steel plates,laser vision sensors are widely used to profile the weld seam to implement automatic seam tracking.The weld seam profile extraction(WSPE)result is a crucial step ...In the robotic welding process with thick steel plates,laser vision sensors are widely used to profile the weld seam to implement automatic seam tracking.The weld seam profile extraction(WSPE)result is a crucial step for identifying the feature points of the extracted profile to guide the welding torch in real time.The visual information processing system may collapse when interference data points in the image survive during the phase of feature point identification,which results in low tracking accuracy and poor welding quality.This paper presents a visual attention featurebased method to extract the weld seam profile(WSP)from the strong arc background using clustering results.First,a binary image is obtained through the preprocessing stage.Second,all data points with a gray value 255 are clustered with the nearest neighborhood clustering algorithm.Third,a strategy is developed to discern one cluster belonging to the WSP from the appointed candidate clusters in each loop,and a scheme is proposed to extract the entire WSP using visual continuity.Compared with the previous methods the proposed method in this paper can extract more useful details of the WSP and has better stability in terms of removing the interference data.Considerable WSPE tests with butt joints and T-joints show the anti-interference ability of the proposed method,which contributes to smoothing the welding process and shows its practical value in robotic automated welding with thick steel plates.展开更多
In aerial robots＇ visual navigation, it is essential yet very difficult to detect the attitude and position of the robots operated in real time. By introducing a new parametric model, the problem can be reduced from ...In aerial robots＇ visual navigation, it is essential yet very difficult to detect the attitude and position of the robots operated in real time. By introducing a new parametric model, the problem can be reduced from almost unmanageable to be partly solved, though not fully, as per the requirement. In this parametric approach, a multi-scale least square method is formulated first. By propagating as well as improving the parameters down from layer to layer of the image pyramid, a new global feature line can then be detected to parameterize the attitude of the robots. Furthermore, this approach paves the way for segmenting the image into distinct parts, which can be realized by deploying a Bayesian classifier on the picture cell level. Comparison with the Hough transform based method in terms of robustness and precision shows that this multi-scale least square algorithm is considerably more robust to noises. Some discussions are also given.展开更多
Visual information acquisition is an important component of the AGV robot. The system adopts STM32F4 embedded application of the ARM Cortex-M4 kernel as the main control module,using shift algorithm to finish on a spe...Visual information acquisition is an important component of the AGV robot. The system adopts STM32F4 embedded application of the ARM Cortex-M4 kernel as the main control module,using shift algorithm to finish on a specific color piece of target tracking. For multi-sensor fusion of three methods,quaternion method is used to correct the attitude,the stability of AGV robot visual information acquisition and image clarity are improved.展开更多
Research on intelligent and robotic excavator has become a focus both at home and abroad, and this type of excavator becomes more and more important in application. In this paper, we developed a control system which c...Research on intelligent and robotic excavator has become a focus both at home and abroad, and this type of excavator becomes more and more important in application. In this paper, we developed a control system which can make the intelligent robotic excavator perform excavating operation autonomously. It can recognize the excavating targets by itself, program the operation automatically based on the original parameter, and finish all the tasks. Experimental results indicate the validity in real-time performance and precision of the control system. The intelligent robotic excavator can remarkably ease the labor intensity and enhance the working efficiency.展开更多
To measure the latency between human motion stimulation and stereo image display response in a visual feedback-based minimally invasive surgical(MIS) robotic system,a method was proposed by comparing the orientations ...To measure the latency between human motion stimulation and stereo image display response in a visual feedback-based minimally invasive surgical(MIS) robotic system,a method was proposed by comparing the orientations of input and output events through image-processing technology. This method used a black bar to keep pace with the measured joint rotating at a number of speeds. During tests,an external camera was placed in front of the apparatus with a proper visual field,so that it can simultaneously view orientations of both bars fixed on the corresponding joints. After quantitatively analyzing the accuracy of the proposed measurement method,the method was applied to a visual feedback-based master–slave robotic system with two-degrees-of-freedom. Experimental results show that the latency of the overall system was approximately 250 ms,and the opposite clearance of the measured joint was in the range of 1.7°–1.9°.展开更多
煤矿井下视觉同步定位与地图构建SLAM(Simultaneous Localization and Mapping)应用中,光照变化与低纹理场景严重影响特征点的提取和匹配结果,导致位姿估计失败,影响定位精度。提出一种基于改进定向快速旋转二值描述符ORB(Oriented Fast...煤矿井下视觉同步定位与地图构建SLAM(Simultaneous Localization and Mapping)应用中,光照变化与低纹理场景严重影响特征点的提取和匹配结果,导致位姿估计失败,影响定位精度。提出一种基于改进定向快速旋转二值描述符ORB(Oriented Fast and Rotated Brief)-SLAM3算法的煤矿井下移动机器人双目视觉定位算法SL-SLAM。针对光照变化场景,在前端使用光照稳定性的Super-Point特征点提取网络替换原始ORB特征点提取算法,并提出一种特征点网格限定法,有效剔除无效特征点区域,增加位姿估计稳定性。针对低纹理场景,在前端引入稳定的线段检测器LSD(Line Segment Detector)线特征提取算法,并提出一种点线联合算法,按照特征点网格对线特征进行分组,根据特征点的匹配结果进行线特征匹配,降低线特征匹配复杂度,节约位姿估计时间。构建了点特征和线特征的重投影误差模型,在线特征残差模型中添加角度约束,通过点特征和线特征的位姿增量雅可比矩阵建立点线特征重投影误差统一成本函数。局部建图线程使用ORB-SLAM3经典的局部优化方法调整点、线特征和关键帧位姿,并在后端线程中进行回环修正、子图融合和全局捆绑调整BA(Bundle Adjustment)。在EuRoC数据集上的试验结果表明,SL-SLAM的绝对位姿误差APE(Absolute Pose Error)指标优于其他对比算法,并取得了与真值最接近的轨迹预测结果:均方根误差相较于ORB-SLAM3降低了17.3%。在煤矿井下模拟场景中的试验结果表明,SL-SLAM能适应光照变化和低纹理场景,可以满足煤矿井下移动机器人的定位精度和稳定性要求。展开更多
A new visual servo control scheme for a robotic manipulator is presented in this paper, where a back propagation （BP） neural network is used to make a direct transition from image feature to joint angles without req...A new visual servo control scheme for a robotic manipulator is presented in this paper, where a back propagation （BP） neural network is used to make a direct transition from image feature to joint angles without requiring robot kinematics and camera calibration. To speed up the convergence and avoid local minimum of the neural network, this paper uses a genetic algorithm to find the optimal initial weights and thresholds and then uses the BP Mgorithm to train the neural network according to the data given. The proposed method can effectively combine the good global searching ability of genetic algorithms with the accurate local searching feature of BP neural network. The Simulink model for PUMA560 robot visual servo system based on the improved BP neural network is built with the Robotics Toolbox of Matlab. The simulation results indicate that the proposed method can accelerate convergence of the image errors and provide a simple and effective way of robot control.展开更多
基金This work was supported in part by the Foundation of Guangdong Educational Committee （2014KTSCX191） and the National Natural Science Foundation of China （61201087）.
基金Supported by National Natural Science Foundation of China(Grant Nos.51575349,51665037,51575348)State Key Laboratory of Smart Manufacturing for Special Vehicles and Transmission System(Grant No.GZ2016KF002).
文摘In the robotic welding process with thick steel plates,laser vision sensors are widely used to profile the weld seam to implement automatic seam tracking.The weld seam profile extraction(WSPE)result is a crucial step for identifying the feature points of the extracted profile to guide the welding torch in real time.The visual information processing system may collapse when interference data points in the image survive during the phase of feature point identification,which results in low tracking accuracy and poor welding quality.This paper presents a visual attention featurebased method to extract the weld seam profile(WSP)from the strong arc background using clustering results.First,a binary image is obtained through the preprocessing stage.Second,all data points with a gray value 255 are clustered with the nearest neighborhood clustering algorithm.Third,a strategy is developed to discern one cluster belonging to the WSP from the appointed candidate clusters in each loop,and a scheme is proposed to extract the entire WSP using visual continuity.Compared with the previous methods the proposed method in this paper can extract more useful details of the WSP and has better stability in terms of removing the interference data.Considerable WSPE tests with butt joints and T-joints show the anti-interference ability of the proposed method,which contributes to smoothing the welding process and shows its practical value in robotic automated welding with thick steel plates.
基金Supported by National Natural Science Foundation of China（60874002） Key Project of Shanghai Education Committee （09ZZ158） Leading Academic Discipline Project of Shanghai Municipal Government （S30501）
文摘In aerial robots＇ visual navigation, it is essential yet very difficult to detect the attitude and position of the robots operated in real time. By introducing a new parametric model, the problem can be reduced from almost unmanageable to be partly solved, though not fully, as per the requirement. In this parametric approach, a multi-scale least square method is formulated first. By propagating as well as improving the parameters down from layer to layer of the image pyramid, a new global feature line can then be detected to parameterize the attitude of the robots. Furthermore, this approach paves the way for segmenting the image into distinct parts, which can be realized by deploying a Bayesian classifier on the picture cell level. Comparison with the Hough transform based method in terms of robustness and precision shows that this multi-scale least square algorithm is considerably more robust to noises. Some discussions are also given.
基金supported by the National Key Technology R&D Program(2015BAK06B04)the key technology R&D Program of Tianjin(14ZCZDSF00022,15ZXZNGX00260)
文摘Visual information acquisition is an important component of the AGV robot. The system adopts STM32F4 embedded application of the ARM Cortex-M4 kernel as the main control module,using shift algorithm to finish on a specific color piece of target tracking. For multi-sensor fusion of three methods,quaternion method is used to correct the attitude,the stability of AGV robot visual information acquisition and image clarity are improved.
文摘Research on intelligent and robotic excavator has become a focus both at home and abroad, and this type of excavator becomes more and more important in application. In this paper, we developed a control system which can make the intelligent robotic excavator perform excavating operation autonomously. It can recognize the excavating targets by itself, program the operation automatically based on the original parameter, and finish all the tasks. Experimental results indicate the validity in real-time performance and precision of the control system. The intelligent robotic excavator can remarkably ease the labor intensity and enhance the working efficiency.
基金supported by the International S&T Cooperation Program of China (No. 2014DFA70710) the National Natural Science Foundation of China (No. 51475323)
文摘To measure the latency between human motion stimulation and stereo image display response in a visual feedback-based minimally invasive surgical(MIS) robotic system,a method was proposed by comparing the orientations of input and output events through image-processing technology. This method used a black bar to keep pace with the measured joint rotating at a number of speeds. During tests,an external camera was placed in front of the apparatus with a proper visual field,so that it can simultaneously view orientations of both bars fixed on the corresponding joints. After quantitatively analyzing the accuracy of the proposed measurement method,the method was applied to a visual feedback-based master–slave robotic system with two-degrees-of-freedom. Experimental results show that the latency of the overall system was approximately 250 ms,and the opposite clearance of the measured joint was in the range of 1.7°–1.9°.
文摘煤矿井下视觉同步定位与地图构建SLAM(Simultaneous Localization and Mapping)应用中,光照变化与低纹理场景严重影响特征点的提取和匹配结果,导致位姿估计失败,影响定位精度。提出一种基于改进定向快速旋转二值描述符ORB(Oriented Fast and Rotated Brief)-SLAM3算法的煤矿井下移动机器人双目视觉定位算法SL-SLAM。针对光照变化场景,在前端使用光照稳定性的Super-Point特征点提取网络替换原始ORB特征点提取算法,并提出一种特征点网格限定法,有效剔除无效特征点区域,增加位姿估计稳定性。针对低纹理场景,在前端引入稳定的线段检测器LSD(Line Segment Detector)线特征提取算法,并提出一种点线联合算法,按照特征点网格对线特征进行分组,根据特征点的匹配结果进行线特征匹配,降低线特征匹配复杂度,节约位姿估计时间。构建了点特征和线特征的重投影误差模型,在线特征残差模型中添加角度约束,通过点特征和线特征的位姿增量雅可比矩阵建立点线特征重投影误差统一成本函数。局部建图线程使用ORB-SLAM3经典的局部优化方法调整点、线特征和关键帧位姿,并在后端线程中进行回环修正、子图融合和全局捆绑调整BA(Bundle Adjustment)。在EuRoC数据集上的试验结果表明,SL-SLAM的绝对位姿误差APE(Absolute Pose Error)指标优于其他对比算法,并取得了与真值最接近的轨迹预测结果:均方根误差相较于ORB-SLAM3降低了17.3%。在煤矿井下模拟场景中的试验结果表明,SL-SLAM能适应光照变化和低纹理场景,可以满足煤矿井下移动机器人的定位精度和稳定性要求。
文摘A new visual servo control scheme for a robotic manipulator is presented in this paper, where a back propagation （BP） neural network is used to make a direct transition from image feature to joint angles without requiring robot kinematics and camera calibration. To speed up the convergence and avoid local minimum of the neural network, this paper uses a genetic algorithm to find the optimal initial weights and thresholds and then uses the BP Mgorithm to train the neural network according to the data given. The proposed method can effectively combine the good global searching ability of genetic algorithms with the accurate local searching feature of BP neural network. The Simulink model for PUMA560 robot visual servo system based on the improved BP neural network is built with the Robotics Toolbox of Matlab. The simulation results indicate that the proposed method can accelerate convergence of the image errors and provide a simple and effective way of robot control.
