High performance hardware architecture for depth measurement by using binocular-camera is proposed.In the system,at first,video streams of the target are captured by left and right charge-coupled device(CCD)cameras to...High performance hardware architecture for depth measurement by using binocular-camera is proposed.In the system,at first,video streams of the target are captured by left and right charge-coupled device(CCD)cameras to obtain an image including the target.Then,two different images with two different view points are obtained,and they are used in calculating the position deviation of the image's pixels based on triangular measurement.Finally,the three-dimensional coordinate of the object is reconstructed.All the video data is processed by using field-programmable gate array(FPGA)in real-time.Hardware implementation speeds up the performance and reduces the power,thus,this hardware architecture can be applied in the portable environment.展开更多
传统松科球果采摘面临效率低、风险高和成本不可控等挑战,针对自动化松科球果采摘对果实的实时识别与定位问题,提出改进的YOLOv5s-7.0(You Only Look Once)目标检测模型,基于此模型,构建基于双目深度相机的松科球果检测与定位网络。为...传统松科球果采摘面临效率低、风险高和成本不可控等挑战,针对自动化松科球果采摘对果实的实时识别与定位问题,提出改进的YOLOv5s-7.0(You Only Look Once)目标检测模型,基于此模型,构建基于双目深度相机的松科球果检测与定位网络。为提高目标检测精度及效率,对YOLOv5s模型进行改进,将部分卷积PConv嵌入到模型的颈部网络neck多分枝堆叠结构中,面对松科球果的复杂场景增强对稀疏特征的处理能力,提升鲁棒性,减轻特征信息的冗余。在骨干网络backbone的深层及backbone与neck的连接处嵌入简单注意力机制SimAM,在不引入过多参数的基础上优化模型复杂背景下特征提取能力和信息传递的有效性。为满足高效率检测定位,基于双目深度相机测距原理和改进的YOLOv5s模型搭建目标检测及实时定位代码,通过深度匹配,构建松科球果检测与定位系统。根据构建的大兴安岭樟子松球果与小兴安岭红松球果数据集,改进后YOLOv5s模型目标检测精确率达96.8%,召回率和平均精度分别达94%、96.3%,松科球果检测与定位系统在x轴、y轴、z轴的平均绝对误差分别为0.644、0.620、0.740 cm,顺、侧、逆光照下定位试验成功率93.3%,暗光下定位成功率83.3%,视场角等其他性能符合松科球果采摘需求。研究提出的松科球果检测与定位系统为机械化采摘的实时目标检测与定位问题提供可靠的解决方案。展开更多
Although deep learning methods have been widely applied in slam visual odometry(VO)over the past decade with impressive improvements,the accuracy remains limited in complex dynamic environments.In this paper,a composi...Although deep learning methods have been widely applied in slam visual odometry(VO)over the past decade with impressive improvements,the accuracy remains limited in complex dynamic environments.In this paper,a composite mask-based generative adversarial network(CMGAN)is introduced to predict camera motion and binocular depth maps.Specifically,a perceptual generator is constructed to obtain the corresponding parallax map and optical flow between two neighboring frames.Then,an iterative pose improvement strategy is proposed to improve the accuracy of pose estimation.Finally,a composite mask is embedded in the discriminator to sense structural deformation in the synthesized virtual image,thereby increasing the overall structural constraints of the network model,improving the accuracy of camera pose estimation,and reducing drift issues in the VO.Detailed quantitative and qualitative evaluations on the KITTI dataset show that the proposed framework outperforms existing conventional,supervised learning and unsupervised depth VO methods,providing better results in both pose estimation and depth estimation.展开更多
极线校正是一种针对双目相机原始图像对的投影变换方法,使校正后图像对应的极线位于同一水平线上,消除垂直视差,将立体匹配优化为一维搜索问题。针对现今极线校正的不足,本文提出一种基于双目相机平移矩阵的极线校正方法:首先利用奇异...极线校正是一种针对双目相机原始图像对的投影变换方法,使校正后图像对应的极线位于同一水平线上,消除垂直视差,将立体匹配优化为一维搜索问题。针对现今极线校正的不足,本文提出一种基于双目相机平移矩阵的极线校正方法:首先利用奇异值分解(singular value decomposition,SVD)平移矩阵,求得校正后的新旋转矩阵;其次通过校正前后的图像关系确立一个新相机内参矩阵,完成极线校正。运用本文方法对SYNTIM数据库的不同场景多组双目图像进行验证,实验结果表明平均校正误差在0.6像素内,图像几乎不产生畸变,平均偏斜在2.4°左右,平均运行时间为0.2302 s,该方法具有应用价值,完全满足极线校正的需求,解决了双目相机在立体匹配过程中由于相机的机械偏差而产生的误差和繁琐的计算过程。展开更多
文摘High performance hardware architecture for depth measurement by using binocular-camera is proposed.In the system,at first,video streams of the target are captured by left and right charge-coupled device(CCD)cameras to obtain an image including the target.Then,two different images with two different view points are obtained,and they are used in calculating the position deviation of the image's pixels based on triangular measurement.Finally,the three-dimensional coordinate of the object is reconstructed.All the video data is processed by using field-programmable gate array(FPGA)in real-time.Hardware implementation speeds up the performance and reduces the power,thus,this hardware architecture can be applied in the portable environment.
文摘传统松科球果采摘面临效率低、风险高和成本不可控等挑战,针对自动化松科球果采摘对果实的实时识别与定位问题,提出改进的YOLOv5s-7.0(You Only Look Once)目标检测模型,基于此模型,构建基于双目深度相机的松科球果检测与定位网络。为提高目标检测精度及效率,对YOLOv5s模型进行改进,将部分卷积PConv嵌入到模型的颈部网络neck多分枝堆叠结构中,面对松科球果的复杂场景增强对稀疏特征的处理能力,提升鲁棒性,减轻特征信息的冗余。在骨干网络backbone的深层及backbone与neck的连接处嵌入简单注意力机制SimAM,在不引入过多参数的基础上优化模型复杂背景下特征提取能力和信息传递的有效性。为满足高效率检测定位,基于双目深度相机测距原理和改进的YOLOv5s模型搭建目标检测及实时定位代码,通过深度匹配,构建松科球果检测与定位系统。根据构建的大兴安岭樟子松球果与小兴安岭红松球果数据集,改进后YOLOv5s模型目标检测精确率达96.8%,召回率和平均精度分别达94%、96.3%,松科球果检测与定位系统在x轴、y轴、z轴的平均绝对误差分别为0.644、0.620、0.740 cm,顺、侧、逆光照下定位试验成功率93.3%,暗光下定位成功率83.3%,视场角等其他性能符合松科球果采摘需求。研究提出的松科球果检测与定位系统为机械化采摘的实时目标检测与定位问题提供可靠的解决方案。
基金supported by the Program of Graduate Education and Teaching Reform in Tianjin University of Technology(Nos.YBXM2204 and ZDXM2202)the National Natural Science Foundation of China(Nos.62203331 and 62103299)。
文摘Although deep learning methods have been widely applied in slam visual odometry(VO)over the past decade with impressive improvements,the accuracy remains limited in complex dynamic environments.In this paper,a composite mask-based generative adversarial network(CMGAN)is introduced to predict camera motion and binocular depth maps.Specifically,a perceptual generator is constructed to obtain the corresponding parallax map and optical flow between two neighboring frames.Then,an iterative pose improvement strategy is proposed to improve the accuracy of pose estimation.Finally,a composite mask is embedded in the discriminator to sense structural deformation in the synthesized virtual image,thereby increasing the overall structural constraints of the network model,improving the accuracy of camera pose estimation,and reducing drift issues in the VO.Detailed quantitative and qualitative evaluations on the KITTI dataset show that the proposed framework outperforms existing conventional,supervised learning and unsupervised depth VO methods,providing better results in both pose estimation and depth estimation.
文摘极线校正是一种针对双目相机原始图像对的投影变换方法,使校正后图像对应的极线位于同一水平线上,消除垂直视差,将立体匹配优化为一维搜索问题。针对现今极线校正的不足,本文提出一种基于双目相机平移矩阵的极线校正方法:首先利用奇异值分解(singular value decomposition,SVD)平移矩阵,求得校正后的新旋转矩阵;其次通过校正前后的图像关系确立一个新相机内参矩阵,完成极线校正。运用本文方法对SYNTIM数据库的不同场景多组双目图像进行验证,实验结果表明平均校正误差在0.6像素内,图像几乎不产生畸变,平均偏斜在2.4°左右,平均运行时间为0.2302 s,该方法具有应用价值,完全满足极线校正的需求,解决了双目相机在立体匹配过程中由于相机的机械偏差而产生的误差和繁琐的计算过程。
