Generation of a depth-map from 2D video is the kernel of DIBR (Depth Image Based Rendering) in 2D-3D video conversion systems. However it occupies over most of the system resource where the motion search module takes ...Generation of a depth-map from 2D video is the kernel of DIBR (Depth Image Based Rendering) in 2D-3D video conversion systems. However it occupies over most of the system resource where the motion search module takes up 90% time-consuming in typical motion estimation-based depth-map generation algorithms. In order to reduce the computational complexity, in this paper a new fast depth-map generation algorithm based on motion search is developed, in which a fast diamond search algorithm is adopted to decide whether a 16x16 or 4x4 block size is used based on Sobel operator in the motion search module to obtain a sub-depth-map. Then the sub-depth-map will be fused with the sub-depth-maps gotten from depth from color component Cr and depth from linear perspective modules to compensate and refine detail of the depth-map, finally obtain a better depth-map. The simulation results demonstrate that the new approach can greatly reduce over 50% computational complexity compared to other existing methods.展开更多
彩色图像引导的深度图超分辨率重建能有效解决深度相机采集深度图时分辨率低和无效空洞的问题。然而,由于彩色图和深度图的结构不一致性,此类方法容易产生纹理转移伪影。针对此问题,提出了基于结构纹理分解的加权最小二乘(structure tex...彩色图像引导的深度图超分辨率重建能有效解决深度相机采集深度图时分辨率低和无效空洞的问题。然而,由于彩色图和深度图的结构不一致性,此类方法容易产生纹理转移伪影。针对此问题,提出了基于结构纹理分解的加权最小二乘(structure texture decomposition based weighted least square,STDWLS)深度超分辨率重建算法。该方法首先使用原深度图引导彩色图片结构纹理分解得到结构图,随后使用加权最小二乘优化框架来同时建模深度图上采样和空洞填充问题,并对有效深度值和空洞区域分别构建相应的惩罚函数。具体而言,对于有效深度值,该方法结合静态的结构图和动态更新的深度值计算引导权重;对于空洞,该方法仅使用结构图计算引导权重,并加入边缘自适应窗口来防止深度图边缘模糊。实验结果表明,STDWLS算法能够同时完成深度图上采样和空洞填充任务,有效抑制了纹理转移伪影,提高了深度图重建的精确度与表面结构相似度。展开更多
自监督单目深度估计受到了国内外研究人员的广泛关注。现有基于深度学习的自监督单目深度估计方法主要采用编码器-解码器结构。然而,这些方法在编码过程中对输入图像进行下采样操作,导致部分图像信息,尤其是图像的边界信息丢失,进而影...自监督单目深度估计受到了国内外研究人员的广泛关注。现有基于深度学习的自监督单目深度估计方法主要采用编码器-解码器结构。然而,这些方法在编码过程中对输入图像进行下采样操作,导致部分图像信息,尤其是图像的边界信息丢失,进而影响深度图的精度。针对上述问题,提出一种基于拉普拉斯金字塔的自监督单目深度估计方法(Self-supervised Monocular Depth Estimation Based on the Laplace Pyramid,LpDepth)。此方法的核心思想是:首先,使用拉普拉斯残差图丰富编码特征,以弥补在下采样过程中丢失的特征信息;其次,在下采样过程中使用最大池化层突显和放大特征信息,使编码器在特征提取过程中更容易地提取到训练模型所需要的特征信息;最后,使用残差模块解决过拟合问题,提高解码器对特征的利用效率。在KITTI和Make3D等数据集上对所提方法进行了测试,同时将其与现有经典方法进行了比较。实验结果证明了所提方法的有效性。展开更多
文摘Generation of a depth-map from 2D video is the kernel of DIBR (Depth Image Based Rendering) in 2D-3D video conversion systems. However it occupies over most of the system resource where the motion search module takes up 90% time-consuming in typical motion estimation-based depth-map generation algorithms. In order to reduce the computational complexity, in this paper a new fast depth-map generation algorithm based on motion search is developed, in which a fast diamond search algorithm is adopted to decide whether a 16x16 or 4x4 block size is used based on Sobel operator in the motion search module to obtain a sub-depth-map. Then the sub-depth-map will be fused with the sub-depth-maps gotten from depth from color component Cr and depth from linear perspective modules to compensate and refine detail of the depth-map, finally obtain a better depth-map. The simulation results demonstrate that the new approach can greatly reduce over 50% computational complexity compared to other existing methods.
文摘彩色图像引导的深度图超分辨率重建能有效解决深度相机采集深度图时分辨率低和无效空洞的问题。然而,由于彩色图和深度图的结构不一致性,此类方法容易产生纹理转移伪影。针对此问题,提出了基于结构纹理分解的加权最小二乘(structure texture decomposition based weighted least square,STDWLS)深度超分辨率重建算法。该方法首先使用原深度图引导彩色图片结构纹理分解得到结构图,随后使用加权最小二乘优化框架来同时建模深度图上采样和空洞填充问题,并对有效深度值和空洞区域分别构建相应的惩罚函数。具体而言,对于有效深度值,该方法结合静态的结构图和动态更新的深度值计算引导权重;对于空洞,该方法仅使用结构图计算引导权重,并加入边缘自适应窗口来防止深度图边缘模糊。实验结果表明,STDWLS算法能够同时完成深度图上采样和空洞填充任务,有效抑制了纹理转移伪影,提高了深度图重建的精确度与表面结构相似度。
文摘自监督单目深度估计受到了国内外研究人员的广泛关注。现有基于深度学习的自监督单目深度估计方法主要采用编码器-解码器结构。然而,这些方法在编码过程中对输入图像进行下采样操作,导致部分图像信息,尤其是图像的边界信息丢失,进而影响深度图的精度。针对上述问题,提出一种基于拉普拉斯金字塔的自监督单目深度估计方法(Self-supervised Monocular Depth Estimation Based on the Laplace Pyramid,LpDepth)。此方法的核心思想是:首先,使用拉普拉斯残差图丰富编码特征,以弥补在下采样过程中丢失的特征信息;其次,在下采样过程中使用最大池化层突显和放大特征信息,使编码器在特征提取过程中更容易地提取到训练模型所需要的特征信息;最后,使用残差模块解决过拟合问题,提高解码器对特征的利用效率。在KITTI和Make3D等数据集上对所提方法进行了测试,同时将其与现有经典方法进行了比较。实验结果证明了所提方法的有效性。
