In three-dimensional television (3DTV), an interactive free viewpoint selection application has received more attention so far. This paper presents a novel method that synthesizes a free-viewpoint based on multiple te...In three-dimensional television (3DTV), an interactive free viewpoint selection application has received more attention so far. This paper presents a novel method that synthesizes a free-viewpoint based on multiple textures and depth maps in multi-view camera configuration. This method solves the cracks and holes problem due to sampling rate by performing an inverse warping to retrieve texture images. This step allows a simple and accurate re-sampling of synthetic pixels. To enforce the spatial consistency of color and remove the pixels wrapped incorrectly because of inaccuracy depth maps, we propose some processing steps. The warped depth and warped texture images are used to classify pixels as stable, unstable and disoccluded pixels. The stable pixels are used to create an initial new view by weighted interpolation. To refine the new view, Graph cuts are used to select the best candidates for each unstable pixel. Finally, the remaining disoccluded regions are filled by our inpainting method based on depth information and texture neighboring pixel values. Our experiment on several multi-view data sets is encouraging in both subjective and objective results. Furthermore, our proposal can flexibly use more than two views in multi-view system to create a new view with higher quality.展开更多
Depth maps are used for synthesis virtual view in free-viewpoint television （FTV） systems. When depth maps are derived using existing depth estimation methods, the depth distortions will cause undesirable artifacts ...Depth maps are used for synthesis virtual view in free-viewpoint television （FTV） systems. When depth maps are derived using existing depth estimation methods, the depth distortions will cause undesirable artifacts in the synthesized views. To solve this problem, a 3D video quality model base depth maps （D-3DV） for virtual view synthesis and depth map coding in the FTV applications is proposed. First, the relationships between distortions in coded depth map and rendered view are derived. Then, a precisely 3DV quality model based depth characteristics is develop for the synthesized virtual views. Finally, based on D-3DV model, a multilateral filtering is applied as a pre-processed filter to reduce rendering artifacts. The experimental results evaluated by objective and subjective methods indicate that the proposed D-3DV model can reduce bit-rate of depth coding and achieve better rendering quality.展开更多
Depth image based rendering （DIBR） is an effective approach for virtual view synthesis in free viewpoint television and 3D video.One of the important steps in DIBR is filling the holes caused by disoeclusion regions...Depth image based rendering （DIBR） is an effective approach for virtual view synthesis in free viewpoint television and 3D video.One of the important steps in DIBR is filling the holes caused by disoeclusion regions and wrong depth values.Most of the existing hole-filling methods work well in areas of low spatial activity but fail to obtain satisfactory results in high spatial activity regions.In this paper,we combine the depth based hole-filling and the adaptive recursive interpolation algorithm which is capable of handling edges passing through the missing areas.Accoring to the experimental results,we confirm that the depth based adaptive recursive interpolation algorithm can provide better rendering quality objectively and subjectively.展开更多
目的基于深度图的绘制(DIBR)是一种新型的虚拟视点生成技术,在诸多方面得到了广泛的应用。然而,该技术还不能满足实时性的绘制需求。为了在保证绘制质量不下降的前提下,尽可能地提高绘制速度,提出了一种高效的3D-Warping(3维坐标变换)...目的基于深度图的绘制(DIBR)是一种新型的虚拟视点生成技术,在诸多方面得到了广泛的应用。然而,该技术还不能满足实时性的绘制需求。为了在保证绘制质量不下降的前提下,尽可能地提高绘制速度,提出了一种高效的3D-Warping(3维坐标变换)算法。方法主要在以下3个方面进行了改进:1)引入了深度—视差映射表技术,避免了重复地进行视差求取操作。2)对深度平坦的像素块进行基于块的3D-Warping,减少了映射的次数。对深度非平坦像素块中的像素点采取传统的基于像素点的3D-Warping,保证了映射的准确性。3)针对两种不同的3D-Warping方式,分别提出了相应的插值算法。在水平方向上,改进的像素插值算法对紧邻插值和Splatting(散射)插值算法进行了折中,只在映射像素点与待插值像素点很近的情况下才进行紧邻插值,否则进行Splatting插值;在深度方向上,它对Z-Buffer(深度缓存)技术进行了改进,舍弃了与前景物体太远的映射像素点,而对其他映射像素点按深度值进行加权操作。结果实验结果表明,与标准绘制方案的整像素精度相比,绘制时间平均节省了72.05%;与标准绘制方案的半像素精度相比,PSNR平均提高了0.355 d B,SSIM平均提高了0.001 15。结论改进算法非常适用于水平设置相机系统的DIBR技术中的整像素精度绘制,对包含大量深度平坦区域的视频序列效果明显,不但能够提高绘制的速度,而且可以有效地改善绘制的客观质量。展开更多
平面相机阵列四参考视点的深度图像绘制(Depth Image Based Rendering,DIBR)方案允许用户全方位身临其境地体验场景,可有效避免虚拟视点图像边界空洞,然而该方案引入了较为显著的伪影、背景渗透等失真.为此,提出一种深度和结构相似性(St...平面相机阵列四参考视点的深度图像绘制(Depth Image Based Rendering,DIBR)方案允许用户全方位身临其境地体验场景,可有效避免虚拟视点图像边界空洞,然而该方案引入了较为显著的伪影、背景渗透等失真.为此,提出一种深度和结构相似性(Structural Similarity, SSIM)引导的四参考视点融合算法.首先,深入分析了针对平面相机阵列的四参考视点DIBR方案中失真产生的原因;然后,利用参考视点与虚拟视点间的相对位置关系进行视野错误排除,并根据恰可察觉失真模型提取融合图像的失真掩膜;最后,利用失真区域各视点的深度信息和SSIM进行自适应视点融合,进而绘制出高质量的虚拟视点图像.实验结果表明,本文算法绘制的虚拟视点图像比标准方案在SSIM和沉浸式视频峰值信噪比方面分别提升了0.001 8和1.46 d B,比文献方法在主观视觉感知方面更接近于真实图像.展开更多
文摘In three-dimensional television (3DTV), an interactive free viewpoint selection application has received more attention so far. This paper presents a novel method that synthesizes a free-viewpoint based on multiple textures and depth maps in multi-view camera configuration. This method solves the cracks and holes problem due to sampling rate by performing an inverse warping to retrieve texture images. This step allows a simple and accurate re-sampling of synthetic pixels. To enforce the spatial consistency of color and remove the pixels wrapped incorrectly because of inaccuracy depth maps, we propose some processing steps. The warped depth and warped texture images are used to classify pixels as stable, unstable and disoccluded pixels. The stable pixels are used to create an initial new view by weighted interpolation. To refine the new view, Graph cuts are used to select the best candidates for each unstable pixel. Finally, the remaining disoccluded regions are filled by our inpainting method based on depth information and texture neighboring pixel values. Our experiment on several multi-view data sets is encouraging in both subjective and objective results. Furthermore, our proposal can flexibly use more than two views in multi-view system to create a new view with higher quality.
基金supported by the National Natural Science Foundation of China(Grant No.60832003)Key Laboratory of Advanced Display and System Application(Shanghai University),Ministry of Education,China(Grant No.P200902)the Key Project of Science and Technology Commission of Shanghai Municipality(Grant No.10510500500)
文摘Depth maps are used for synthesis virtual view in free-viewpoint television （FTV） systems. When depth maps are derived using existing depth estimation methods, the depth distortions will cause undesirable artifacts in the synthesized views. To solve this problem, a 3D video quality model base depth maps （D-3DV） for virtual view synthesis and depth map coding in the FTV applications is proposed. First, the relationships between distortions in coded depth map and rendered view are derived. Then, a precisely 3DV quality model based depth characteristics is develop for the synthesized virtual views. Finally, based on D-3DV model, a multilateral filtering is applied as a pre-processed filter to reduce rendering artifacts. The experimental results evaluated by objective and subjective methods indicate that the proposed D-3DV model can reduce bit-rate of depth coding and achieve better rendering quality.
基金The MSIP(Ministry of Science,ICT & Future Planning),Korea,under the ITRC(Information Technology Research Center)support program(NIPA-2013-H0301-13-2006)supervised by the NIPA(National IT Industry Promotion Agency)
文摘Depth image based rendering （DIBR） is an effective approach for virtual view synthesis in free viewpoint television and 3D video.One of the important steps in DIBR is filling the holes caused by disoeclusion regions and wrong depth values.Most of the existing hole-filling methods work well in areas of low spatial activity but fail to obtain satisfactory results in high spatial activity regions.In this paper,we combine the depth based hole-filling and the adaptive recursive interpolation algorithm which is capable of handling edges passing through the missing areas.Accoring to the experimental results,we confirm that the depth based adaptive recursive interpolation algorithm can provide better rendering quality objectively and subjectively.
文摘目的基于深度图的绘制(DIBR)是一种新型的虚拟视点生成技术,在诸多方面得到了广泛的应用。然而,该技术还不能满足实时性的绘制需求。为了在保证绘制质量不下降的前提下,尽可能地提高绘制速度,提出了一种高效的3D-Warping(3维坐标变换)算法。方法主要在以下3个方面进行了改进:1)引入了深度—视差映射表技术,避免了重复地进行视差求取操作。2)对深度平坦的像素块进行基于块的3D-Warping,减少了映射的次数。对深度非平坦像素块中的像素点采取传统的基于像素点的3D-Warping,保证了映射的准确性。3)针对两种不同的3D-Warping方式,分别提出了相应的插值算法。在水平方向上,改进的像素插值算法对紧邻插值和Splatting(散射)插值算法进行了折中,只在映射像素点与待插值像素点很近的情况下才进行紧邻插值,否则进行Splatting插值;在深度方向上,它对Z-Buffer(深度缓存)技术进行了改进,舍弃了与前景物体太远的映射像素点,而对其他映射像素点按深度值进行加权操作。结果实验结果表明,与标准绘制方案的整像素精度相比,绘制时间平均节省了72.05%;与标准绘制方案的半像素精度相比,PSNR平均提高了0.355 d B,SSIM平均提高了0.001 15。结论改进算法非常适用于水平设置相机系统的DIBR技术中的整像素精度绘制,对包含大量深度平坦区域的视频序列效果明显,不但能够提高绘制的速度,而且可以有效地改善绘制的客观质量。
文摘平面相机阵列四参考视点的深度图像绘制(Depth Image Based Rendering,DIBR)方案允许用户全方位身临其境地体验场景,可有效避免虚拟视点图像边界空洞,然而该方案引入了较为显著的伪影、背景渗透等失真.为此,提出一种深度和结构相似性(Structural Similarity, SSIM)引导的四参考视点融合算法.首先,深入分析了针对平面相机阵列的四参考视点DIBR方案中失真产生的原因;然后,利用参考视点与虚拟视点间的相对位置关系进行视野错误排除,并根据恰可察觉失真模型提取融合图像的失真掩膜;最后,利用失真区域各视点的深度信息和SSIM进行自适应视点融合,进而绘制出高质量的虚拟视点图像.实验结果表明,本文算法绘制的虚拟视点图像比标准方案在SSIM和沉浸式视频峰值信噪比方面分别提升了0.001 8和1.46 d B,比文献方法在主观视觉感知方面更接近于真实图像.
