We present a method of 3D image mosaicing for real 3D representation of roadside buildings, and implement a Web-based interactive visualization environment for the 3D video mosaics created by 3D image mosaicing. The 3...We present a method of 3D image mosaicing for real 3D representation of roadside buildings, and implement a Web-based interactive visualization environment for the 3D video mosaics created by 3D image mosaicing. The 3D image mo- saicing technique developed in our previous work is a very powerful method for creating textured 3D-GIS data without excessive data processing like the laser or stereo system. For the Web-based open access to the 3D video mosaics, we build an interactive visualization environment using X3D, the emerging standard of Web 3D. We conduct the data preprocessing for 3D video mosaics and the X3D modeling for textured 3D data. The data preprocessing includes the conversion of each frame of 3D video mosaics into concatenated image files that can be hyperlinked on the Web. The X3D modeling handles the representation of concatenated images using necessary X3D nodes. By employing X3D as the data format for 3D image mosaics, the real 3D representation of roadside buildings is extended to the Web and mobile service systems.展开更多
Thanks to the rapid development of naked-eye 3D and wireless communication technology,3D video related applications on mobile devices have attracted a lot of attention.Nevertheless,the time-varying characteristics of ...Thanks to the rapid development of naked-eye 3D and wireless communication technology,3D video related applications on mobile devices have attracted a lot of attention.Nevertheless,the time-varying characteristics of the wireless channel is very challenging for conventional source-channel coding based transmission strategy.Also,the high complexity of source-channel coding based transmission scheme is undesired for low power mobile terminals.An advanced transmission scheme named Softcast was proposed to achieve efficient transmission performance for 2D image/video.Unfortunately,it cannot be directly applied to wireless 3D video transmission with high efficiency.This paper proposes a more efficient soft transmission scheme for 3D video with a graceful quality adaptation within a wide range of channel Signal-to-Noise Ratio(SNR).The proposed method first extends the linear transform to 4 dimensions with additional view dimension to eliminate the view redundancy,and then metadata optimization and chunk interleaving are designed to further improve the transmission performance.Meanwhile,a synthesis distortion based chunk discard strategy is developed to improve the overall 3D video quality under the condition of limited bandwidth.The experimental results demonstrate that the proposed method significantly improves the 3D video transmission performance over the wireless channel for low power and low complexity scenarios.展开更多
Following the success of the audio video standard （AVS） for 2D video coding, in 2008, the China AVS workgroup started developing 3D video （3DV） coding techniques. In this paper, we discuss the background, technica...Following the success of the audio video standard （AVS） for 2D video coding, in 2008, the China AVS workgroup started developing 3D video （3DV） coding techniques. In this paper, we discuss the background, technical features, and applications of AVS 3DV coding technology. We introduce two core techniques used in AVS 3DV coding： inter-view prediction and enhanced stereo packing coding. We elaborate on these techniques, which are used in the AVS real-time 3DV encoder. An application of the AVS 3DV coding system is presented to show the great practical value of this system. Simulation results show that the advanced techniques used in AVS 3DV coding provide remarkable coding gain compared with techniques used in a simulcast scheme.展开更多
2D-to-3D video conversion is a feasible way to generate 3D programs for the current 3DTV industry. However, for large-scale 3D video production, current systems are no longer adequate in terms of the time and labor re...2D-to-3D video conversion is a feasible way to generate 3D programs for the current 3DTV industry. However, for large-scale 3D video production, current systems are no longer adequate in terms of the time and labor required for conversion. In this paper, we introduce a distributed 2D-to-3D video conversion system that includes a 2D-to-3D video conversion module, architecture of the parallel computation on the cloud, and 3D video coding in the system. The system enables cooperation among multiple users in the simultaneous completion of their conversion tasks so that the conversion efficiency is greatly promoted. In the experiments, we evaluate the system based on criteria related to both time consumption and video coding performance.展开更多
In this paper, we summarize 3D perception-oriented algorithms for perceptually driven 3D video coding. Several perceptual ef- fects have been exploited for 2D video viewing; however, this is not yet the case for 3D vi...In this paper, we summarize 3D perception-oriented algorithms for perceptually driven 3D video coding. Several perceptual ef- fects have been exploited for 2D video viewing; however, this is not yet the case for 3D video viewing. 3D video requires depth perception, which implies binocular effects such as con fl icts, fusion, and rivalry. A better understanding of these effects is necessary for 3D perceptual compression, which provides users with a more comfortable visual experience for video that is de- livered over a channel with limited bandwidth. We present state-of-the-art of 3D visual attention models, 3D just-notice- able difference models, and 3D texture-synthesis models that address 3D human vision issues in 3D video coding and trans-mission.展开更多
Recently, 3D display technology, and content creation tools have been undergone rigorous development and as a result they have been widely adopted by home and professional users. 3D digital repositories are increasing...Recently, 3D display technology, and content creation tools have been undergone rigorous development and as a result they have been widely adopted by home and professional users. 3D digital repositories are increasing and becoming available ubiquitously. However, searching and visualizing 3D content remains a great challenge. In this paper, we propose and present the development of a novel approach for creating hypervideos, which ease the 3D content search and retrieval. It is called the dynamic hyperlinker for 3D content search and retrieval process. It advances 3D multimedia navigability and searchability by creating dynamic links for selectable and clickable objects in the video scene whilst the user consumes the 3D video clip. The proposed system involves 3D video processing, such as detecting/tracking clickable objects, annotating objects, and metadata engineering including 3D content descriptive protocol. Such system attracts the attention from both home and professional users and more specifically broadcasters and digital content providers. The experiment is conducted on full parallax holoscopic 3D videos “also known as integral images”.展开更多
Nowadays, the switchable matrix is widely used in China's mine video monitoring technology. This kind of video monitoring ability is weak, easy to produce discrete phenomenon, and the overall quality is poor, the ...Nowadays, the switchable matrix is widely used in China's mine video monitoring technology. This kind of video monitoring ability is weak, easy to produce discrete phenomenon, and the overall quality is poor, the relevance cannot meet the relevant demand standards. Therefore, in order to avoid the real-time monitoring technology of the mine, the relevant designers integrate a 3D video fusion method, which fuses the video picture with the scene model in the mine. On this basis, combined with various kinds of information technology to enhance its own performance, so we developed a real-time monitoring system combined with three-dimensional video, which integrated monitoring and management of the original fragmented lens on the basis of time and space, and dynamically managed its mine content.展开更多
在基于移动边缘计算(mobile edge computing,MEC)的第5代移动通信网络(5th generation mobile communication network,5G)中,针对无线虚拟现实(virtual reality,VR)用户在小小区间频繁切换而降低业务体验质量(quality of experience,QoE...在基于移动边缘计算(mobile edge computing,MEC)的第5代移动通信网络(5th generation mobile communication network,5G)中,针对无线虚拟现实(virtual reality,VR)用户在小小区间频繁切换而降低业务体验质量(quality of experience,QoE)的问题。考虑建立多视角3维(3-dimensional,3D)视频的主动缓存、计算和通信(caching,computing and communication,3C)资源分配数学模型,并采用深度强化学习(deep reinforcement learning,DRL)算法进行求解。将多视角3D视频的主动资源分配系统建模为联合视角选择和小基站3C资源分配的马尔科夫决策过程(Markov decision process,MDP),提出了一种基于深度确定性策略梯度(deep deterministic policy gradient,DDPG)算法来寻找有效解。仿真结果表明,与另外2种算法相比,所提算法可以为无线VR用户在小小区间移动时提供更好的业务体验。展开更多
目的 探讨基于3D打印技术结合单侧双通道内镜(unilateral biportal endoscopy,UBE)手术视频在骨科住院医师规范化培训(以下简称住培)脊柱外科教学中的应用效果。方法 选择2023年1—12月在首都医科大学宣武医院参加骨科住培的100名医师...目的 探讨基于3D打印技术结合单侧双通道内镜(unilateral biportal endoscopy,UBE)手术视频在骨科住院医师规范化培训(以下简称住培)脊柱外科教学中的应用效果。方法 选择2023年1—12月在首都医科大学宣武医院参加骨科住培的100名医师为研究对象,随机分为试验组(n=50)和对照组(n=50),对照组接受传统教学模式教学,试验组接受3D打印技术结合UBE手术视频教学模式。通过考核和满意度调查评价两组教学效果。结果 试验组学生的理论考核成绩、实践成绩、最终成绩均高于对照组(79.4±7.5 vs.73.8±9.2;78.8±7.3 vs. 73.7±8.7;79.1±7.3 vs. 73.8±8.8,P<0.05);试验组的临床教学考核的总优良率高于对照组(80.0%vs.60.0%,P<0.05);在总体教学模式满意度(91.3±3.4 vs. 85.4±2.8)、提升自学能力(92.4±2.5 vs. 89.2±2.2)、加深知识掌握(94.0±1.9 vs. 87.0±3.5)、提升解决问题能力(91.6±3.7 vs. 83.9±2.7)及激发学习兴趣方面(93.0±2.0 vs. 85.8±3.8),试验组均优于对照组,且差异均有统计学意义(P<0.05);两组在讲解满意度的比较(91.3±2.9 vs. 91.2±2.9)差异无统计学意义(P>0.05)。结论 在脊柱外科教学中,3D打印技术结合UBE手术视频教学模式取得良好的教学效果,可有效提升骨科住培教学质量并提升学生满意度。展开更多
We propose a disparity-constrained retargeting method for stereoscopic 3D video, which simultaneously resizes a binocular video to a new aspect ratio and remaps the depth to the perceptual comfort zone. First, we mode...We propose a disparity-constrained retargeting method for stereoscopic 3D video, which simultaneously resizes a binocular video to a new aspect ratio and remaps the depth to the perceptual comfort zone. First, we model distortion energies to prevent important video contents from deforming. Then, to maintain depth mapping stability, we model disparity variation energies to constraint the disparity range both in spatial and temporal domains. The last component of our method is a non-uniform, pixel-wise warp to the target resolution based on these energy models. Using this method, we can process the original stereoscopic video to generate new, high-perceptual-quality versions at different display resolutions. For evaluation, we conduct a user study; we also discuss the performance of our method.展开更多
Several approaches for fast generation of digital holograms of a three-dimensional （3D） object have been discussed. Among them, the novel look-up table （N-LUT） method is analyzed to dramatically reduce the number ...Several approaches for fast generation of digital holograms of a three-dimensional （3D） object have been discussed. Among them, the novel look-up table （N-LUT） method is analyzed to dramatically reduce the number of pre-calculated fringe patterns required for computation of digital holograms of a 3D object by employing a new concept of principal fringe patterns, so that problems of computational complexity and huge memory size of the conventional ray-tracing and look-up table methods have been considerably alleviated. Meanwhile, as the 3D video images have a lot of temporally or spatially redundant data in their inter- and intra-frames, computation time of the 3D video holograms could be also reduced just by removing these redundant data. Thus, a couple of computational methods for generation of 3D video holograms by combined use of the N-LUT method and data compression algorithms are also presented and discussed. Some experimental results finally reveal that by using this approach a great reduction of computation time of 3D video holograms could be achieved.展开更多
文摘We present a method of 3D image mosaicing for real 3D representation of roadside buildings, and implement a Web-based interactive visualization environment for the 3D video mosaics created by 3D image mosaicing. The 3D image mo- saicing technique developed in our previous work is a very powerful method for creating textured 3D-GIS data without excessive data processing like the laser or stereo system. For the Web-based open access to the 3D video mosaics, we build an interactive visualization environment using X3D, the emerging standard of Web 3D. We conduct the data preprocessing for 3D video mosaics and the X3D modeling for textured 3D data. The data preprocessing includes the conversion of each frame of 3D video mosaics into concatenated image files that can be hyperlinked on the Web. The X3D modeling handles the representation of concatenated images using necessary X3D nodes. By employing X3D as the data format for 3D image mosaics, the real 3D representation of roadside buildings is extended to the Web and mobile service systems.
基金supported in part by the National Natural Science Foundation of China under Grant 61501074.
文摘Thanks to the rapid development of naked-eye 3D and wireless communication technology,3D video related applications on mobile devices have attracted a lot of attention.Nevertheless,the time-varying characteristics of the wireless channel is very challenging for conventional source-channel coding based transmission strategy.Also,the high complexity of source-channel coding based transmission scheme is undesired for low power mobile terminals.An advanced transmission scheme named Softcast was proposed to achieve efficient transmission performance for 2D image/video.Unfortunately,it cannot be directly applied to wireless 3D video transmission with high efficiency.This paper proposes a more efficient soft transmission scheme for 3D video with a graceful quality adaptation within a wide range of channel Signal-to-Noise Ratio(SNR).The proposed method first extends the linear transform to 4 dimensions with additional view dimension to eliminate the view redundancy,and then metadata optimization and chunk interleaving are designed to further improve the transmission performance.Meanwhile,a synthesis distortion based chunk discard strategy is developed to improve the overall 3D video quality under the condition of limited bandwidth.The experimental results demonstrate that the proposed method significantly improves the 3D video transmission performance over the wireless channel for low power and low complexity scenarios.
文摘Following the success of the audio video standard （AVS） for 2D video coding, in 2008, the China AVS workgroup started developing 3D video （3DV） coding techniques. In this paper, we discuss the background, technical features, and applications of AVS 3DV coding technology. We introduce two core techniques used in AVS 3DV coding： inter-view prediction and enhanced stereo packing coding. We elaborate on these techniques, which are used in the AVS real-time 3DV encoder. An application of the AVS 3DV coding system is presented to show the great practical value of this system. Simulation results show that the advanced techniques used in AVS 3DV coding provide remarkable coding gain compared with techniques used in a simulcast scheme.
基金supported by the National Key Basic Research Program of China (973 Program) under Grant No. 2009CB320904the National Natural Science Foundation of China under Grants No. 61121002, No. 61231010, 91120004the Key Projects in the National Science and Technology Pillar Program under Grant No. 2011BAH08B03
文摘2D-to-3D video conversion is a feasible way to generate 3D programs for the current 3DTV industry. However, for large-scale 3D video production, current systems are no longer adequate in terms of the time and labor required for conversion. In this paper, we introduce a distributed 2D-to-3D video conversion system that includes a 2D-to-3D video conversion module, architecture of the parallel computation on the cloud, and 3D video coding in the system. The system enables cooperation among multiple users in the simultaneous completion of their conversion tasks so that the conversion efficiency is greatly promoted. In the experiments, we evaluate the system based on criteria related to both time consumption and video coding performance.
文摘In this paper, we summarize 3D perception-oriented algorithms for perceptually driven 3D video coding. Several perceptual ef- fects have been exploited for 2D video viewing; however, this is not yet the case for 3D video viewing. 3D video requires depth perception, which implies binocular effects such as con fl icts, fusion, and rivalry. A better understanding of these effects is necessary for 3D perceptual compression, which provides users with a more comfortable visual experience for video that is de- livered over a channel with limited bandwidth. We present state-of-the-art of 3D visual attention models, 3D just-notice- able difference models, and 3D texture-synthesis models that address 3D human vision issues in 3D video coding and trans-mission.
文摘Recently, 3D display technology, and content creation tools have been undergone rigorous development and as a result they have been widely adopted by home and professional users. 3D digital repositories are increasing and becoming available ubiquitously. However, searching and visualizing 3D content remains a great challenge. In this paper, we propose and present the development of a novel approach for creating hypervideos, which ease the 3D content search and retrieval. It is called the dynamic hyperlinker for 3D content search and retrieval process. It advances 3D multimedia navigability and searchability by creating dynamic links for selectable and clickable objects in the video scene whilst the user consumes the 3D video clip. The proposed system involves 3D video processing, such as detecting/tracking clickable objects, annotating objects, and metadata engineering including 3D content descriptive protocol. Such system attracts the attention from both home and professional users and more specifically broadcasters and digital content providers. The experiment is conducted on full parallax holoscopic 3D videos “also known as integral images”.
文摘Nowadays, the switchable matrix is widely used in China's mine video monitoring technology. This kind of video monitoring ability is weak, easy to produce discrete phenomenon, and the overall quality is poor, the relevance cannot meet the relevant demand standards. Therefore, in order to avoid the real-time monitoring technology of the mine, the relevant designers integrate a 3D video fusion method, which fuses the video picture with the scene model in the mine. On this basis, combined with various kinds of information technology to enhance its own performance, so we developed a real-time monitoring system combined with three-dimensional video, which integrated monitoring and management of the original fragmented lens on the basis of time and space, and dynamically managed its mine content.
文摘在基于移动边缘计算(mobile edge computing,MEC)的第5代移动通信网络(5th generation mobile communication network,5G)中,针对无线虚拟现实(virtual reality,VR)用户在小小区间频繁切换而降低业务体验质量(quality of experience,QoE)的问题。考虑建立多视角3维(3-dimensional,3D)视频的主动缓存、计算和通信(caching,computing and communication,3C)资源分配数学模型,并采用深度强化学习(deep reinforcement learning,DRL)算法进行求解。将多视角3D视频的主动资源分配系统建模为联合视角选择和小基站3C资源分配的马尔科夫决策过程(Markov decision process,MDP),提出了一种基于深度确定性策略梯度(deep deterministic policy gradient,DDPG)算法来寻找有效解。仿真结果表明,与另外2种算法相比,所提算法可以为无线VR用户在小小区间移动时提供更好的业务体验。
文摘目的 探讨基于3D打印技术结合单侧双通道内镜(unilateral biportal endoscopy,UBE)手术视频在骨科住院医师规范化培训(以下简称住培)脊柱外科教学中的应用效果。方法 选择2023年1—12月在首都医科大学宣武医院参加骨科住培的100名医师为研究对象,随机分为试验组(n=50)和对照组(n=50),对照组接受传统教学模式教学,试验组接受3D打印技术结合UBE手术视频教学模式。通过考核和满意度调查评价两组教学效果。结果 试验组学生的理论考核成绩、实践成绩、最终成绩均高于对照组(79.4±7.5 vs.73.8±9.2;78.8±7.3 vs. 73.7±8.7;79.1±7.3 vs. 73.8±8.8,P<0.05);试验组的临床教学考核的总优良率高于对照组(80.0%vs.60.0%,P<0.05);在总体教学模式满意度(91.3±3.4 vs. 85.4±2.8)、提升自学能力(92.4±2.5 vs. 89.2±2.2)、加深知识掌握(94.0±1.9 vs. 87.0±3.5)、提升解决问题能力(91.6±3.7 vs. 83.9±2.7)及激发学习兴趣方面(93.0±2.0 vs. 85.8±3.8),试验组均优于对照组,且差异均有统计学意义(P<0.05);两组在讲解满意度的比较(91.3±2.9 vs. 91.2±2.9)差异无统计学意义(P>0.05)。结论 在脊柱外科教学中,3D打印技术结合UBE手术视频教学模式取得良好的教学效果,可有效提升骨科住培教学质量并提升学生满意度。
基金supported by the National Basic Research Program of China under Grant No. 2011CB302206the National Natural Science Foundation of China under Grant Nos. 61272226 and 61272231Beijing Key Laboratory of Networked Multimedia
文摘We propose a disparity-constrained retargeting method for stereoscopic 3D video, which simultaneously resizes a binocular video to a new aspect ratio and remaps the depth to the perceptual comfort zone. First, we model distortion energies to prevent important video contents from deforming. Then, to maintain depth mapping stability, we model disparity variation energies to constraint the disparity range both in spatial and temporal domains. The last component of our method is a non-uniform, pixel-wise warp to the target resolution based on these energy models. Using this method, we can process the original stereoscopic video to generate new, high-perceptual-quality versions at different display resolutions. For evaluation, we conduct a user study; we also discuss the performance of our method.
基金supported by the MKE (Ministry of Knowledge Economy), Korea, under the ITRC (Informa-tion Technology Research Center)support program su-pervised by the NIPA (National IT Industry Promotion Agency) (NIPA-2009-C1090-0902-0018)
文摘Several approaches for fast generation of digital holograms of a three-dimensional （3D） object have been discussed. Among them, the novel look-up table （N-LUT） method is analyzed to dramatically reduce the number of pre-calculated fringe patterns required for computation of digital holograms of a 3D object by employing a new concept of principal fringe patterns, so that problems of computational complexity and huge memory size of the conventional ray-tracing and look-up table methods have been considerably alleviated. Meanwhile, as the 3D video images have a lot of temporally or spatially redundant data in their inter- and intra-frames, computation time of the 3D video holograms could be also reduced just by removing these redundant data. Thus, a couple of computational methods for generation of 3D video holograms by combined use of the N-LUT method and data compression algorithms are also presented and discussed. Some experimental results finally reveal that by using this approach a great reduction of computation time of 3D video holograms could be achieved.
