针对现有SLAM算法在渲染真实感、内存占用和复杂场景适应性方面的不足,提出了一种基于3D Gaussians Splatting的密集SLAM算法——TIGO-SLAM(tensor illumination and Gaussian optimization for indoor SLAM)。该算法集成了基于神经网...针对现有SLAM算法在渲染真实感、内存占用和复杂场景适应性方面的不足,提出了一种基于3D Gaussians Splatting的密集SLAM算法——TIGO-SLAM(tensor illumination and Gaussian optimization for indoor SLAM)。该算法集成了基于神经网络的张量光照模型、改进的高斯遮罩算法以及网格化神经场的几何和颜色属性表示,具体创新包括:a)基于神经网络的张量光照模型,增强镜面反射与漫反射效果,从而提升了渲染真实感;b)通过冗余高斯剔除机制改进高斯遮罩算法,有效降低了内存消耗并提高了实时性;c)结合网格化神经场的几何与颜色属性表示,采用优化的码本存储方式,显著提高了渲染性能和场景重建精度。实验结果表明,TIGO-SLAM在室内场景渲染、内存优化和复杂场景适应性方面均有显著提升,特别是在动态室内环境中的渲染和重建效果表现突出,为SLAM技术在资源受限设备上的应用提供了新的可能。展开更多
With the widespread application of 3D visualization in digital exhibition halls and virtual reality,achieving efficient rendering and high-fidelity presentation has become a key challenge.This study proposes a hybrid ...With the widespread application of 3D visualization in digital exhibition halls and virtual reality,achieving efficient rendering and high-fidelity presentation has become a key challenge.This study proposes a hybrid point cloud generation method that combines traditional sampling with 3D Gaussian splatting,aiming to address the issues of rendering delay and missing details in existing 3D displays.By improving the OBJ model parsing process and incorporating an adaptive area-weighted sampling algorithm,we achieve adaptive point cloud generation based on triangle density.Innovatively,we advance the ellipsoidal parameter estimation process of 3D Gaussian splatting to the point cloud generation stage.By establishing a mathematical relationship between the covariance matrix and local curvature,the generated point cloud naturally exhibits Gaussian distribution characteristics.Experimental results show that,compared to traditional methods,our approach reduces point cloud data by 38% while maintaining equivalent visual quality at a 4096×4096 texture resolution.By introducing mipmap texture optimization strategies and a GPU-accelerated rasterization pipeline,stable rendering at 60 frames per second is achieved in a WebGL environment.Additionally,we quantize and compress the spherical harmonic function parameters specific to 3D Gaussian splatting,reducing network transmission bandwidth to 52% of the original data.This study provides a new technical pathway for fields requiring high-precision display,such as the digitization of cultural heritage.展开更多
This paper proposes a unified 3D Gaussian splatting framework consisting of three key components for motion and defocus blur reconstruction.First,a dual-blur perception module is designed to generate pixel-wise masks ...This paper proposes a unified 3D Gaussian splatting framework consisting of three key components for motion and defocus blur reconstruction.First,a dual-blur perception module is designed to generate pixel-wise masks and predict the types of motion and defocus blur,guiding structural feature extraction.Second,a blur-aware Gaussian splatting integrates blur-aware features into the splatting process for accurate modeling of the global and local scene structure.Third,an Unoptimized Gaussian Ratio(UGR)-opacity joint optimization strategy is proposed to refine under-optimized regions,improving reconstruction accuracy under complex blur conditions.Experiments on a newly constructed motion and defocus blur dataset demonstrate the effectiveness of the proposed method for novel view synthesis.Compared with state-of-the-art methods,our framework achieves improvements of 0.28 dB,2.46%and 39.88%on PSNR,SSIM,and LPIPS,respectively.For deblurring tasks,it achieves improvements of 0.36 dB,3.24%and 28.96%on the same metrics.These results highlight the robustness and effectiveness of this approach.展开更多
Generating and inserting new objects into 3D content is a compelling approach for achieving versatile scene recreation.Existing methods,which rely on SDS optimization or single-view inpainting,often struggle to produc...Generating and inserting new objects into 3D content is a compelling approach for achieving versatile scene recreation.Existing methods,which rely on SDS optimization or single-view inpainting,often struggle to produce high-quality results.To address this,we propose a novel method for object inser-tion in 3D content represented by Gaussian Splatting.Our approach introduces a multi-view diffusion model,dubbed MVInpainter,which is built upon a pre-trained stable video diffusion model to facilitate view-consistent object inpainting.Within MVInpainter,we incorporate a ControlNet-based conditional injection module to enable controlled and more predictable multi-view generation.After generating the multi-view inpainted results,we further propose a mask-aware 3D reconstruction technique to refine Gaussian Splatting reconstruction from these sparse inpainted views.By leveraging these fabricate techniques,our approach yields diverse results,ensures view-consistent and harmonious insertions,and produces better object quality.Extensive experiments demonstrate that our approach outperforms existing methods.展开更多
The emergence of 3D Gaussian splatting(3DGS)has greatly accelerated rendering in novel view synthesis.Unlike neural implicit representations like neural radiance fields(NeRFs)that represent a 3D scene with position an...The emergence of 3D Gaussian splatting(3DGS)has greatly accelerated rendering in novel view synthesis.Unlike neural implicit representations like neural radiance fields(NeRFs)that represent a 3D scene with position and viewpoint-conditioned neural networks,3D Gaussian splatting utilizes a set of Gaussian ellipsoids to model the scene so that efficient rendering can be accomplished by rasterizing Gaussian ellipsoids into images.Apart from fast rendering,the explicit representation of 3D Gaussian splatting also facilitates downstream tasks like dynamic reconstruction,geometry editing,and physical simulation.Considering the rapid changes and growing number of works in this field,we present a literature review of recent 3D Gaussian splatting methods,which can be roughly classified by functionality into 3D reconstruction,3D editing,and other downstream applications.Traditional point-based rendering methods and the rendering formulation of 3D Gaussian splatting are also covered to aid understanding of this technique.This survey aims to help beginners to quickly get started in this field and to provide experienced researchers with a comprehensive overview,aiming to stimulate future development of the 3D Gaussian splatting representation.展开更多
文摘针对现有SLAM算法在渲染真实感、内存占用和复杂场景适应性方面的不足,提出了一种基于3D Gaussians Splatting的密集SLAM算法——TIGO-SLAM(tensor illumination and Gaussian optimization for indoor SLAM)。该算法集成了基于神经网络的张量光照模型、改进的高斯遮罩算法以及网格化神经场的几何和颜色属性表示,具体创新包括:a)基于神经网络的张量光照模型,增强镜面反射与漫反射效果,从而提升了渲染真实感;b)通过冗余高斯剔除机制改进高斯遮罩算法,有效降低了内存消耗并提高了实时性;c)结合网格化神经场的几何与颜色属性表示,采用优化的码本存储方式,显著提高了渲染性能和场景重建精度。实验结果表明,TIGO-SLAM在室内场景渲染、内存优化和复杂场景适应性方面均有显著提升,特别是在动态室内环境中的渲染和重建效果表现突出,为SLAM技术在资源受限设备上的应用提供了新的可能。
文摘With the widespread application of 3D visualization in digital exhibition halls and virtual reality,achieving efficient rendering and high-fidelity presentation has become a key challenge.This study proposes a hybrid point cloud generation method that combines traditional sampling with 3D Gaussian splatting,aiming to address the issues of rendering delay and missing details in existing 3D displays.By improving the OBJ model parsing process and incorporating an adaptive area-weighted sampling algorithm,we achieve adaptive point cloud generation based on triangle density.Innovatively,we advance the ellipsoidal parameter estimation process of 3D Gaussian splatting to the point cloud generation stage.By establishing a mathematical relationship between the covariance matrix and local curvature,the generated point cloud naturally exhibits Gaussian distribution characteristics.Experimental results show that,compared to traditional methods,our approach reduces point cloud data by 38% while maintaining equivalent visual quality at a 4096×4096 texture resolution.By introducing mipmap texture optimization strategies and a GPU-accelerated rasterization pipeline,stable rendering at 60 frames per second is achieved in a WebGL environment.Additionally,we quantize and compress the spherical harmonic function parameters specific to 3D Gaussian splatting,reducing network transmission bandwidth to 52% of the original data.This study provides a new technical pathway for fields requiring high-precision display,such as the digitization of cultural heritage.
基金supported by the National Natural Science Foundation of China(62262036,62362043)the Yunnan Xingdian Talent Support Project(No.CYCX202203008)the Science and Technology Plan Projects of Yunnan Province(No.202502AD080003).
文摘This paper proposes a unified 3D Gaussian splatting framework consisting of three key components for motion and defocus blur reconstruction.First,a dual-blur perception module is designed to generate pixel-wise masks and predict the types of motion and defocus blur,guiding structural feature extraction.Second,a blur-aware Gaussian splatting integrates blur-aware features into the splatting process for accurate modeling of the global and local scene structure.Third,an Unoptimized Gaussian Ratio(UGR)-opacity joint optimization strategy is proposed to refine under-optimized regions,improving reconstruction accuracy under complex blur conditions.Experiments on a newly constructed motion and defocus blur dataset demonstrate the effectiveness of the proposed method for novel view synthesis.Compared with state-of-the-art methods,our framework achieves improvements of 0.28 dB,2.46%and 39.88%on PSNR,SSIM,and LPIPS,respectively.For deblurring tasks,it achieves improvements of 0.36 dB,3.24%and 28.96%on the same metrics.These results highlight the robustness and effectiveness of this approach.
文摘Generating and inserting new objects into 3D content is a compelling approach for achieving versatile scene recreation.Existing methods,which rely on SDS optimization or single-view inpainting,often struggle to produce high-quality results.To address this,we propose a novel method for object inser-tion in 3D content represented by Gaussian Splatting.Our approach introduces a multi-view diffusion model,dubbed MVInpainter,which is built upon a pre-trained stable video diffusion model to facilitate view-consistent object inpainting.Within MVInpainter,we incorporate a ControlNet-based conditional injection module to enable controlled and more predictable multi-view generation.After generating the multi-view inpainted results,we further propose a mask-aware 3D reconstruction technique to refine Gaussian Splatting reconstruction from these sparse inpainted views.By leveraging these fabricate techniques,our approach yields diverse results,ensures view-consistent and harmonious insertions,and produces better object quality.Extensive experiments demonstrate that our approach outperforms existing methods.
基金supported by the National Natural Science Foundation of China(62322210)Beijing Municipal Natural Science Foundation for Distinguished Young Scholars(JQ21013)+1 种基金Beijing Municipal Science and Technology Commission(Z231100005923031)2023 Tencent AI Lab Rhino-Bird Focused Research Program.
文摘The emergence of 3D Gaussian splatting(3DGS)has greatly accelerated rendering in novel view synthesis.Unlike neural implicit representations like neural radiance fields(NeRFs)that represent a 3D scene with position and viewpoint-conditioned neural networks,3D Gaussian splatting utilizes a set of Gaussian ellipsoids to model the scene so that efficient rendering can be accomplished by rasterizing Gaussian ellipsoids into images.Apart from fast rendering,the explicit representation of 3D Gaussian splatting also facilitates downstream tasks like dynamic reconstruction,geometry editing,and physical simulation.Considering the rapid changes and growing number of works in this field,we present a literature review of recent 3D Gaussian splatting methods,which can be roughly classified by functionality into 3D reconstruction,3D editing,and other downstream applications.Traditional point-based rendering methods and the rendering formulation of 3D Gaussian splatting are also covered to aid understanding of this technique.This survey aims to help beginners to quickly get started in this field and to provide experienced researchers with a comprehensive overview,aiming to stimulate future development of the 3D Gaussian splatting representation.
