针对现有SLAM算法在渲染真实感、内存占用和复杂场景适应性方面的不足,提出了一种基于3D Gaussians Splatting的密集SLAM算法——TIGO-SLAM(tensor illumination and Gaussian optimization for indoor SLAM)。该算法集成了基于神经网...针对现有SLAM算法在渲染真实感、内存占用和复杂场景适应性方面的不足,提出了一种基于3D Gaussians Splatting的密集SLAM算法——TIGO-SLAM(tensor illumination and Gaussian optimization for indoor SLAM)。该算法集成了基于神经网络的张量光照模型、改进的高斯遮罩算法以及网格化神经场的几何和颜色属性表示,具体创新包括:a)基于神经网络的张量光照模型,增强镜面反射与漫反射效果,从而提升了渲染真实感;b)通过冗余高斯剔除机制改进高斯遮罩算法,有效降低了内存消耗并提高了实时性;c)结合网格化神经场的几何与颜色属性表示,采用优化的码本存储方式,显著提高了渲染性能和场景重建精度。实验结果表明,TIGO-SLAM在室内场景渲染、内存优化和复杂场景适应性方面均有显著提升,特别是在动态室内环境中的渲染和重建效果表现突出,为SLAM技术在资源受限设备上的应用提供了新的可能。展开更多
With the widespread application of 3D visualization in digital exhibition halls and virtual reality,achieving efficient rendering and high-fidelity presentation has become a key challenge.This study proposes a hybrid ...With the widespread application of 3D visualization in digital exhibition halls and virtual reality,achieving efficient rendering and high-fidelity presentation has become a key challenge.This study proposes a hybrid point cloud generation method that combines traditional sampling with 3D Gaussian splatting,aiming to address the issues of rendering delay and missing details in existing 3D displays.By improving the OBJ model parsing process and incorporating an adaptive area-weighted sampling algorithm,we achieve adaptive point cloud generation based on triangle density.Innovatively,we advance the ellipsoidal parameter estimation process of 3D Gaussian splatting to the point cloud generation stage.By establishing a mathematical relationship between the covariance matrix and local curvature,the generated point cloud naturally exhibits Gaussian distribution characteristics.Experimental results show that,compared to traditional methods,our approach reduces point cloud data by 38% while maintaining equivalent visual quality at a 4096×4096 texture resolution.By introducing mipmap texture optimization strategies and a GPU-accelerated rasterization pipeline,stable rendering at 60 frames per second is achieved in a WebGL environment.Additionally,we quantize and compress the spherical harmonic function parameters specific to 3D Gaussian splatting,reducing network transmission bandwidth to 52% of the original data.This study provides a new technical pathway for fields requiring high-precision display,such as the digitization of cultural heritage.展开更多
Accurately representing and rendering dynamic scenes over time remains a central challenge in neural rendering and computer graphics.Existing dynamic Gaussian-based methods often suffer from limited temporal consisten...Accurately representing and rendering dynamic scenes over time remains a central challenge in neural rendering and computer graphics.Existing dynamic Gaussian-based methods often suffer from limited temporal consistency,flickering under fast motion,and poor adaptability to non-human structures.To address these issues,we propose DG-4DGS,a deformation-graph-constrained 4D Gaussian splatting framework for temporally stable dynamic rendering.The method anchors all Gaussians in a canonical space and enforces cross-frame geometric alignment through a deformation graph.Based on neighborhood-consistency features,a multi-head residual decoder refines position,rotation/scale,and color attributes to achieve fine-detail fidelity without relying on online densification or pruning.Compared with 4DGS and avatar-based approaches,DG-4DGS achieves higher PSNR(peak signal-to-noise ratio)and SSIM(structural similarity index measure)scores and significantly smaller model size on both the TalkBody4D(human)and Horse(non-human)datasets.It effectively suppresses temporal flickering and cross-frame drift in high-frequency regions such as hair strands,cloth wrinkles,and limb extremities.The framework does not depend on parametric templates,facilitating extension to non-human and complex clothing scenarios,though its performance still depends on deformation-tracking quality and neighborhood topology selection.展开更多
3D Gaussian splatting(3DGS)has gained significant attention for its real-time,photorealistic rendering in novel view synthesis.However,its performance degrades severely when applied to real-world scenes with transient...3D Gaussian splatting(3DGS)has gained significant attention for its real-time,photorealistic rendering in novel view synthesis.However,its performance degrades severely when applied to real-world scenes with transients that break cross-view consistency.Existing methods typically attempt to identify and mask out these transients,but inherent masking errors often leave behind incoherent floating Gaussians,resulting in spotted artifacts that degrade the final scene quality.To address these limitations,we propose CleanSplat,a curriculum structural Gaussian splatting method that purifies the scene Gaussians through curriculum 3DGS optimization for transient removal and structural pruning of spotted artifacts.We introduce a curriculum-guided masking paradigm that generates coarse-to-fine transient masks from multi-scale features.The progressive optimization is driven by modulating the masking supervision based on current training state.To clear the spots,we propose a structure-aware handling strategy that employs a superpoint graph(SPG)partitioning of the Gaussians to perform principled identification and hierarchical pruning.This allows for the filtering of both intra-superpoint outliers and entire spurious superpoints based on local 3D coherence instead of only simple photometric consistency.By integrating curriculum 3DGS optimization and structural pruning,our method effectively separates the transients and purifies the static scene Gaussians.Extensive experiments on challenging datasets demonstrate that CleanSplat significantly outperforms state-of-the-art methods,delivering more detailed and cleaner novel view synthesis.展开更多
The 3D reconstruction and physical simulation of plants in natural scenes are of significant research and practical value in fields such as agronomy,forestry,ecology,and remote sensing.However,mainstream 3D reconstruc...The 3D reconstruction and physical simulation of plants in natural scenes are of significant research and practical value in fields such as agronomy,forestry,ecology,and remote sensing.However,mainstream 3D reconstruction methods generally focus on geometric detail recovery but lack integration with physics-driven approaches,making it challenging to accurately and efficiently simulate the dynamic changes in plant structures.Although the latest physical Gaussian methods can simulate a variety of nonrigid deformations,there is limited consideration of plant-specific structural features,which affects the accuracy of reconstruction and simulation.To address this challenge,an end-to-end woody-plant Gaussian is proposed,which is a framework of high-precision 3D reconstruction and physical simulation for woody plants.This framework begins by fine-tuning a pre-trained plant instance segmentation model tailored for this purpose to reduce environmental noise interference and improve the accuracy of skeleton extraction from point cloud data.It leverages the extracted topology to guide fine-grained hierarchical classification of branches.By segmenting hierarchical radii and cross-sectional proportions,the Gaussian point distribution is constrained,enabling the Gaussian ellipsoids to better align with branch surfaces,thereby enhancing reconstruction details.In the physical simulation stage,the framework incorporates material property variation rules.Using topological guidance,Gaussian ellipsoids are mapped to branch hierarchies,and a cantilever beam physical model predicts Gaussian distributions and covariance matrix parameters.This approach not only improves rendering quality but also enhances the realism of branch-bending simulations.Finally,we evaluate our framework on the photos of real woody plants we took(3D deformable wood plant)and a public dataset(NeRFsynthetic).Compared to existing plant reconstruction methods,woody-plant Gaussian achieves state-of-the-art performance and significantly improves the visual quality of plant physical simulations.展开更多
This paper proposes a unified 3D Gaussian splatting framework consisting of three key components for motion and defocus blur reconstruction.First,a dual-blur perception module is designed to generate pixel-wise masks ...This paper proposes a unified 3D Gaussian splatting framework consisting of three key components for motion and defocus blur reconstruction.First,a dual-blur perception module is designed to generate pixel-wise masks and predict the types of motion and defocus blur,guiding structural feature extraction.Second,a blur-aware Gaussian splatting integrates blur-aware features into the splatting process for accurate modeling of the global and local scene structure.Third,an Unoptimized Gaussian Ratio(UGR)-opacity joint optimization strategy is proposed to refine under-optimized regions,improving reconstruction accuracy under complex blur conditions.Experiments on a newly constructed motion and defocus blur dataset demonstrate the effectiveness of the proposed method for novel view synthesis.Compared with state-of-the-art methods,our framework achieves improvements of 0.28 dB,2.46%and 39.88%on PSNR,SSIM,and LPIPS,respectively.For deblurring tasks,it achieves improvements of 0.36 dB,3.24%and 28.96%on the same metrics.These results highlight the robustness and effectiveness of this approach.展开更多
A hyperspectral image compression framework,Gaussian splatting for hyperspectral images(GS-HSI),is proposed based on2D Gaussian splatting,which simplifies the traditional 3D splatting process and enhances compression ...A hyperspectral image compression framework,Gaussian splatting for hyperspectral images(GS-HSI),is proposed based on2D Gaussian splatting,which simplifies the traditional 3D splatting process and enhances compression efficiency.By improving image representation through a cross-band prior information reuse mechanism,GS-HSI facilitates the efficient transfer of key parameters and incorporates an adaptive resampling module to optimize local structures at low bit rates.Compared to existing methods,GS-HSI reduces training time by a factor of 10,achieving an average peak signal-to-noise ratio(PSNR)improvement of 2 dB.Experiments show that the method balances compression efficiency and image quality.It provides a new approach to hyperspectral image compression.展开更多
Generating and inserting new objects into 3D content is a compelling approach for achieving versatile scene recreation.Existing methods,which rely on SDS optimization or single-view inpainting,often struggle to produc...Generating and inserting new objects into 3D content is a compelling approach for achieving versatile scene recreation.Existing methods,which rely on SDS optimization or single-view inpainting,often struggle to produce high-quality results.To address this,we propose a novel method for object inser-tion in 3D content represented by Gaussian Splatting.Our approach introduces a multi-view diffusion model,dubbed MVInpainter,which is built upon a pre-trained stable video diffusion model to facilitate view-consistent object inpainting.Within MVInpainter,we incorporate a ControlNet-based conditional injection module to enable controlled and more predictable multi-view generation.After generating the multi-view inpainted results,we further propose a mask-aware 3D reconstruction technique to refine Gaussian Splatting reconstruction from these sparse inpainted views.By leveraging these fabricate techniques,our approach yields diverse results,ensures view-consistent and harmonious insertions,and produces better object quality.Extensive experiments demonstrate that our approach outperforms existing methods.展开更多
The emergence of 3D Gaussian splatting(3DGS)has greatly accelerated rendering in novel view synthesis.Unlike neural implicit representations like neural radiance fields(NeRFs)that represent a 3D scene with position an...The emergence of 3D Gaussian splatting(3DGS)has greatly accelerated rendering in novel view synthesis.Unlike neural implicit representations like neural radiance fields(NeRFs)that represent a 3D scene with position and viewpoint-conditioned neural networks,3D Gaussian splatting utilizes a set of Gaussian ellipsoids to model the scene so that efficient rendering can be accomplished by rasterizing Gaussian ellipsoids into images.Apart from fast rendering,the explicit representation of 3D Gaussian splatting also facilitates downstream tasks like dynamic reconstruction,geometry editing,and physical simulation.Considering the rapid changes and growing number of works in this field,we present a literature review of recent 3D Gaussian splatting methods,which can be roughly classified by functionality into 3D reconstruction,3D editing,and other downstream applications.Traditional point-based rendering methods and the rendering formulation of 3D Gaussian splatting are also covered to aid understanding of this technique.This survey aims to help beginners to quickly get started in this field and to provide experienced researchers with a comprehensive overview,aiming to stimulate future development of the 3D Gaussian splatting representation.展开更多
文摘针对现有SLAM算法在渲染真实感、内存占用和复杂场景适应性方面的不足,提出了一种基于3D Gaussians Splatting的密集SLAM算法——TIGO-SLAM(tensor illumination and Gaussian optimization for indoor SLAM)。该算法集成了基于神经网络的张量光照模型、改进的高斯遮罩算法以及网格化神经场的几何和颜色属性表示,具体创新包括:a)基于神经网络的张量光照模型,增强镜面反射与漫反射效果,从而提升了渲染真实感;b)通过冗余高斯剔除机制改进高斯遮罩算法,有效降低了内存消耗并提高了实时性;c)结合网格化神经场的几何与颜色属性表示,采用优化的码本存储方式,显著提高了渲染性能和场景重建精度。实验结果表明,TIGO-SLAM在室内场景渲染、内存优化和复杂场景适应性方面均有显著提升,特别是在动态室内环境中的渲染和重建效果表现突出,为SLAM技术在资源受限设备上的应用提供了新的可能。
文摘With the widespread application of 3D visualization in digital exhibition halls and virtual reality,achieving efficient rendering and high-fidelity presentation has become a key challenge.This study proposes a hybrid point cloud generation method that combines traditional sampling with 3D Gaussian splatting,aiming to address the issues of rendering delay and missing details in existing 3D displays.By improving the OBJ model parsing process and incorporating an adaptive area-weighted sampling algorithm,we achieve adaptive point cloud generation based on triangle density.Innovatively,we advance the ellipsoidal parameter estimation process of 3D Gaussian splatting to the point cloud generation stage.By establishing a mathematical relationship between the covariance matrix and local curvature,the generated point cloud naturally exhibits Gaussian distribution characteristics.Experimental results show that,compared to traditional methods,our approach reduces point cloud data by 38% while maintaining equivalent visual quality at a 4096×4096 texture resolution.By introducing mipmap texture optimization strategies and a GPU-accelerated rasterization pipeline,stable rendering at 60 frames per second is achieved in a WebGL environment.Additionally,we quantize and compress the spherical harmonic function parameters specific to 3D Gaussian splatting,reducing network transmission bandwidth to 52% of the original data.This study provides a new technical pathway for fields requiring high-precision display,such as the digitization of cultural heritage.
文摘Accurately representing and rendering dynamic scenes over time remains a central challenge in neural rendering and computer graphics.Existing dynamic Gaussian-based methods often suffer from limited temporal consistency,flickering under fast motion,and poor adaptability to non-human structures.To address these issues,we propose DG-4DGS,a deformation-graph-constrained 4D Gaussian splatting framework for temporally stable dynamic rendering.The method anchors all Gaussians in a canonical space and enforces cross-frame geometric alignment through a deformation graph.Based on neighborhood-consistency features,a multi-head residual decoder refines position,rotation/scale,and color attributes to achieve fine-detail fidelity without relying on online densification or pruning.Compared with 4DGS and avatar-based approaches,DG-4DGS achieves higher PSNR(peak signal-to-noise ratio)and SSIM(structural similarity index measure)scores and significantly smaller model size on both the TalkBody4D(human)and Horse(non-human)datasets.It effectively suppresses temporal flickering and cross-frame drift in high-frequency regions such as hair strands,cloth wrinkles,and limb extremities.The framework does not depend on parametric templates,facilitating extension to non-human and complex clothing scenarios,though its performance still depends on deformation-tracking quality and neighborhood topology selection.
基金Supported by Science and Technology Project of Hainan Provincial Department of Transportation(Grant No.HNJTT-KXC-2024-3-22-02)the National Natural Science Foundation of China(Grant Nos.62272018,62206184).
文摘3D Gaussian splatting(3DGS)has gained significant attention for its real-time,photorealistic rendering in novel view synthesis.However,its performance degrades severely when applied to real-world scenes with transients that break cross-view consistency.Existing methods typically attempt to identify and mask out these transients,but inherent masking errors often leave behind incoherent floating Gaussians,resulting in spotted artifacts that degrade the final scene quality.To address these limitations,we propose CleanSplat,a curriculum structural Gaussian splatting method that purifies the scene Gaussians through curriculum 3DGS optimization for transient removal and structural pruning of spotted artifacts.We introduce a curriculum-guided masking paradigm that generates coarse-to-fine transient masks from multi-scale features.The progressive optimization is driven by modulating the masking supervision based on current training state.To clear the spots,we propose a structure-aware handling strategy that employs a superpoint graph(SPG)partitioning of the Gaussians to perform principled identification and hierarchical pruning.This allows for the filtering of both intra-superpoint outliers and entire spurious superpoints based on local 3D coherence instead of only simple photometric consistency.By integrating curriculum 3DGS optimization and structural pruning,our method effectively separates the transients and purifies the static scene Gaussians.Extensive experiments on challenging datasets demonstrate that CleanSplat significantly outperforms state-of-the-art methods,delivering more detailed and cleaner novel view synthesis.
基金supported by the National Natural Science Foundations of China(No.32271983)Beijing Natural Science Foundation Joint Fund(No.L241056),Zhejiang public welfare technology research plan/social development project(Nos.LTGY23F020005 and LTGY24F020001)in part by the Wenzhou Business School 2024 Talent launch program(No.RC202401).
文摘The 3D reconstruction and physical simulation of plants in natural scenes are of significant research and practical value in fields such as agronomy,forestry,ecology,and remote sensing.However,mainstream 3D reconstruction methods generally focus on geometric detail recovery but lack integration with physics-driven approaches,making it challenging to accurately and efficiently simulate the dynamic changes in plant structures.Although the latest physical Gaussian methods can simulate a variety of nonrigid deformations,there is limited consideration of plant-specific structural features,which affects the accuracy of reconstruction and simulation.To address this challenge,an end-to-end woody-plant Gaussian is proposed,which is a framework of high-precision 3D reconstruction and physical simulation for woody plants.This framework begins by fine-tuning a pre-trained plant instance segmentation model tailored for this purpose to reduce environmental noise interference and improve the accuracy of skeleton extraction from point cloud data.It leverages the extracted topology to guide fine-grained hierarchical classification of branches.By segmenting hierarchical radii and cross-sectional proportions,the Gaussian point distribution is constrained,enabling the Gaussian ellipsoids to better align with branch surfaces,thereby enhancing reconstruction details.In the physical simulation stage,the framework incorporates material property variation rules.Using topological guidance,Gaussian ellipsoids are mapped to branch hierarchies,and a cantilever beam physical model predicts Gaussian distributions and covariance matrix parameters.This approach not only improves rendering quality but also enhances the realism of branch-bending simulations.Finally,we evaluate our framework on the photos of real woody plants we took(3D deformable wood plant)and a public dataset(NeRFsynthetic).Compared to existing plant reconstruction methods,woody-plant Gaussian achieves state-of-the-art performance and significantly improves the visual quality of plant physical simulations.
基金supported by the National Natural Science Foundation of China(62262036,62362043)the Yunnan Xingdian Talent Support Project(No.CYCX202203008)the Science and Technology Plan Projects of Yunnan Province(No.202502AD080003).
文摘This paper proposes a unified 3D Gaussian splatting framework consisting of three key components for motion and defocus blur reconstruction.First,a dual-blur perception module is designed to generate pixel-wise masks and predict the types of motion and defocus blur,guiding structural feature extraction.Second,a blur-aware Gaussian splatting integrates blur-aware features into the splatting process for accurate modeling of the global and local scene structure.Third,an Unoptimized Gaussian Ratio(UGR)-opacity joint optimization strategy is proposed to refine under-optimized regions,improving reconstruction accuracy under complex blur conditions.Experiments on a newly constructed motion and defocus blur dataset demonstrate the effectiveness of the proposed method for novel view synthesis.Compared with state-of-the-art methods,our framework achieves improvements of 0.28 dB,2.46%and 39.88%on PSNR,SSIM,and LPIPS,respectively.For deblurring tasks,it achieves improvements of 0.36 dB,3.24%and 28.96%on the same metrics.These results highlight the robustness and effectiveness of this approach.
基金supported by the Fundamental Research Funds for Universities of Liaoning Province(No.LJ222410143071)。
文摘A hyperspectral image compression framework,Gaussian splatting for hyperspectral images(GS-HSI),is proposed based on2D Gaussian splatting,which simplifies the traditional 3D splatting process and enhances compression efficiency.By improving image representation through a cross-band prior information reuse mechanism,GS-HSI facilitates the efficient transfer of key parameters and incorporates an adaptive resampling module to optimize local structures at low bit rates.Compared to existing methods,GS-HSI reduces training time by a factor of 10,achieving an average peak signal-to-noise ratio(PSNR)improvement of 2 dB.Experiments show that the method balances compression efficiency and image quality.It provides a new approach to hyperspectral image compression.
文摘Generating and inserting new objects into 3D content is a compelling approach for achieving versatile scene recreation.Existing methods,which rely on SDS optimization or single-view inpainting,often struggle to produce high-quality results.To address this,we propose a novel method for object inser-tion in 3D content represented by Gaussian Splatting.Our approach introduces a multi-view diffusion model,dubbed MVInpainter,which is built upon a pre-trained stable video diffusion model to facilitate view-consistent object inpainting.Within MVInpainter,we incorporate a ControlNet-based conditional injection module to enable controlled and more predictable multi-view generation.After generating the multi-view inpainted results,we further propose a mask-aware 3D reconstruction technique to refine Gaussian Splatting reconstruction from these sparse inpainted views.By leveraging these fabricate techniques,our approach yields diverse results,ensures view-consistent and harmonious insertions,and produces better object quality.Extensive experiments demonstrate that our approach outperforms existing methods.
基金supported by the National Natural Science Foundation of China(62322210)Beijing Municipal Natural Science Foundation for Distinguished Young Scholars(JQ21013)+1 种基金Beijing Municipal Science and Technology Commission(Z231100005923031)2023 Tencent AI Lab Rhino-Bird Focused Research Program.
文摘The emergence of 3D Gaussian splatting(3DGS)has greatly accelerated rendering in novel view synthesis.Unlike neural implicit representations like neural radiance fields(NeRFs)that represent a 3D scene with position and viewpoint-conditioned neural networks,3D Gaussian splatting utilizes a set of Gaussian ellipsoids to model the scene so that efficient rendering can be accomplished by rasterizing Gaussian ellipsoids into images.Apart from fast rendering,the explicit representation of 3D Gaussian splatting also facilitates downstream tasks like dynamic reconstruction,geometry editing,and physical simulation.Considering the rapid changes and growing number of works in this field,we present a literature review of recent 3D Gaussian splatting methods,which can be roughly classified by functionality into 3D reconstruction,3D editing,and other downstream applications.Traditional point-based rendering methods and the rendering formulation of 3D Gaussian splatting are also covered to aid understanding of this technique.This survey aims to help beginners to quickly get started in this field and to provide experienced researchers with a comprehensive overview,aiming to stimulate future development of the 3D Gaussian splatting representation.
