针对现有SLAM算法在渲染真实感、内存占用和复杂场景适应性方面的不足,提出了一种基于3D Gaussians Splatting的密集SLAM算法——TIGO-SLAM(tensor illumination and Gaussian optimization for indoor SLAM)。该算法集成了基于神经网...针对现有SLAM算法在渲染真实感、内存占用和复杂场景适应性方面的不足,提出了一种基于3D Gaussians Splatting的密集SLAM算法——TIGO-SLAM(tensor illumination and Gaussian optimization for indoor SLAM)。该算法集成了基于神经网络的张量光照模型、改进的高斯遮罩算法以及网格化神经场的几何和颜色属性表示,具体创新包括:a)基于神经网络的张量光照模型,增强镜面反射与漫反射效果,从而提升了渲染真实感;b)通过冗余高斯剔除机制改进高斯遮罩算法,有效降低了内存消耗并提高了实时性;c)结合网格化神经场的几何与颜色属性表示,采用优化的码本存储方式,显著提高了渲染性能和场景重建精度。实验结果表明,TIGO-SLAM在室内场景渲染、内存优化和复杂场景适应性方面均有显著提升,特别是在动态室内环境中的渲染和重建效果表现突出,为SLAM技术在资源受限设备上的应用提供了新的可能。展开更多
Observing plants across time and diverse scenes is critical in uncovering plant growth patterns.Classic methods often struggle to observe or measure plants against complex backgrounds and at different growth stages.Th...Observing plants across time and diverse scenes is critical in uncovering plant growth patterns.Classic methods often struggle to observe or measure plants against complex backgrounds and at different growth stages.This highlights the need for a universal approach capable of providing realistic plant visualizations across time and scene.Here,we introduce PlantGaussian,an approach for generating realistic three-dimensional(3D)visualization for plants across time and scenes.It marks one of the first applications of 3D Gaussian splatting techniques in plant science,achieving high-quality visualization across species and growth stages.By integrating the Segment Anything Model(SAM)and tracking algorithms,PlantGaussian overcomes the limitations of classic Gaussian reconstruction techniques in complex planting environments.A new mesh partitioning technique is employed to convert Gaussian rendering results into measurable plant meshes,offering a methodology for accurate 3D plant morphology phenotyping.To support this approach,PlantGaussian dataset is developed,which includes images of four crop species captured under multiple conditions and growth stages.Using only plant image sequences as input,it computes high-fidelity plant visualization models and 3D meshes for 3D plant morphological phenotyping.Visualization results indicate that most plant models achieve a Peak Signal-to-Noise Ratio(PSNR)exceeding 25,outperforming all models including the original 3D Gaussian Splatting and enhanced NeRF.The mesh results indicate an average relative error of 4%between the calculated values and the true measurements.As a generic 3D digital plant model,PlantGaussian will support expansion of plant phenotype databases,ecological research,and remote expert consultations.展开更多
With the widespread application of 3D visualization in digital exhibition halls and virtual reality,achieving efficient rendering and high-fidelity presentation has become a key challenge.This study proposes a hybrid ...With the widespread application of 3D visualization in digital exhibition halls and virtual reality,achieving efficient rendering and high-fidelity presentation has become a key challenge.This study proposes a hybrid point cloud generation method that combines traditional sampling with 3D Gaussian splatting,aiming to address the issues of rendering delay and missing details in existing 3D displays.By improving the OBJ model parsing process and incorporating an adaptive area-weighted sampling algorithm,we achieve adaptive point cloud generation based on triangle density.Innovatively,we advance the ellipsoidal parameter estimation process of 3D Gaussian splatting to the point cloud generation stage.By establishing a mathematical relationship between the covariance matrix and local curvature,the generated point cloud naturally exhibits Gaussian distribution characteristics.Experimental results show that,compared to traditional methods,our approach reduces point cloud data by 38% while maintaining equivalent visual quality at a 4096×4096 texture resolution.By introducing mipmap texture optimization strategies and a GPU-accelerated rasterization pipeline,stable rendering at 60 frames per second is achieved in a WebGL environment.Additionally,we quantize and compress the spherical harmonic function parameters specific to 3D Gaussian splatting,reducing network transmission bandwidth to 52% of the original data.This study provides a new technical pathway for fields requiring high-precision display,such as the digitization of cultural heritage.展开更多
The emergence of 3D Gaussian splatting(3DGS)has greatly accelerated rendering in novel view synthesis.Unlike neural implicit representations like neural radiance fields(NeRFs)that represent a 3D scene with position an...The emergence of 3D Gaussian splatting(3DGS)has greatly accelerated rendering in novel view synthesis.Unlike neural implicit representations like neural radiance fields(NeRFs)that represent a 3D scene with position and viewpoint-conditioned neural networks,3D Gaussian splatting utilizes a set of Gaussian ellipsoids to model the scene so that efficient rendering can be accomplished by rasterizing Gaussian ellipsoids into images.Apart from fast rendering,the explicit representation of 3D Gaussian splatting also facilitates downstream tasks like dynamic reconstruction,geometry editing,and physical simulation.Considering the rapid changes and growing number of works in this field,we present a literature review of recent 3D Gaussian splatting methods,which can be roughly classified by functionality into 3D reconstruction,3D editing,and other downstream applications.Traditional point-based rendering methods and the rendering formulation of 3D Gaussian splatting are also covered to aid understanding of this technique.This survey aims to help beginners to quickly get started in this field and to provide experienced researchers with a comprehensive overview,aiming to stimulate future development of the 3D Gaussian splatting representation.展开更多
This paper proposes a unified 3D Gaussian splatting framework consisting of three key components for motion and defocus blur reconstruction.First,a dual-blur perception module is designed to generate pixel-wise masks ...This paper proposes a unified 3D Gaussian splatting framework consisting of three key components for motion and defocus blur reconstruction.First,a dual-blur perception module is designed to generate pixel-wise masks and predict the types of motion and defocus blur,guiding structural feature extraction.Second,a blur-aware Gaussian splatting integrates blur-aware features into the splatting process for accurate modeling of the global and local scene structure.Third,an Unoptimized Gaussian Ratio(UGR)-opacity joint optimization strategy is proposed to refine under-optimized regions,improving reconstruction accuracy under complex blur conditions.Experiments on a newly constructed motion and defocus blur dataset demonstrate the effectiveness of the proposed method for novel view synthesis.Compared with state-of-the-art methods,our framework achieves improvements of 0.28 dB,2.46%and 39.88%on PSNR,SSIM,and LPIPS,respectively.For deblurring tasks,it achieves improvements of 0.36 dB,3.24%and 28.96%on the same metrics.These results highlight the robustness and effectiveness of this approach.展开更多
针对现有的基于机载LiDAR数据的滤波算法未能充分利用数据提供的所有信息及其所采用的数据结构表达复杂、存在信息损失等缺陷,提出了一种灰度体素结构分割模型下的机载LiDAR 3D滤波算法。算法首先以综合利用机载LiDAR数据的高程及强度...针对现有的基于机载LiDAR数据的滤波算法未能充分利用数据提供的所有信息及其所采用的数据结构表达复杂、存在信息损失等缺陷,提出了一种灰度体素结构分割模型下的机载LiDAR 3D滤波算法。算法首先以综合利用机载LiDAR数据的高程及强度信息为目的将点云数据规则化为灰度(体素内激光点的平均强度的离散化表示)体素结构,然后基于各体素间的空间连通性和灰度相似性准则,将灰度体素结构分割并标记为若干个3D连通区域,最后依据地面与其它目标的高差特性提取与其对应的3D连通区域。算法优势在于:基于体素结构设计,为3D滤波算法;综合利用了地面目标的几何及辐射特征,对比传统滤波算法可应用于更复杂的场景;滤波结果为3D地面体素形式,可直接用于创建地面3D模型。实验采用国际摄影测量与遥感协会(International Society for Photogrammetry and Remote Sensing,ISPRS)提供的不同密度的机载LiDAR基准测试数据测试了邻域尺度参数的敏感性及提出的算法的有效性,并和其他经典滤波算法做对比。定量评价的结果表明,51邻域为最佳空间邻域尺度;点云密度为0.67点/m2的数据集1的滤波平均完整率、正确率及质量分别为0.9611、0.9248及0.8934;点云密度为4点/m2的数据集2的滤波平均完整率、正确率及质量分别为0.8490、0.8531及0.7404;对比其全经典滤波算法本文算法在高密度点云数据滤波时表现更佳。展开更多
文摘针对现有SLAM算法在渲染真实感、内存占用和复杂场景适应性方面的不足,提出了一种基于3D Gaussians Splatting的密集SLAM算法——TIGO-SLAM(tensor illumination and Gaussian optimization for indoor SLAM)。该算法集成了基于神经网络的张量光照模型、改进的高斯遮罩算法以及网格化神经场的几何和颜色属性表示,具体创新包括:a)基于神经网络的张量光照模型,增强镜面反射与漫反射效果,从而提升了渲染真实感;b)通过冗余高斯剔除机制改进高斯遮罩算法,有效降低了内存消耗并提高了实时性;c)结合网格化神经场的几何与颜色属性表示,采用优化的码本存储方式,显著提高了渲染性能和场景重建精度。实验结果表明,TIGO-SLAM在室内场景渲染、内存优化和复杂场景适应性方面均有显著提升,特别是在动态室内环境中的渲染和重建效果表现突出,为SLAM技术在资源受限设备上的应用提供了新的可能。
基金supported by the Central Government’s Guidance Fund for Local Science and Technology Development(2024ZY-CGZY-19)。
文摘Observing plants across time and diverse scenes is critical in uncovering plant growth patterns.Classic methods often struggle to observe or measure plants against complex backgrounds and at different growth stages.This highlights the need for a universal approach capable of providing realistic plant visualizations across time and scene.Here,we introduce PlantGaussian,an approach for generating realistic three-dimensional(3D)visualization for plants across time and scenes.It marks one of the first applications of 3D Gaussian splatting techniques in plant science,achieving high-quality visualization across species and growth stages.By integrating the Segment Anything Model(SAM)and tracking algorithms,PlantGaussian overcomes the limitations of classic Gaussian reconstruction techniques in complex planting environments.A new mesh partitioning technique is employed to convert Gaussian rendering results into measurable plant meshes,offering a methodology for accurate 3D plant morphology phenotyping.To support this approach,PlantGaussian dataset is developed,which includes images of four crop species captured under multiple conditions and growth stages.Using only plant image sequences as input,it computes high-fidelity plant visualization models and 3D meshes for 3D plant morphological phenotyping.Visualization results indicate that most plant models achieve a Peak Signal-to-Noise Ratio(PSNR)exceeding 25,outperforming all models including the original 3D Gaussian Splatting and enhanced NeRF.The mesh results indicate an average relative error of 4%between the calculated values and the true measurements.As a generic 3D digital plant model,PlantGaussian will support expansion of plant phenotype databases,ecological research,and remote expert consultations.
文摘With the widespread application of 3D visualization in digital exhibition halls and virtual reality,achieving efficient rendering and high-fidelity presentation has become a key challenge.This study proposes a hybrid point cloud generation method that combines traditional sampling with 3D Gaussian splatting,aiming to address the issues of rendering delay and missing details in existing 3D displays.By improving the OBJ model parsing process and incorporating an adaptive area-weighted sampling algorithm,we achieve adaptive point cloud generation based on triangle density.Innovatively,we advance the ellipsoidal parameter estimation process of 3D Gaussian splatting to the point cloud generation stage.By establishing a mathematical relationship between the covariance matrix and local curvature,the generated point cloud naturally exhibits Gaussian distribution characteristics.Experimental results show that,compared to traditional methods,our approach reduces point cloud data by 38% while maintaining equivalent visual quality at a 4096×4096 texture resolution.By introducing mipmap texture optimization strategies and a GPU-accelerated rasterization pipeline,stable rendering at 60 frames per second is achieved in a WebGL environment.Additionally,we quantize and compress the spherical harmonic function parameters specific to 3D Gaussian splatting,reducing network transmission bandwidth to 52% of the original data.This study provides a new technical pathway for fields requiring high-precision display,such as the digitization of cultural heritage.
基金supported by the National Natural Science Foundation of China(62322210)Beijing Municipal Natural Science Foundation for Distinguished Young Scholars(JQ21013)+1 种基金Beijing Municipal Science and Technology Commission(Z231100005923031)2023 Tencent AI Lab Rhino-Bird Focused Research Program.
文摘The emergence of 3D Gaussian splatting(3DGS)has greatly accelerated rendering in novel view synthesis.Unlike neural implicit representations like neural radiance fields(NeRFs)that represent a 3D scene with position and viewpoint-conditioned neural networks,3D Gaussian splatting utilizes a set of Gaussian ellipsoids to model the scene so that efficient rendering can be accomplished by rasterizing Gaussian ellipsoids into images.Apart from fast rendering,the explicit representation of 3D Gaussian splatting also facilitates downstream tasks like dynamic reconstruction,geometry editing,and physical simulation.Considering the rapid changes and growing number of works in this field,we present a literature review of recent 3D Gaussian splatting methods,which can be roughly classified by functionality into 3D reconstruction,3D editing,and other downstream applications.Traditional point-based rendering methods and the rendering formulation of 3D Gaussian splatting are also covered to aid understanding of this technique.This survey aims to help beginners to quickly get started in this field and to provide experienced researchers with a comprehensive overview,aiming to stimulate future development of the 3D Gaussian splatting representation.
基金supported by the National Natural Science Foundation of China(62262036,62362043)the Yunnan Xingdian Talent Support Project(No.CYCX202203008)the Science and Technology Plan Projects of Yunnan Province(No.202502AD080003).
文摘This paper proposes a unified 3D Gaussian splatting framework consisting of three key components for motion and defocus blur reconstruction.First,a dual-blur perception module is designed to generate pixel-wise masks and predict the types of motion and defocus blur,guiding structural feature extraction.Second,a blur-aware Gaussian splatting integrates blur-aware features into the splatting process for accurate modeling of the global and local scene structure.Third,an Unoptimized Gaussian Ratio(UGR)-opacity joint optimization strategy is proposed to refine under-optimized regions,improving reconstruction accuracy under complex blur conditions.Experiments on a newly constructed motion and defocus blur dataset demonstrate the effectiveness of the proposed method for novel view synthesis.Compared with state-of-the-art methods,our framework achieves improvements of 0.28 dB,2.46%and 39.88%on PSNR,SSIM,and LPIPS,respectively.For deblurring tasks,it achieves improvements of 0.36 dB,3.24%and 28.96%on the same metrics.These results highlight the robustness and effectiveness of this approach.
文摘针对现有的基于机载LiDAR数据的滤波算法未能充分利用数据提供的所有信息及其所采用的数据结构表达复杂、存在信息损失等缺陷,提出了一种灰度体素结构分割模型下的机载LiDAR 3D滤波算法。算法首先以综合利用机载LiDAR数据的高程及强度信息为目的将点云数据规则化为灰度(体素内激光点的平均强度的离散化表示)体素结构,然后基于各体素间的空间连通性和灰度相似性准则,将灰度体素结构分割并标记为若干个3D连通区域,最后依据地面与其它目标的高差特性提取与其对应的3D连通区域。算法优势在于:基于体素结构设计,为3D滤波算法;综合利用了地面目标的几何及辐射特征,对比传统滤波算法可应用于更复杂的场景;滤波结果为3D地面体素形式,可直接用于创建地面3D模型。实验采用国际摄影测量与遥感协会(International Society for Photogrammetry and Remote Sensing,ISPRS)提供的不同密度的机载LiDAR基准测试数据测试了邻域尺度参数的敏感性及提出的算法的有效性,并和其他经典滤波算法做对比。定量评价的结果表明,51邻域为最佳空间邻域尺度;点云密度为0.67点/m2的数据集1的滤波平均完整率、正确率及质量分别为0.9611、0.9248及0.8934;点云密度为4点/m2的数据集2的滤波平均完整率、正确率及质量分别为0.8490、0.8531及0.7404;对比其全经典滤波算法本文算法在高密度点云数据滤波时表现更佳。
