In recent years,3D editing has become a significant research topic,primarily due to its ability to manipulate 3D assets in ways that fulfill the growing demand for personalized customization.The advent of radiance fie...In recent years,3D editing has become a significant research topic,primarily due to its ability to manipulate 3D assets in ways that fulfill the growing demand for personalized customization.The advent of radiance field-based methods,exemplified by pioneering frameworks such as Neural Radiance Fields(NeRF)and 3D Gaussian Splatting(3DGS),represents a pivotal innovation in scene representation and novel view synthesis,greatly enhancing the effectiveness and efficiency of 3D editing.This survey provides a comprehensive overview of the current advancements in 3D editing based on NeRF and 3DGS,systematically categorizing existing methods according to specific editing tasks while analyzing the current challenges and potential research directions.Our goal through this survey is to offer a comprehensive and valuable resource for researchers in the field,encouraging innovative ideas that may drive further progress in 3D editing.展开更多
基金supported in part by the National Natural Science Foundation of China(Grant Nos.62372457,62325211,and 62132021)the Major Program of Xiangjiang Laboratory(23XJ01009)+2 种基金the Young Elite Scientists Sponsorship Program by CAST(2023QNRC001)the Natural Science Foundation of Hunan Province of China(2022RC1104)the NUDT Research Grants(ZK22-52).
文摘In recent years,3D editing has become a significant research topic,primarily due to its ability to manipulate 3D assets in ways that fulfill the growing demand for personalized customization.The advent of radiance field-based methods,exemplified by pioneering frameworks such as Neural Radiance Fields(NeRF)and 3D Gaussian Splatting(3DGS),represents a pivotal innovation in scene representation and novel view synthesis,greatly enhancing the effectiveness and efficiency of 3D editing.This survey provides a comprehensive overview of the current advancements in 3D editing based on NeRF and 3DGS,systematically categorizing existing methods according to specific editing tasks while analyzing the current challenges and potential research directions.Our goal through this survey is to offer a comprehensive and valuable resource for researchers in the field,encouraging innovative ideas that may drive further progress in 3D editing.
文摘目的逆渲染旨在从二维多视图图像中同时恢复场景几何、材质及光照。近期,三维高斯泼溅(3D Gaussian splatting,3DGS)因其高效渲染特性被引入该领域,然而,当前基于物理真实的逆渲染时面临两大核心挑战:其一,3DGS基元本身主要为新视角合成优化,其提取的网格难以满足物理渲染的精度需求;其二,准确解耦材质与光照依赖对复杂光照传输和高频材质细节的精确建模,但现有方法在估计具有复杂可见性的直接光照时常面临高方差与计算瓶颈,影响了材质恢复的保真度和训练效率。方法为此,提出一种两阶段快速物理逆渲染框架:首先,在几何恢复阶段,引入扁平高斯基元压缩与多视图双向重投影误差约束,实现精度与速度的平衡,生成可直接用于下游渲染引擎的高精度三角网格;其次,在材质与光照恢复阶段,在提取的网格上采用基于单样本加权池采样的高效直接光照估计,并基于多分辨率哈希网格的神经表示实现复杂高频材质细节的恢复,在大幅降低渲染方差的同时显著缩短训练时间。结果为全面验证本文方法的有效性,本研究在基准数据集上开展了系统实验。在几何恢复方面,本文方法在TensoIR(tensorial inverse rendering)数据集上的法线平均角误差相比次优方法降低了19.59%;在DTU(Technical University of Denmark)数据集上,生成的网格在倒角距离分数上与最优方法持平,但训练速度提升了一倍。在材质恢复和新视角合成任务上,本文方法同样表现出色:在TensoIR数据集中,材质恢复的峰值信噪比(peak signal-to-noise ratio,PSNR)值较次优方法提升了2.84%,新视角合成的PSNR值提高了0.08。结论本工作成功构建了从三维高斯泼溅表达到可物理渲染的网格与材质贴图的快速、端到端逆渲染流程,为逆渲染技术在实时交互与工业级场景中的应用提供了高效且鲁棒的新范式。
