针对复杂室内环境中视觉同步定位与建图(simultaneous localization and mapping,SLAM)算法在高质量三维重建中的效率问题,提出了一种高效的神经辐射场SLAM(NeRF-SLAM)算法——EN-SLAM。该算法利用多分辨率哈希网格表示场景,结合其快速...针对复杂室内环境中视觉同步定位与建图(simultaneous localization and mapping,SLAM)算法在高质量三维重建中的效率问题,提出了一种高效的神经辐射场SLAM(NeRF-SLAM)算法——EN-SLAM。该算法利用多分辨率哈希网格表示场景,结合其快速收敛特性及高频局部特征表示能力,显著提升了三维重建效率。为进一步增强未观测区域的表面连贯性及细节补全,算法引入球谐函数进行方向编码,从而保证了重建结果的一致性与细节完整性,同时提高实时性。此外,设计了一种信息引导采样策略,优先采样对重建贡献较大的光线,同时实现全局优化(BA)在所有关键帧上的高效执行。在Replica、ScanNet、TUM RGBD和Neural RGB-D数据集上的实验表明,该算法在提高建图精度、跟踪精度及渲染质量的同时,在Replica数据集上的运行时间较iMAP、NICE-SLAM、Vox-Fusion、ESLAM和Co-SLAM分别提升了98.99%、92.80%、91.97%、63.77%和19.15%,且场景重建完成率达到94.14%。展开更多
This article proposes a three-dimensional light field reconstruction method based on neural radiation field(NeRF)called Infrared NeRF for low resolution thermal infrared scenes.Based on the characteristics of the low ...This article proposes a three-dimensional light field reconstruction method based on neural radiation field(NeRF)called Infrared NeRF for low resolution thermal infrared scenes.Based on the characteristics of the low resolution thermal infrared imaging,various optimizations have been carried out to improve the speed and accuracy of thermal infrared 3D reconstruction.Firstly,inspired by Boltzmann's law of thermal radiation,distance is incorporated into the NeRF model for the first time,resulting in a nonlinear propagation of a single ray and a more accurate description of the physical property that infrared radiation intensity decreases with increasing distance.Secondly,in terms of improving inference speed,based on the phenomenon of high and low frequency distribution of foreground and background in infrared images,a multi ray non-uniform light synthesis strategy is proposed to make the model pay more attention to foreground objects in the scene,reduce the distribution of light in the background,and significantly reduce training time without reducing accuracy.In addition,compared to visible light scenes,infrared images only have a single channel,so fewer network parameters are required.Experiments using the same training data and data filtering method showed that,compared to the original NeRF,the improved network achieved an average improvement of 13.8%and 4.62%in PSNR and SSIM,respectively,while an average decreases of 46%in LPIPS.And thanks to the optimization of network layers and data filtering methods,training only takes about 25%of the original method's time to achieve convergence.Finally,for scenes with weak backgrounds,this article improves the inference speed of the model by 4-6 times compared to the original NeRF by limiting the query interval of the model.展开更多
基金Support by the Fundamental Research Funds for the Central Universities(2024300443)the National Natural Science Foundation of China(NSFC)Young Scientists Fund(62405131)。
文摘This article proposes a three-dimensional light field reconstruction method based on neural radiation field(NeRF)called Infrared NeRF for low resolution thermal infrared scenes.Based on the characteristics of the low resolution thermal infrared imaging,various optimizations have been carried out to improve the speed and accuracy of thermal infrared 3D reconstruction.Firstly,inspired by Boltzmann's law of thermal radiation,distance is incorporated into the NeRF model for the first time,resulting in a nonlinear propagation of a single ray and a more accurate description of the physical property that infrared radiation intensity decreases with increasing distance.Secondly,in terms of improving inference speed,based on the phenomenon of high and low frequency distribution of foreground and background in infrared images,a multi ray non-uniform light synthesis strategy is proposed to make the model pay more attention to foreground objects in the scene,reduce the distribution of light in the background,and significantly reduce training time without reducing accuracy.In addition,compared to visible light scenes,infrared images only have a single channel,so fewer network parameters are required.Experiments using the same training data and data filtering method showed that,compared to the original NeRF,the improved network achieved an average improvement of 13.8%and 4.62%in PSNR and SSIM,respectively,while an average decreases of 46%in LPIPS.And thanks to the optimization of network layers and data filtering methods,training only takes about 25%of the original method's time to achieve convergence.Finally,for scenes with weak backgrounds,this article improves the inference speed of the model by 4-6 times compared to the original NeRF by limiting the query interval of the model.
文摘【目的】解决复杂果园环境下的果实精准分割问题。【方法】本文提出一种新颖的柑橘果树三维重建与果实语义分割方法。首先,利用神经辐射场(Neural radiance field,NeRF)技术从多视角图像中学习果树的隐式三维表示,生成高质量的果树点云模型;然后,采用改进后的随机局部点云特征聚合网络(Random local point cloud feature aggregation network,RandLA-Net)对果树点云进行端到端的语义分割,准确提取出果实点云。对RandLA-Net进行针对性改进,在编码器层后增加双边增强模块,采用更适合果实点云分割任务的损失函数,并通过柑橘果树数据集对改进后的分割网络进行验证试验。【结果】所提出的方法能够有效地重建果树三维结构,改进后网络的平均交并比提高了2.64个百分点,果实的交并比提高了7.33个百分点,验证了该方法在智慧果园场景下的实用性。【结论】研究为实现果园智能化管理和自动化采摘提供了新的技术支撑。
