This article proposes a three-dimensional light field reconstruction method based on neural radiation field(NeRF)called Infrared NeRF for low resolution thermal infrared scenes.Based on the characteristics of the low ...This article proposes a three-dimensional light field reconstruction method based on neural radiation field(NeRF)called Infrared NeRF for low resolution thermal infrared scenes.Based on the characteristics of the low resolution thermal infrared imaging,various optimizations have been carried out to improve the speed and accuracy of thermal infrared 3D reconstruction.Firstly,inspired by Boltzmann's law of thermal radiation,distance is incorporated into the NeRF model for the first time,resulting in a nonlinear propagation of a single ray and a more accurate description of the physical property that infrared radiation intensity decreases with increasing distance.Secondly,in terms of improving inference speed,based on the phenomenon of high and low frequency distribution of foreground and background in infrared images,a multi ray non-uniform light synthesis strategy is proposed to make the model pay more attention to foreground objects in the scene,reduce the distribution of light in the background,and significantly reduce training time without reducing accuracy.In addition,compared to visible light scenes,infrared images only have a single channel,so fewer network parameters are required.Experiments using the same training data and data filtering method showed that,compared to the original NeRF,the improved network achieved an average improvement of 13.8%and 4.62%in PSNR and SSIM,respectively,while an average decreases of 46%in LPIPS.And thanks to the optimization of network layers and data filtering methods,training only takes about 25%of the original method's time to achieve convergence.Finally,for scenes with weak backgrounds,this article improves the inference speed of the model by 4-6 times compared to the original NeRF by limiting the query interval of the model.展开更多
The problems of the earth's gravity fields' visualization are both focus and puzzle currently. Aiming at multiresolution rendering, modeling of the Earth's gravity fields' data is discussed in the pape...The problems of the earth's gravity fields' visualization are both focus and puzzle currently. Aiming at multiresolution rendering, modeling of the Earth's gravity fields' data is discussed in the paper by using LOD algorithm based on Quad Tree. First, this paper employed the method of LOD based on Quad Tree to divide up the regional gravity anomaly data, introduced the combined node evaluation system that was composed of viewpoint related and roughness related systems, and then eliminated the T-cracks that appeared among the gravity anomaly data grids with different resolutions. The test results demonstrated that the gravity anomaly data grids' rendering effects were living, and the computational power was low. Therefore, the proposed algorithm was a suitable method for modeling the gravity anomaly data and has potential applications in visualization of the earth's gravity fields.展开更多
森林的实时渲染及光照是视景系统中的一个难题.基于图像的渲染方法(IBR)由于渲染速度与模型复杂度无关,被广泛应用于场景重建.基于光流场(Light Field Rendering)的IBR技术,提出一种迭代投射算法来进行外形重建,实现了具有实时光影特征...森林的实时渲染及光照是视景系统中的一个难题.基于图像的渲染方法(IBR)由于渲染速度与模型复杂度无关,被广泛应用于场景重建.基于光流场(Light Field Rendering)的IBR技术,提出一种迭代投射算法来进行外形重建,实现了具有实时光影特征的森林效果.实验表明该算法结合了传统迭代、投射算法各自的优点,在质量和效率方面取得了平衡.展开更多
同步定位与建图(simultaneous localization and mapping,SLAM)是指在未知环境中同时实现自主移动机器人的定位和环境地图构建,其在机器人技术和自动驾驶等领域有着重要价值。本文首先回顾SLAM技术的发展历程,从早期的手工特征提取方法...同步定位与建图(simultaneous localization and mapping,SLAM)是指在未知环境中同时实现自主移动机器人的定位和环境地图构建,其在机器人技术和自动驾驶等领域有着重要价值。本文首先回顾SLAM技术的发展历程,从早期的手工特征提取方法到现代的深度学习驱动的解决方案。其中,基于神经辐射场(neural radiance fields,NeRF)的SLAM方法利用神经网络进行场景表征,进一步提高了建图的可视化效果。然而,这类方法在渲染速度上仍然面临挑战,限制了其实时应用的可能性。相比之下,基于高斯溅射(Gaussian splatting,GS)的SLAM方法以其实时的渲染速度和照片级的场景渲染效果,为SLAM领域带来新的研究热点和机遇。接着,按照RGB/RGBD、多模态数据以及语义信息3种不同应用类型对基于高斯溅射的SLAM方法进行分类和总结,并针对每种情况讨论相应SLAM方法的优势和局限性。最后,针对当前基于高斯溅射的SLAM方法面临的实时性、基准一致化、大场景的扩展性以及灾难性遗忘等问题进行分析,并对未来研究方向进行展望。通过这些探讨和分析,旨在为SLAM领域的研究人员和工程师提供全面的视角和启发,帮助分析和理解当前SLAM系统面临的关键问题,推动该领域的技术进步和应用拓展。展开更多
文摘【目的】解决复杂果园环境下的果实精准分割问题。【方法】本文提出一种新颖的柑橘果树三维重建与果实语义分割方法。首先,利用神经辐射场(Neural radiance field,NeRF)技术从多视角图像中学习果树的隐式三维表示,生成高质量的果树点云模型;然后,采用改进后的随机局部点云特征聚合网络(Random local point cloud feature aggregation network,RandLA-Net)对果树点云进行端到端的语义分割,准确提取出果实点云。对RandLA-Net进行针对性改进,在编码器层后增加双边增强模块,采用更适合果实点云分割任务的损失函数,并通过柑橘果树数据集对改进后的分割网络进行验证试验。【结果】所提出的方法能够有效地重建果树三维结构,改进后网络的平均交并比提高了2.64个百分点,果实的交并比提高了7.33个百分点,验证了该方法在智慧果园场景下的实用性。【结论】研究为实现果园智能化管理和自动化采摘提供了新的技术支撑。
基金Support by the Fundamental Research Funds for the Central Universities(2024300443)the National Natural Science Foundation of China(NSFC)Young Scientists Fund(62405131)。
文摘This article proposes a three-dimensional light field reconstruction method based on neural radiation field(NeRF)called Infrared NeRF for low resolution thermal infrared scenes.Based on the characteristics of the low resolution thermal infrared imaging,various optimizations have been carried out to improve the speed and accuracy of thermal infrared 3D reconstruction.Firstly,inspired by Boltzmann's law of thermal radiation,distance is incorporated into the NeRF model for the first time,resulting in a nonlinear propagation of a single ray and a more accurate description of the physical property that infrared radiation intensity decreases with increasing distance.Secondly,in terms of improving inference speed,based on the phenomenon of high and low frequency distribution of foreground and background in infrared images,a multi ray non-uniform light synthesis strategy is proposed to make the model pay more attention to foreground objects in the scene,reduce the distribution of light in the background,and significantly reduce training time without reducing accuracy.In addition,compared to visible light scenes,infrared images only have a single channel,so fewer network parameters are required.Experiments using the same training data and data filtering method showed that,compared to the original NeRF,the improved network achieved an average improvement of 13.8%and 4.62%in PSNR and SSIM,respectively,while an average decreases of 46%in LPIPS.And thanks to the optimization of network layers and data filtering methods,training only takes about 25%of the original method's time to achieve convergence.Finally,for scenes with weak backgrounds,this article improves the inference speed of the model by 4-6 times compared to the original NeRF by limiting the query interval of the model.
文摘The problems of the earth's gravity fields' visualization are both focus and puzzle currently. Aiming at multiresolution rendering, modeling of the Earth's gravity fields' data is discussed in the paper by using LOD algorithm based on Quad Tree. First, this paper employed the method of LOD based on Quad Tree to divide up the regional gravity anomaly data, introduced the combined node evaluation system that was composed of viewpoint related and roughness related systems, and then eliminated the T-cracks that appeared among the gravity anomaly data grids with different resolutions. The test results demonstrated that the gravity anomaly data grids' rendering effects were living, and the computational power was low. Therefore, the proposed algorithm was a suitable method for modeling the gravity anomaly data and has potential applications in visualization of the earth's gravity fields.
文摘森林的实时渲染及光照是视景系统中的一个难题.基于图像的渲染方法(IBR)由于渲染速度与模型复杂度无关,被广泛应用于场景重建.基于光流场(Light Field Rendering)的IBR技术,提出一种迭代投射算法来进行外形重建,实现了具有实时光影特征的森林效果.实验表明该算法结合了传统迭代、投射算法各自的优点,在质量和效率方面取得了平衡.
文摘目的 基于点云的神经渲染方法受点云质量及特征提取的影响,易导致新视角合成图像渲染质量下降,为此提出一种融合局部空间信息的新视角合成方法。方法 针对点云质量及提取特征不足的问题,首先,设计一种神经点云特征对齐模块,将点云与图像匹配区域的特征进行对齐,融合后构成神经点云,提升其特征的局部表达能力;其次,提出一种神经点云Transformer模块,用于融合局部神经点云的上下文信息,在点云质量不佳的情况下仍能提取可靠的局部空间信息,有效增强了点云神经渲染方法的合成质量。结果 实验结果表明,在真实场景数据集中,对于只包含单一物品的数据集Tanks and Temples,本文方法在峰值信噪比(peak signal to noise ratio,PSNR)指标上与NeRF(neural radiance field)方法相比提升19.2%,相较于使用点云输入的方法 Tetra-NeRF和Point-NeRF分别提升了6.4%和3.8%,即使在场景更为复杂的ScanNet数据集中,与NeRF方法及Point-NeRF相比分别提升了34.6%和2.1%。结论 本文方法能够更好地利用点云的局部空间信息,有效改善了稀疏视角图像输入下因点云质量和提取特征导致的渲染质量下降,实验结果验证了本文方法的有效性。
