摘要
点云因其丰富的信息表达能力已成为三维视觉的主要表现形式,然而实际采集到的点云数据往往因各种因素导致稀疏或残缺,严重影响点云后续处理。点云补全算法旨在从残缺点云数据中重建完整点云模型,是3D重建、目标检测和形状分类等领域的重要研究基础。目前,基于深度学习的点云补全算法逐渐成为三维点云领域的研究热点,但补全任务中模型结构、精度和效率等挑战正阻碍点云补全算法的发展。本文对深度学习背景下的点云补全算法进行系统综述,首先根据网络输入模态将点云补全算法分为两大类,即基于单模态的方法以及基于多模态的方法。接着根据三维数据表征方式将基于单模态的方法分为三大类,即基于体素的方法、基于视图的方法以及基于点的方法,并对经典方法和最新方法进行系统的分析和总结,同时结合热点模型,如生成对抗网络(generative adversarial network,GAN)、Transformer模型等进一步分类对比,评述各类模型下点云补全算法的方法特点与网络性能。再对基于多模态的方法进行实际应用分析,结合扩散模型等方法进行算法性能对比。然后总结点云补全任务中常用的数据集及评价标准,分别以多种评价标准对比分析现有基于深度学习的点云补全算法在真实数据集与多种合成数据集上的性能表现。最后根据各分类的优缺点提出点云补全算法在深度学习领域的未来发展和研究趋势,为三维视觉领域的补全算法研究者提供重要参考价值。
Point clouds have become the main form of 3D vision because of their rich information expression ability.How⁃ever,the actual collected point cloud data are often sparse or incomplete due to the characteristics of the measured object,the performance of the measuring instrument,and environmental and human factors,which seriously affect the subsequent processing of the point cloud.The point cloud completion algorithm aims to reconstruct a complete point cloud model from incomplete point cloud data,which is an important research basis for 3D reconstruction,object detection,and shape classi⁃fication.With the rapid development of deep learning methods,their efficient feature extraction ability and excellent data processing ability have led them to be widely used in 3D point cloud algorithms.At present,point cloud completion algo⁃rithms based on deep learning have gradually become a research hotspot in the field of 3D point clouds.However,chal⁃lenges such as model structure,accuracy,and efficiency in completion tasks hinder the development of point cloud comple⁃tion algorithms.Examples include the problems of missing key structural information,fine-grained reconstruction,and inefficiency of the algorithm model.This study systematically reviews point cloud completion algorithms in the context of deep learning.First,according to the network input modality,the point cloud completion algorithms are divided into two categories,namely,single-modality-based methods and multimodality-based methods.Then,according to the representa⁃tion of 3D data,the methods based on a single modality are divided into three categories,namely,voxel-based methods,view-based methods,and point-based methods.The classical methods and the latest methods are systematically analyzed and summarized.The method characteristics and network performance of point cloud completion algorithms under various models were reviewed.Then,practical application analysis of the multimodal method is conducted,and the performance of the algorithm is compared with that of the diffusion model and other methods.Then,different datasets and evaluation crite⁃ria commonly used in point cloud completion tasks are summarized,the performance of existing point cloud completion algorithms based on deep learning on real datasets and synthetic datasets with a variety of evaluation criteria is compared,and the performance of existing point cloud completion algorithms is analyzed.Finally,according to the advantages and dis⁃advantages of each classification,the future development and research trends of point cloud completion algorithms in the field of deep learning are proposed.The research results are as follows:since the concept of the point cloud completion algorithm was proposed in 2018,most methods based on a single mode use the point method for completion and combine hotspot models for algorithm optimization,such as generative adversarial networks(GANs),Transformer models,and the Mamba model.Multimodal methods have developed rapidly since they were proposed in 2021,especially since the diffu⁃sion model was applied to the point cloud completion algorithm,which truly realizes multimodal input and output.Many researchers have explored multimodal information fusion at the feature level to improve the model accuracy of the comple⁃tion algorithm.This scheme also provides an updated algorithm theoretical basis for multivehicle cooperative intelligent per⁃ception technology in robotics and autonomous driving.Point cloud completion based on multimodal methods is also the development trend of point cloud completion algorithms in the future.Through a comprehensive survey and review of point cloud completion algorithms based on deep learning,the current research results have improved the ability of point cloud data feature extraction and model generation to a certain extent.However,the following research difficulties still exist:1)features and fine-grained methods:at present,most algorithms are dedicated to making full use of structural information for prediction and generating fine-grained and more complete point cloud shapes.Multiple fusion operations of the geomet⁃ric structure and attribute information based on the point cloud data structure must still be performed to enrich the highquality generation of point cloud data.2)Multimodal data fusion:point cloud data are usually fused with other sensor data to obtain more comprehensive information,such as RGB images and depth images.How to improve the method of multi⁃modal feature extraction and fusion and explore the smart fusion of multimodal data to improve the accuracy and robustness of the point cloud completion algorithm will be the difficulties of future research.In the future,the development of point cloud completion algorithms will realize that all modes from text and image to point cloud will be completely opened,and any input and output will be realized in the real sense.3)Data augmentation and diversity:large models of point clouds will be popular research topics in the future.Improving the generalization ability and data diversity of point cloud comple⁃tion algorithms in various scenarios via data augmentation or model diffusion has also become difficult in the field of point clouds.4)Real-time and interactivity:real-time requirements limit the development of point cloud completion algorithms in applications such as autonomous driving and robotics.The high complexity of the algorithm,the difficulty of multimodal feature information fusion,and the difficulty of large-scale data processing make the algorithm model inefficient,resulting in poor real-time performance.How to reduce the size of the data through data preprocessing,downsampling,and selecting a relatively lightweight model structure,such as the Mamba model,to improve model efficiency was investigated.More⁃over,the rapid adjustment and optimization of high-quality point cloud completion results according to user interaction information will be difficult for future development.A systematic review of point cloud completion algorithms against the background of deep learning provides important reference value for researchers of completion algorithms in the field of 3D vision.
作者
胡伏原
李晨露
周涛
程洪福
顾敏明
Hu Fuyuan;Li Chenlu;Zhou Tao;Cheng Hongfu;Gu Minming(School of Electronic and Information Engineering,Suzhou University of Science and Technology,Suzhou 215009,China;Jiangsu Industrial Intelligent and Low-carbon Technology Engineering Center,Suzhou 215009,China;Suzhou Key Laboratory of Intelligent and Low-carbon Technology Application,Suzhou 215009,China;School of Computer Science and Engineering,North Minzu University,Yinchuan 750021,China;Suzhou Humble Administrator’s Garden Administration Office(Suzhou Garden Museum),Suzhou 215001,China)
出处
《中国图象图形学报》
北大核心
2025年第2期309-333,共25页
Journal of Image and Graphics
基金
国家重点研发计划资助(2023YFE0116300)
国家自然科学基金项目(61876121)
苏州市科技发展计划项目(SS202133)。
