The complexity of application scenarios and the enormous volume of point cloud data make it difficult to quickly and effectively segment the scenario only based on the point cloud.In this paper,to address the semantic...The complexity of application scenarios and the enormous volume of point cloud data make it difficult to quickly and effectively segment the scenario only based on the point cloud.In this paper,to address the semantic segmentation for safety driving of unmanned shuttle buses,an accurate and effective point cloud-based semantic segmentation method is proposed for specified scenarios(such as campus).Firstly,we analyze the characteristic of the shuttle bus scenarios and propose to use ROI selection to reduce the total points in computation,and then propose an improved semantic segmentation model based on Cylinder3D,which improves mean Intersection over Union(mIoU)by 1.3%over the original model on SemanticKITTI data;then,a semantic category division method is proposed for road scenario of shuttle bus and practical application requirements,and then we further simplify the model to improve the efficiency without losing the accuracy.Finally,the nuScenes dataset and the real gathered campus scene data are used to validate and analyze the proposed method.The experimental results on the nuScenes dataset and our data demonstrate that the proposed method performs better than other point cloud semantic segmentation methods in terms of application requirements for unmanned shuttle buses.Which has a higher accuracy(82.73%in mIoU)and a higher computational efficiency(inference speed of 90 ms).展开更多
This paper focuses on the effective utilization of data augmentation techniques for 3Dlidar point clouds to enhance the performance of neural network models.These point clouds,which represent spatial information throu...This paper focuses on the effective utilization of data augmentation techniques for 3Dlidar point clouds to enhance the performance of neural network models.These point clouds,which represent spatial information through a collection of 3D coordinates,have found wide-ranging applications.Data augmentation has emerged as a potent solution to the challenges posed by limited labeled data and the need to enhance model generalization capabilities.Much of the existing research is devoted to crafting novel data augmentation methods specifically for 3D lidar point clouds.However,there has been a lack of focus on making the most of the numerous existing augmentation techniques.Addressing this deficiency,this research investigates the possibility of combining two fundamental data augmentation strategies.The paper introduces PolarMix andMix3D,two commonly employed augmentation techniques,and presents a new approach,named RandomFusion.Instead of using a fixed or predetermined combination of augmentation methods,RandomFusion randomly chooses one method from a pool of options for each instance or sample.This innovative data augmentation technique randomly augments each point in the point cloud with either PolarMix or Mix3D.The crux of this strategy is the random choice between PolarMix and Mix3Dfor the augmentation of each point within the point cloud data set.The results of the experiments conducted validate the efficacy of the RandomFusion strategy in enhancing the performance of neural network models for 3D lidar point cloud semantic segmentation tasks.This is achieved without compromising computational efficiency.By examining the potential of merging different augmentation techniques,the research contributes significantly to a more comprehensive understanding of how to utilize existing augmentation methods for 3D lidar point clouds.RandomFusion data augmentation technique offers a simple yet effective method to leverage the diversity of augmentation techniques and boost the robustness of models.The insights gained from this research can pave the way for future work aimed at developing more advanced and efficient data augmentation strategies for 3D lidar point cloud analysis.展开更多
Traditional models for semantic segmentation in point clouds primarily focus on smaller scales.However,in real-world applications,point clouds often exhibit larger scales,leading to heavy computational and memory requ...Traditional models for semantic segmentation in point clouds primarily focus on smaller scales.However,in real-world applications,point clouds often exhibit larger scales,leading to heavy computational and memory requirements.The key to handling large-scale point clouds lies in leveraging random sampling,which offers higher computational efficiency and lower memory consumption compared to other sampling methods.Nevertheless,the use of random sampling can potentially result in the loss of crucial points during the encoding stage.To address these issues,this paper proposes cross-fusion self-attention network(CFSA-Net),a lightweight and efficient network architecture specifically designed for directly processing large-scale point clouds.At the core of this network is the incorporation of random sampling alongside a local feature extraction module based on cross-fusion self-attention(CFSA).This module effectively integrates long-range contextual dependencies between points by employing hierarchical position encoding(HPC).Furthermore,it enhances the interaction between each point’s coordinates and feature information through cross-fusion self-attention pooling,enabling the acquisition of more comprehensive geometric information.Finally,a residual optimization(RO)structure is introduced to extend the receptive field of individual points by stacking hierarchical position encoding and cross-fusion self-attention pooling,thereby reducing the impact of information loss caused by random sampling.Experimental results on the Stanford Large-Scale 3D Indoor Spaces(S3DIS),Semantic3D,and SemanticKITTI datasets demonstrate the superiority of this algorithm over advanced approaches such as RandLA-Net and KPConv.These findings underscore the excellent performance of CFSA-Net in large-scale 3D semantic segmentation.展开更多
This paper focuses on the task of few-shot 3D point cloud semantic segmentation.Despite some progress,this task still encounters many issues due to the insufficient samples given,e.g.,incomplete object segmentation an...This paper focuses on the task of few-shot 3D point cloud semantic segmentation.Despite some progress,this task still encounters many issues due to the insufficient samples given,e.g.,incomplete object segmentation and inaccurate semantic discrimination.To tackle these issues,we first leverage part-whole relationships into the task of 3D point cloud semantic segmentation to capture semantic integrity,which is empowered by the dynamic capsule routing with the module of 3D Capsule Networks(CapsNets)in the embedding network.Concretely,the dynamic routing amalgamates geometric information of the 3D point cloud data to construct higher-level feature representations,which capture the relationships between object parts and their wholes.Secondly,we designed a multi-prototype enhancement module to enhance the prototype discriminability.Specifically,the single-prototype enhancement mechanism is expanded to the multi-prototype enhancement version for capturing rich semantics.Besides,the shot-correlation within the category is calculated via the interaction of different samples to enhance the intra-category similarity.Ablation studies prove that the involved part-whole relations and proposed multi-prototype enhancement module help to achieve complete object segmentation and improve semantic discrimination.Moreover,under the integration of these two modules,quantitative and qualitative experiments on two public benchmarks,including S3DIS and ScanNet,indicate the superior performance of the proposed framework on the task of 3D point cloud semantic segmentation,compared to some state-of-the-art methods.展开更多
With the increasing popularity of 3D sensors(e.g.,Kinect)and light field cameras,technologies such as driverless,smart home and virtual reality have become hot spots for engineering applications.As an important part o...With the increasing popularity of 3D sensors(e.g.,Kinect)and light field cameras,technologies such as driverless,smart home and virtual reality have become hot spots for engineering applications.As an important part of 3D vision tasks,point cloud semantic segmentation has received a lot of attention from researchers.In this work,we focus on realistically collected indoor point cloud data and propose a point cloud semantic segmentation method based on PAConv and SE_variant.The SE_variant module captures global perception from a broad perspective of feature space by fusing different pooling methods,which fully utilize the channel information of point clouds.The effectiveness of the method is verified by comparing with other methods on S3DIS and ScanNetV2 semantic tagging benchmarks,and achieving 65.3%mIoU in S3DIS,47.6%mIoU in ScanNetV2.The results of the ablation experiments verify the effectiveness of the key modules and analyze how to use the attention mechanism to improve the 3D semantic segmentation performance.展开更多
Semantic segmentation in the context of 3D point clouds for the railway environment holds a significant economic value,but its development is severely hindered by the lack of suitable and specific datasets.Additionall...Semantic segmentation in the context of 3D point clouds for the railway environment holds a significant economic value,but its development is severely hindered by the lack of suitable and specific datasets.Additionally,the models trained on existing urban road point cloud datasets demonstrate poor generalisation on railway data due to a large domain gap caused by non-overlapping special/rare categories,for example,rail track,track bed etc.To harness the potential of supervised learning methods in the domain of 3D railway semantic segmentation,we introduce RailPC,a new point cloud benchmark.RailPC provides a large-scale dataset with rich annotations for semantic segmentation in the railway environment.Notably,RailPC contains twice the number of annotated points compared to the largest available mobile laser scanning(MLS)point cloud dataset and is the first railway-specific 3D dataset for semantic segmentation.It covers a total of nearly 25 km railway in two different scenes(urban and mountain),with 3 billion points that are finely labelled as 16 most typical classes with respect to railway,and the data acquisition process is completed in China by MLS systems.Through extensive experimentation,we evaluate the performance of advanced scene understanding methods on the annotated dataset and present a synthetic analysis of semantic segmentation results.Based on our findings,we establish some critical challenges towards railway-scale point cloud semantic segmentation.The dataset is available at https://github.com/NNU-GISA/GISA-RailPC,and we will continuously update it based on community feedback.展开更多
In recent years,semantic segmentation on 3D point cloud data has attracted much attention.Unlike 2D images where pixels distribute regularly in the image domain,3D point clouds in non-Euclidean space are irregular and...In recent years,semantic segmentation on 3D point cloud data has attracted much attention.Unlike 2D images where pixels distribute regularly in the image domain,3D point clouds in non-Euclidean space are irregular and inherently sparse.Therefore,it is very difficult to extract long-range contexts and effectively aggregate local features for semantic segmentation in 3D point cloud space.Most current methods either focus on local feature aggregation or long-range context dependency,but fail to directly establish a global-local feature extractor to complete the point cloud semantic segmentation tasks.In this paper,we propose a Transformer-based stratified graph convolutional network(SGT-Net),which enlarges the effective receptive field and builds direct long-range dependency.Specifically,we first propose a novel dense-sparse sampling strategy that provides dense local vertices and sparse long-distance vertices for subsequent graph convolutional network(GCN).Secondly,we propose a multi-key self-attention mechanism based on the Transformer to further weight augmentation for crucial neighboring relationships and enlarge the effective receptive field.In addition,to further improve the efficiency of the network,we propose a similarity measurement module to determine whether the neighborhood near the center point is effective.We demonstrate the validity and superiority of our method on the S3DIS and ShapeNet datasets.Through ablation experiments and segmentation visualization,we verify that the SGT model can improve the performance of the point cloud semantic segmentation.展开更多
Neurons can be abstractly represented as skeletons due to the filament nature of neurites.With the rapid development of imaging and image analysis techniques,an increasing amount of neuron skeleton data is being produ...Neurons can be abstractly represented as skeletons due to the filament nature of neurites.With the rapid development of imaging and image analysis techniques,an increasing amount of neuron skeleton data is being produced.In some scienti fic studies,it is necessary to dissect the axons and dendrites,which is typically done manually and is both tedious and time-consuming.To automate this process,we have developed a method that relies solely on neuronal skeletons using Geometric Deep Learning(GDL).We demonstrate the effectiveness of this method using pyramidal neurons in mammalian brains,and the results are promising for its application in neuroscience studies.展开更多
Volume parameter is the basic content of a spatial body object morphology analysis.However,the challenge lies in the volume calculation of irregular objects.The point cloud slicing method proposed in this study effect...Volume parameter is the basic content of a spatial body object morphology analysis.However,the challenge lies in the volume calculation of irregular objects.The point cloud slicing method proposed in this study effectively works in calculating the volume of the point cloud of the spatial object obtained through three-dimensional laser scanning(3DLS).In this method,a uniformly spaced sequent slicing process is first conducted in a specific direction on the point cloud of the spatial object obtained through 3DLS.A series of discrete point cloud slices corresponding to the point cloud bodies are then obtained.Subsequently,the outline boundary polygon of the point cloud slicing is searched one by one in accordance with the slicing sequence and areas of the polygon.The point cloud slice is also calculated.Finally,the individual point cloud section volume is calculated through the slicing areas and the adjacent slicing gap.Thus,the total volume of the scanned spatial object can be calculated by summing up the individual volumes.According to the results and analysis of the calculated examples,the slice-based volume-calculating method for the point cloud of irregular objects obtained through 3DLS is correct,concise in process,reliable in results,efficient in calculation methods,and controllable on accuracy.This method comes as a good solution to the volume calculation of irregular objects.展开更多
The process of segmenting point cloud data into several homogeneous areas with points in the same region having the same attributes is known as 3D segmentation.Segmentation is challenging with point cloud data due to...The process of segmenting point cloud data into several homogeneous areas with points in the same region having the same attributes is known as 3D segmentation.Segmentation is challenging with point cloud data due to substantial redundancy,fluctuating sample density and lack of apparent organization.The research area has a wide range of robotics applications,including intelligent vehicles,autonomous mapping and navigation.A number of researchers have introduced various methodologies and algorithms.Deep learning has been successfully used to a spectrum of 2D vision domains as a prevailing A.I.methods.However,due to the specific problems of processing point clouds with deep neural networks,deep learning on point clouds is still in its initial stages.This study examines many strategies that have been presented to 3D instance and semantic segmentation and gives a complete assessment of current developments in deep learning-based 3D segmentation.In these approaches’benefits,draw backs,and design mechanisms are studied and addressed.This study evaluates the impact of various segmentation algorithms on competitiveness on various publicly accessible datasets,as well as the most often used pipelines,their advantages and limits,insightful findings and intriguing future research directions.展开更多
Separating wood and leaf components in tree point clouds is one of the key tasks for achieving automated forest inventory and management.To obtain accurate wood-leaf separation results,traditional methods typically re...Separating wood and leaf components in tree point clouds is one of the key tasks for achieving automated forest inventory and management.To obtain accurate wood-leaf separation results,traditional methods typically rely on large amounts of annotated point cloud data to train supervised semantic segmentation networks.However,point wise annotation is not only extremely labor-intensive but also time-consuming and costly,which greatly limits the widespread application and adoption of supervised learning methods in wood-leaf separation tasks.To eliminate the dependence on annotated point clouds,this study explores the feasibility of wood-leaf separation under completely unsupervised conditions.To this end,we propose an unsupervised semantic segmentation network that is capable of directly extracting wood and leaf components in 3D point clouds.The network adopts a sparse convolutional neural network as the backbone and incorporates two custom-designed modules:the dual point attention(DPA)module and the point cloud feature convolutional integrator(PFCI)module,for enhanced feature fusion and extraction.Semantic classification is then achieved by generating pseudolabels via super point clustering.Based on large-scale public datasets containing coniferous and broadleaf forests,in addition to our self-constructed dataset,our proposed network achieved an overall accuracy(oAcc)of 67.583%,a mean clas-sification accuracy(mAcc)of 50.249%,and a mean intersection over union(mIoU)of 38.512%,and in wood and leaf separation at the tree level,it attained an oAcc of 80.856%,a mAcc of 64.013%,and a mIoU of 49.695%.Across both the forest and tree scenarios,our network outperforms the current state-of-the-art methods,namely,GrowSP and PointDC.Ablation experiments further confirm that each of the proposed modules contributes significantly to improving the segmentation accuracy,and in addition,our segmentation network demonstrates strong robustness even under high occlusion rates and exhibits excellent generalization capability.展开更多
Current outdoor point-cloud segmentation methods typically formulate semantic segmentation as a per-point/voxel-classification task.Although this strategy is straightforward because it classifies each point directly,i...Current outdoor point-cloud segmentation methods typically formulate semantic segmentation as a per-point/voxel-classification task.Although this strategy is straightforward because it classifies each point directly,it ignores the overall relationship of the category.As an alternative paradigm,mask classification decouples category classification from region localization,allowing the model to better capture overall category relationships.In this paper,we propose a novel approach called the point mask transformer(PMFormer),which transforms the semantic segmentation of point clouds from per-point classification to mask classification using a transformer architecture.The proposed model comprises a 3D backbone,transformer decoder,and segmentation head that predicts a series of binary masks,each associated with a global class label.Furthermore,to accommodate the unique characteristics of large and sparse outdoor point-cloud scenes,we propose three enhancements for the integration of point-cloud data with the transformer:MaskMix,3D position encoding,and attention weights.We evaluate our model using the SemanticKITTI and nuScenes datasets.Our experimental results show that the proposed method outperforms state-of-the-art semantic segmentation approaches.展开更多
基金supported by the National Natural Science Foundation of China(62103064)Sichuan Science and Technology Program(2021YFG0295,2021YFG0133,2022YFN0020,2020YFG0177,2021YFG0187,2021YFN0104,2021YFH0069,2021YJ0086,21ZDY F3598)+2 种基金the Opening Project of Unmanned System Intelligent Perception Control Technology Engineering Laboratory of Sichuan Province(WRXT2020-005)Scientific Research Foundation of CUIT(KYTZ202109)Key Research and Development Support Program of Chengdu Science and Technology Bureau(2022-YF05-01128-SN).
文摘The complexity of application scenarios and the enormous volume of point cloud data make it difficult to quickly and effectively segment the scenario only based on the point cloud.In this paper,to address the semantic segmentation for safety driving of unmanned shuttle buses,an accurate and effective point cloud-based semantic segmentation method is proposed for specified scenarios(such as campus).Firstly,we analyze the characteristic of the shuttle bus scenarios and propose to use ROI selection to reduce the total points in computation,and then propose an improved semantic segmentation model based on Cylinder3D,which improves mean Intersection over Union(mIoU)by 1.3%over the original model on SemanticKITTI data;then,a semantic category division method is proposed for road scenario of shuttle bus and practical application requirements,and then we further simplify the model to improve the efficiency without losing the accuracy.Finally,the nuScenes dataset and the real gathered campus scene data are used to validate and analyze the proposed method.The experimental results on the nuScenes dataset and our data demonstrate that the proposed method performs better than other point cloud semantic segmentation methods in terms of application requirements for unmanned shuttle buses.Which has a higher accuracy(82.73%in mIoU)and a higher computational efficiency(inference speed of 90 ms).
基金funded in part by the Key Project of Nature Science Research for Universities of Anhui Province of China(No.2022AH051720)in part by the Science and Technology Development Fund,Macao SAR(Grant Nos.0093/2022/A2,0076/2022/A2 and 0008/2022/AGJ)in part by the China University Industry-University-Research Collaborative Innovation Fund(No.2021FNA04017).
文摘This paper focuses on the effective utilization of data augmentation techniques for 3Dlidar point clouds to enhance the performance of neural network models.These point clouds,which represent spatial information through a collection of 3D coordinates,have found wide-ranging applications.Data augmentation has emerged as a potent solution to the challenges posed by limited labeled data and the need to enhance model generalization capabilities.Much of the existing research is devoted to crafting novel data augmentation methods specifically for 3D lidar point clouds.However,there has been a lack of focus on making the most of the numerous existing augmentation techniques.Addressing this deficiency,this research investigates the possibility of combining two fundamental data augmentation strategies.The paper introduces PolarMix andMix3D,two commonly employed augmentation techniques,and presents a new approach,named RandomFusion.Instead of using a fixed or predetermined combination of augmentation methods,RandomFusion randomly chooses one method from a pool of options for each instance or sample.This innovative data augmentation technique randomly augments each point in the point cloud with either PolarMix or Mix3D.The crux of this strategy is the random choice between PolarMix and Mix3Dfor the augmentation of each point within the point cloud data set.The results of the experiments conducted validate the efficacy of the RandomFusion strategy in enhancing the performance of neural network models for 3D lidar point cloud semantic segmentation tasks.This is achieved without compromising computational efficiency.By examining the potential of merging different augmentation techniques,the research contributes significantly to a more comprehensive understanding of how to utilize existing augmentation methods for 3D lidar point clouds.RandomFusion data augmentation technique offers a simple yet effective method to leverage the diversity of augmentation techniques and boost the robustness of models.The insights gained from this research can pave the way for future work aimed at developing more advanced and efficient data augmentation strategies for 3D lidar point cloud analysis.
基金funded by the National Natural Science Foundation of China Youth Project(61603127).
文摘Traditional models for semantic segmentation in point clouds primarily focus on smaller scales.However,in real-world applications,point clouds often exhibit larger scales,leading to heavy computational and memory requirements.The key to handling large-scale point clouds lies in leveraging random sampling,which offers higher computational efficiency and lower memory consumption compared to other sampling methods.Nevertheless,the use of random sampling can potentially result in the loss of crucial points during the encoding stage.To address these issues,this paper proposes cross-fusion self-attention network(CFSA-Net),a lightweight and efficient network architecture specifically designed for directly processing large-scale point clouds.At the core of this network is the incorporation of random sampling alongside a local feature extraction module based on cross-fusion self-attention(CFSA).This module effectively integrates long-range contextual dependencies between points by employing hierarchical position encoding(HPC).Furthermore,it enhances the interaction between each point’s coordinates and feature information through cross-fusion self-attention pooling,enabling the acquisition of more comprehensive geometric information.Finally,a residual optimization(RO)structure is introduced to extend the receptive field of individual points by stacking hierarchical position encoding and cross-fusion self-attention pooling,thereby reducing the impact of information loss caused by random sampling.Experimental results on the Stanford Large-Scale 3D Indoor Spaces(S3DIS),Semantic3D,and SemanticKITTI datasets demonstrate the superiority of this algorithm over advanced approaches such as RandLA-Net and KPConv.These findings underscore the excellent performance of CFSA-Net in large-scale 3D semantic segmentation.
基金This work is supported by the National Natural Science Foundation of China under Grant No.62001341the National Natural Science Foundation of Jiangsu Province under Grant No.BK20221379the Jiangsu Engineering Research Center of Digital Twinning Technology for Key Equipment in Petrochemical Process under Grant No.DTEC202104.
文摘This paper focuses on the task of few-shot 3D point cloud semantic segmentation.Despite some progress,this task still encounters many issues due to the insufficient samples given,e.g.,incomplete object segmentation and inaccurate semantic discrimination.To tackle these issues,we first leverage part-whole relationships into the task of 3D point cloud semantic segmentation to capture semantic integrity,which is empowered by the dynamic capsule routing with the module of 3D Capsule Networks(CapsNets)in the embedding network.Concretely,the dynamic routing amalgamates geometric information of the 3D point cloud data to construct higher-level feature representations,which capture the relationships between object parts and their wholes.Secondly,we designed a multi-prototype enhancement module to enhance the prototype discriminability.Specifically,the single-prototype enhancement mechanism is expanded to the multi-prototype enhancement version for capturing rich semantics.Besides,the shot-correlation within the category is calculated via the interaction of different samples to enhance the intra-category similarity.Ablation studies prove that the involved part-whole relations and proposed multi-prototype enhancement module help to achieve complete object segmentation and improve semantic discrimination.Moreover,under the integration of these two modules,quantitative and qualitative experiments on two public benchmarks,including S3DIS and ScanNet,indicate the superior performance of the proposed framework on the task of 3D point cloud semantic segmentation,compared to some state-of-the-art methods.
文摘With the increasing popularity of 3D sensors(e.g.,Kinect)and light field cameras,technologies such as driverless,smart home and virtual reality have become hot spots for engineering applications.As an important part of 3D vision tasks,point cloud semantic segmentation has received a lot of attention from researchers.In this work,we focus on realistically collected indoor point cloud data and propose a point cloud semantic segmentation method based on PAConv and SE_variant.The SE_variant module captures global perception from a broad perspective of feature space by fusing different pooling methods,which fully utilize the channel information of point clouds.The effectiveness of the method is verified by comparing with other methods on S3DIS and ScanNetV2 semantic tagging benchmarks,and achieving 65.3%mIoU in S3DIS,47.6%mIoU in ScanNetV2.The results of the ablation experiments verify the effectiveness of the key modules and analyze how to use the attention mechanism to improve the 3D semantic segmentation performance.
基金Key Laboratory of Degraded and Unused Land Consolidation Engineering,Ministry of Natural Resources of China,Grant/Award Number:SXDJ2024-22Technology Innovation Centre for Integrated Applications in Remote Sensing and Navigation,Ministry of Natural Resources of China,Grant/Award Number:TICIARSN-2023-06+2 种基金National Natural Science Foundation of China,Grant/Award Numbers:42171446,62302246Zhejiang Provincial Natural Science Foundation of China,Grant/Award Number:LQ23F010008Science and Technology Program of Tianjin,China,Grant/Award Number:23ZGSSSS00010。
文摘Semantic segmentation in the context of 3D point clouds for the railway environment holds a significant economic value,but its development is severely hindered by the lack of suitable and specific datasets.Additionally,the models trained on existing urban road point cloud datasets demonstrate poor generalisation on railway data due to a large domain gap caused by non-overlapping special/rare categories,for example,rail track,track bed etc.To harness the potential of supervised learning methods in the domain of 3D railway semantic segmentation,we introduce RailPC,a new point cloud benchmark.RailPC provides a large-scale dataset with rich annotations for semantic segmentation in the railway environment.Notably,RailPC contains twice the number of annotated points compared to the largest available mobile laser scanning(MLS)point cloud dataset and is the first railway-specific 3D dataset for semantic segmentation.It covers a total of nearly 25 km railway in two different scenes(urban and mountain),with 3 billion points that are finely labelled as 16 most typical classes with respect to railway,and the data acquisition process is completed in China by MLS systems.Through extensive experimentation,we evaluate the performance of advanced scene understanding methods on the annotated dataset and present a synthetic analysis of semantic segmentation results.Based on our findings,we establish some critical challenges towards railway-scale point cloud semantic segmentation.The dataset is available at https://github.com/NNU-GISA/GISA-RailPC,and we will continuously update it based on community feedback.
基金supported in part by the National Natural Science Foundation of China under Grant Nos.U20A20197,62306187the Foundation of Ministry of Industry and Information Technology TC220H05X-04.
文摘In recent years,semantic segmentation on 3D point cloud data has attracted much attention.Unlike 2D images where pixels distribute regularly in the image domain,3D point clouds in non-Euclidean space are irregular and inherently sparse.Therefore,it is very difficult to extract long-range contexts and effectively aggregate local features for semantic segmentation in 3D point cloud space.Most current methods either focus on local feature aggregation or long-range context dependency,but fail to directly establish a global-local feature extractor to complete the point cloud semantic segmentation tasks.In this paper,we propose a Transformer-based stratified graph convolutional network(SGT-Net),which enlarges the effective receptive field and builds direct long-range dependency.Specifically,we first propose a novel dense-sparse sampling strategy that provides dense local vertices and sparse long-distance vertices for subsequent graph convolutional network(GCN).Secondly,we propose a multi-key self-attention mechanism based on the Transformer to further weight augmentation for crucial neighboring relationships and enlarge the effective receptive field.In addition,to further improve the efficiency of the network,we propose a similarity measurement module to determine whether the neighborhood near the center point is effective.We demonstrate the validity and superiority of our method on the S3DIS and ShapeNet datasets.Through ablation experiments and segmentation visualization,we verify that the SGT model can improve the performance of the point cloud semantic segmentation.
基金supported by the Simons Foundation,the National Natural Science Foundation of China(No.NSFC61405038)the Fujian provincial fund(No.2020J01453).
文摘Neurons can be abstractly represented as skeletons due to the filament nature of neurites.With the rapid development of imaging and image analysis techniques,an increasing amount of neuron skeleton data is being produced.In some scienti fic studies,it is necessary to dissect the axons and dendrites,which is typically done manually and is both tedious and time-consuming.To automate this process,we have developed a method that relies solely on neuronal skeletons using Geometric Deep Learning(GDL).We demonstrate the effectiveness of this method using pyramidal neurons in mammalian brains,and the results are promising for its application in neuroscience studies.
文摘Volume parameter is the basic content of a spatial body object morphology analysis.However,the challenge lies in the volume calculation of irregular objects.The point cloud slicing method proposed in this study effectively works in calculating the volume of the point cloud of the spatial object obtained through three-dimensional laser scanning(3DLS).In this method,a uniformly spaced sequent slicing process is first conducted in a specific direction on the point cloud of the spatial object obtained through 3DLS.A series of discrete point cloud slices corresponding to the point cloud bodies are then obtained.Subsequently,the outline boundary polygon of the point cloud slicing is searched one by one in accordance with the slicing sequence and areas of the polygon.The point cloud slice is also calculated.Finally,the individual point cloud section volume is calculated through the slicing areas and the adjacent slicing gap.Thus,the total volume of the scanned spatial object can be calculated by summing up the individual volumes.According to the results and analysis of the calculated examples,the slice-based volume-calculating method for the point cloud of irregular objects obtained through 3DLS is correct,concise in process,reliable in results,efficient in calculation methods,and controllable on accuracy.This method comes as a good solution to the volume calculation of irregular objects.
基金This research was supported by the BB21 plus funded by Busan Metropolitan City and Busan Institute for Talent and Lifelong Education(BIT)and a grant from Tongmyong University Innovated University Research Park(I-URP)funded by Busan Metropolitan City,Republic of Korea.
文摘The process of segmenting point cloud data into several homogeneous areas with points in the same region having the same attributes is known as 3D segmentation.Segmentation is challenging with point cloud data due to substantial redundancy,fluctuating sample density and lack of apparent organization.The research area has a wide range of robotics applications,including intelligent vehicles,autonomous mapping and navigation.A number of researchers have introduced various methodologies and algorithms.Deep learning has been successfully used to a spectrum of 2D vision domains as a prevailing A.I.methods.However,due to the specific problems of processing point clouds with deep neural networks,deep learning on point clouds is still in its initial stages.This study examines many strategies that have been presented to 3D instance and semantic segmentation and gives a complete assessment of current developments in deep learning-based 3D segmentation.In these approaches’benefits,draw backs,and design mechanisms are studied and addressed.This study evaluates the impact of various segmentation algorithms on competitiveness on various publicly accessible datasets,as well as the most often used pipelines,their advantages and limits,insightful findings and intriguing future research directions.
基金This work was supported by the Scientific Research Program of Hunan Provincial Department of Education(No.22B0258)Natural Science Foundation of Hunan Province(No.2024JJ5649).
文摘Separating wood and leaf components in tree point clouds is one of the key tasks for achieving automated forest inventory and management.To obtain accurate wood-leaf separation results,traditional methods typically rely on large amounts of annotated point cloud data to train supervised semantic segmentation networks.However,point wise annotation is not only extremely labor-intensive but also time-consuming and costly,which greatly limits the widespread application and adoption of supervised learning methods in wood-leaf separation tasks.To eliminate the dependence on annotated point clouds,this study explores the feasibility of wood-leaf separation under completely unsupervised conditions.To this end,we propose an unsupervised semantic segmentation network that is capable of directly extracting wood and leaf components in 3D point clouds.The network adopts a sparse convolutional neural network as the backbone and incorporates two custom-designed modules:the dual point attention(DPA)module and the point cloud feature convolutional integrator(PFCI)module,for enhanced feature fusion and extraction.Semantic classification is then achieved by generating pseudolabels via super point clustering.Based on large-scale public datasets containing coniferous and broadleaf forests,in addition to our self-constructed dataset,our proposed network achieved an overall accuracy(oAcc)of 67.583%,a mean clas-sification accuracy(mAcc)of 50.249%,and a mean intersection over union(mIoU)of 38.512%,and in wood and leaf separation at the tree level,it attained an oAcc of 80.856%,a mAcc of 64.013%,and a mIoU of 49.695%.Across both the forest and tree scenarios,our network outperforms the current state-of-the-art methods,namely,GrowSP and PointDC.Ablation experiments further confirm that each of the proposed modules contributes significantly to improving the segmentation accuracy,and in addition,our segmentation network demonstrates strong robustness even under high occlusion rates and exhibits excellent generalization capability.
基金supported by the National Natural Science Foundation of China(62222602,62302167,62176224,62106075,61972157)the Shanghai Sailing Program(23YF1410500)+3 种基金Natural Science Foundation of Shanghai(23ZR1420400)Natural Science Foundation of Chongqing,China(CSTB2023NSCQ-JQX0007,CSTB2023NSCQ-MSX0137)CCF-Tencent Rhino-Bird Young Faculty Open Research Fund(RAGR20230121)CAAI-Huawei MindSpore Open Fund.
文摘Current outdoor point-cloud segmentation methods typically formulate semantic segmentation as a per-point/voxel-classification task.Although this strategy is straightforward because it classifies each point directly,it ignores the overall relationship of the category.As an alternative paradigm,mask classification decouples category classification from region localization,allowing the model to better capture overall category relationships.In this paper,we propose a novel approach called the point mask transformer(PMFormer),which transforms the semantic segmentation of point clouds from per-point classification to mask classification using a transformer architecture.The proposed model comprises a 3D backbone,transformer decoder,and segmentation head that predicts a series of binary masks,each associated with a global class label.Furthermore,to accommodate the unique characteristics of large and sparse outdoor point-cloud scenes,we propose three enhancements for the integration of point-cloud data with the transformer:MaskMix,3D position encoding,and attention weights.We evaluate our model using the SemanticKITTI and nuScenes datasets.Our experimental results show that the proposed method outperforms state-of-the-art semantic segmentation approaches.
