In this paper,a novel compression framework based on 3D point cloud data is proposed for telepresence,which consists of two parts.One is implemented to remove the spatial redundancy,i.e.,a robust Bayesian framework is...In this paper,a novel compression framework based on 3D point cloud data is proposed for telepresence,which consists of two parts.One is implemented to remove the spatial redundancy,i.e.,a robust Bayesian framework is designed to track the human motion and the 3D point cloud data of the human body is acquired by using the tracking 2D box.The other part is applied to remove the temporal redundancy of the 3D point cloud data.The temporal redundancy between point clouds is removed by using the motion vector,i.e.,the most similar cluster in the previous frame is found for the cluster in the current frame by comparing the cluster feature and the cluster in the current frame is replaced by the motion vector for compressing the current frame.The hrst,the B-SHOT(binary signatures of histograms orientation)descriptor is applied to represent the point feature for matching the corresponding point between two frames.The second,the K-mean algorithm is used to generate the cluster because there are a lot of unsuccessfully matched points in the current frame.The matching operation is exploited to find the corresponding clusters between the point cloud data of two frames.Finally,the cluster information in the current frame is replaced by the motion vector for compressing the current frame and the unsuccessfully matched clusters in the curren t and the motion vectors are transmit ted into the rem ote end.In order to reduce calculation time of the B-SHOT descriptor,we introduce an octree structure into the B-SHOT descriptor.In particular,in order to improve the robustness of the matching operation,we design the cluster feature to estimate the similarity bet ween two clusters.Experimen tai results have shown the bet ter performance of the proposed method due to the lower calculation time and the higher compression ratio.The proposed met hod achieves the compression ratio of 8.42 and the delay time of 1228 ms compared with the compression ratio of 5.99 and the delay time of 2163 ms in the octree-based compression method under conditions of similar distortion rate.展开更多
Automatic Digital Orthophoto Map(DOM)generation plays an important role in many downstream works such as land use and cover detection,urban planning,and disaster assessment.Existing DOM generation methods can generate...Automatic Digital Orthophoto Map(DOM)generation plays an important role in many downstream works such as land use and cover detection,urban planning,and disaster assessment.Existing DOM generation methods can generate promising results but always need ground object filtered DEM generation before otho-rectification;this can consume much time and produce building facade contained results.To address this problem,a pixel-by-pixel digital differential rectification-based automatic DOM generation method is proposed in this paper.Firstly,3D point clouds with texture are generated by dense image matching based on an optical flow field for a stereo pair of images,respectively.Then,the grayscale of the digital differential rectification image is extracted directly from the point clouds element by element according to the nearest neighbor method for matched points.Subsequently,the elevation is repaired grid-by-grid using the multi-layer Locally Refined B-spline(LR-B)interpolation method with triangular mesh constraint for the point clouds void area,and the grayscale is obtained by the indirect scheme of digital differential rectification to generate the pixel-by-pixel digital differentially rectified image of a single image slice.Finally,a seamline network is automatically searched using a disparity map optimization algorithm,and DOM is smartly mosaicked.The qualitative and quantitative experimental results on three datasets were produced and evaluated,which confirmed the feasibility of the proposed method,and the DOM accuracy can reach 1 Ground Sample Distance(GSD)level.The comparison experiment with the state-of-the-art commercial softwares showed that the proposed method generated DOM has a better visual effect on building boundaries and roof completeness with comparable accuracy and computational efficiency.展开更多
Generating selfie images on the surface of a celestial body poses several challenges,including the position of the robotic arm,camera field of view,and limited shooting time.To address these challenges,the PCMIS(3D Po...Generating selfie images on the surface of a celestial body poses several challenges,including the position of the robotic arm,camera field of view,and limited shooting time.To address these challenges,the PCMIS(3D Point Cloud Matching Based Image Stitching)algorithm is designed,along with a corresponding shooting plan.This algorithm establishes a correspondence between depth and color information,enabling the generation of stitching views under any given view parameter.Furthermore,the algorithm is accelerated using GPU processing,resulting in a significant reduction in stitching time.The algorithm is successfully applied to generate selfie images for the Chang'e-5 mission.展开更多
With the rapid advancement of mechanical automation and intelligent processing technology,ac-curately measuring the surfaces of complex parts has emerged as a significant research challenge.Robotic measurement technol...With the rapid advancement of mechanical automation and intelligent processing technology,ac-curately measuring the surfaces of complex parts has emerged as a significant research challenge.Robotic measurement technology plays a crucial role in facilitating rapid quality inspections during the manufacturing process due to its inherent flexibility.However,the irregular shapes and viewpoint occlusions of complex parts complicate precise measurement.To address these challenges,this work proposes a point cloud registration network for robotic scanning systems and introduces a DBR-Net(Dual-line Registration Network)to overcome the issues of low overlap rates and perspective occlusion that currently impede the registration of certain workpieces.First,feature extraction is performed using a bilinear encoder and multi-level feature interactions of both point-wise and global features.Subsequently,the features are sampled through unanimous voting and fed into the RANSAC(Random Sample Consensus)algorithm for pose estimation,enabling multi-view point cloud registration.Experimental results demonstrate that this method significantly outperforms many existing techniques in terms of feature extraction and registration accuracy,thereby enhancing the overall performance of point cloud registration.展开更多
When obtaining three-dimensional(3D)face point cloud data based on structured light,factors related to the environment,occlusion,and illumination intensity lead to holes in the collected data,which affect subsequent r...When obtaining three-dimensional(3D)face point cloud data based on structured light,factors related to the environment,occlusion,and illumination intensity lead to holes in the collected data,which affect subsequent recognition.In this study,we propose a hole-filling method based on stereo-matching technology combined with a B-spline.The algorithm uses phase information acquired during raster projection to locate holes in the point cloud,simultaneously extracting boundary point cloud sets.By registering the face point cloud data using the stereo-matching algorithm and the data collected using the raster projection method,some supplementary information points can be obtained at the holes.The shape of the B-spline curve can then be roughly described by a few key points,and the control points are put into the hole area as key points for iterative calculation of surface reconstruction.Simulations using smooth ceramic cups and human face models showed that our model can accurately reproduce details and accurately restore complex shapes on the test surfaces.Simulation results indicated the robustness of the method,which is able to fill holes on complex areas such as the inner side of the nose without a prior model.This approach also effectively supplements the hole information,and the patched point cloud is closer to the original data.This method could be used across a wide range of applications requiring accurate facial recognition.展开更多
This paper surveys state-of-the-art image features and descriptors for the task of 3D scan registration based on panoramic reflectance images.As modern terrestrial laser scanners digitize their environment in a spheri...This paper surveys state-of-the-art image features and descriptors for the task of 3D scan registration based on panoramic reflectance images.As modern terrestrial laser scanners digitize their environment in a spherical way,the sphere has to be projected to a two-dimensional image.To this end,we evaluate the equirectangular,the cylindrical,the Mercator,the rectilinear,the Pannini,the stereographic,and the z-axis projection.We show that the Mercator and the Pannini projection outperform the other projection methods.展开更多
Multi-view laser radar （ladar） data registration in obscure environments is an important research field of obscured target detection from air to ground. There are few overlap regions of the observational data in dif...Multi-view laser radar （ladar） data registration in obscure environments is an important research field of obscured target detection from air to ground. There are few overlap regions of the observational data in different views because of the occluder, so the multi-view data registration is rather difficult. Through indepth analyses of the typical methods and problems, it is obtained that the sequence registration is more appropriate, but needs to improve the registration accuracy. On this basis, a multi-view data registration algorithm based on aggregating the adjacent frames, which are already registered, is proposed. It increases the overlap region between the pending registration frames by aggregation and further improves the registration accuracy. The experiment results show that the proposed algorithm can effectively register the multi-view ladar data in the obscure environment, and it also has a greater robustness and a higher registration accuracy compared with the sequence registration under the condition of equivalent operating efficiency.展开更多
Point clouds are widely used in Earth surface research but usually exhibit gaps of missing data.Previous point cloud restoration methods used in terrain modelling have not fully considered complex terrain characterist...Point clouds are widely used in Earth surface research but usually exhibit gaps of missing data.Previous point cloud restoration methods used in terrain modelling have not fully considered complex terrain characteristics,which can be summarised as the controlling role of topographic features in shaping terrain surfaces and the inherent similarities observed among these surfaces.This work introduces a novel method that integrates Topographic Features and Patch Matching(TFPM)into point cloud restoration processes for terrain modelling.The method mainly contains three steps.First,identifying gap boundary points.Second,topographic feature points are extracted and subsequently interpolated into the identified gaps.Third,searching other parts of the raw point cloud for patches resembling the gaps,and the identified patches are used as templates to restore the point cloud.The proposed method is benchmarked against three state-of-the-art point cloud restoration methods.The experimental results demonstrate that the TFPM method consistently exhibits superior accuracy in terrain modelling and analysis,as evidenced by low values of the root mean square error,average elevation difference,and average slope difference.This work endeavours to incorporate topographic features into point cloud restoration processes and can benefit future research related to terrain modelling and analysis.展开更多
基金This work was supported by National Nature Science Foundation of China(No.61811530281 and 61861136009)Guangdong Regional Joint Foundation(No.2019B1515120076)the Fundamental Research for the Central Universities.
文摘In this paper,a novel compression framework based on 3D point cloud data is proposed for telepresence,which consists of two parts.One is implemented to remove the spatial redundancy,i.e.,a robust Bayesian framework is designed to track the human motion and the 3D point cloud data of the human body is acquired by using the tracking 2D box.The other part is applied to remove the temporal redundancy of the 3D point cloud data.The temporal redundancy between point clouds is removed by using the motion vector,i.e.,the most similar cluster in the previous frame is found for the cluster in the current frame by comparing the cluster feature and the cluster in the current frame is replaced by the motion vector for compressing the current frame.The hrst,the B-SHOT(binary signatures of histograms orientation)descriptor is applied to represent the point feature for matching the corresponding point between two frames.The second,the K-mean algorithm is used to generate the cluster because there are a lot of unsuccessfully matched points in the current frame.The matching operation is exploited to find the corresponding clusters between the point cloud data of two frames.Finally,the cluster information in the current frame is replaced by the motion vector for compressing the current frame and the unsuccessfully matched clusters in the curren t and the motion vectors are transmit ted into the rem ote end.In order to reduce calculation time of the B-SHOT descriptor,we introduce an octree structure into the B-SHOT descriptor.In particular,in order to improve the robustness of the matching operation,we design the cluster feature to estimate the similarity bet ween two clusters.Experimen tai results have shown the bet ter performance of the proposed method due to the lower calculation time and the higher compression ratio.The proposed met hod achieves the compression ratio of 8.42 and the delay time of 1228 ms compared with the compression ratio of 5.99 and the delay time of 2163 ms in the octree-based compression method under conditions of similar distortion rate.
基金supported by the National Natural Science Foundation of China[Grant No.41771479]the National High-Resolution Earth Observation System(the Civil Part)[Grant No.50-H31D01-0508-13/15]the Japan Society for the Promotion of Science[Grant No.22H03573].
文摘Automatic Digital Orthophoto Map(DOM)generation plays an important role in many downstream works such as land use and cover detection,urban planning,and disaster assessment.Existing DOM generation methods can generate promising results but always need ground object filtered DEM generation before otho-rectification;this can consume much time and produce building facade contained results.To address this problem,a pixel-by-pixel digital differential rectification-based automatic DOM generation method is proposed in this paper.Firstly,3D point clouds with texture are generated by dense image matching based on an optical flow field for a stereo pair of images,respectively.Then,the grayscale of the digital differential rectification image is extracted directly from the point clouds element by element according to the nearest neighbor method for matched points.Subsequently,the elevation is repaired grid-by-grid using the multi-layer Locally Refined B-spline(LR-B)interpolation method with triangular mesh constraint for the point clouds void area,and the grayscale is obtained by the indirect scheme of digital differential rectification to generate the pixel-by-pixel digital differentially rectified image of a single image slice.Finally,a seamline network is automatically searched using a disparity map optimization algorithm,and DOM is smartly mosaicked.The qualitative and quantitative experimental results on three datasets were produced and evaluated,which confirmed the feasibility of the proposed method,and the DOM accuracy can reach 1 Ground Sample Distance(GSD)level.The comparison experiment with the state-of-the-art commercial softwares showed that the proposed method generated DOM has a better visual effect on building boundaries and roof completeness with comparable accuracy and computational efficiency.
基金supported by the Leading Goose Research and Development Program of Zhejiang Province of China under Grant No.2024C01103.
文摘Generating selfie images on the surface of a celestial body poses several challenges,including the position of the robotic arm,camera field of view,and limited shooting time.To address these challenges,the PCMIS(3D Point Cloud Matching Based Image Stitching)algorithm is designed,along with a corresponding shooting plan.This algorithm establishes a correspondence between depth and color information,enabling the generation of stitching views under any given view parameter.Furthermore,the algorithm is accelerated using GPU processing,resulting in a significant reduction in stitching time.The algorithm is successfully applied to generate selfie images for the Chang'e-5 mission.
基金co-supported by the National Natural Science Foundation of China(U22A20176)Guangdong Basic and Applied Basic Research Foundation,China(2022B1515120078)+2 种基金the Guangdong Basic and Applied Basic Research Foundation,China(2021A1515110898)GDAS’Project of Science and Technology Development,China(2022GDASZH-2022010108)the Key Areas R&D Program of Dongguan City,China(20201200300062).
文摘With the rapid advancement of mechanical automation and intelligent processing technology,ac-curately measuring the surfaces of complex parts has emerged as a significant research challenge.Robotic measurement technology plays a crucial role in facilitating rapid quality inspections during the manufacturing process due to its inherent flexibility.However,the irregular shapes and viewpoint occlusions of complex parts complicate precise measurement.To address these challenges,this work proposes a point cloud registration network for robotic scanning systems and introduces a DBR-Net(Dual-line Registration Network)to overcome the issues of low overlap rates and perspective occlusion that currently impede the registration of certain workpieces.First,feature extraction is performed using a bilinear encoder and multi-level feature interactions of both point-wise and global features.Subsequently,the features are sampled through unanimous voting and fed into the RANSAC(Random Sample Consensus)algorithm for pose estimation,enabling multi-view point cloud registration.Experimental results demonstrate that this method significantly outperforms many existing techniques in terms of feature extraction and registration accuracy,thereby enhancing the overall performance of point cloud registration.
基金supported by the National Natural Science Foundation of China(No.61405034)the Special Project on Basic Research of Frontier Leading Technology of Jiangsu Province,China(No.BK20192004C)+1 种基金the Shenzhen Science and Technology Innovation Committee(No.JCYJ20180306174455080)the Natural Science Foundation of Jiangsu Province,China(No.BK20181269)。
文摘When obtaining three-dimensional(3D)face point cloud data based on structured light,factors related to the environment,occlusion,and illumination intensity lead to holes in the collected data,which affect subsequent recognition.In this study,we propose a hole-filling method based on stereo-matching technology combined with a B-spline.The algorithm uses phase information acquired during raster projection to locate holes in the point cloud,simultaneously extracting boundary point cloud sets.By registering the face point cloud data using the stereo-matching algorithm and the data collected using the raster projection method,some supplementary information points can be obtained at the holes.The shape of the B-spline curve can then be roughly described by a few key points,and the control points are put into the hole area as key points for iterative calculation of surface reconstruction.Simulations using smooth ceramic cups and human face models showed that our model can accurately reproduce details and accurately restore complex shapes on the test surfaces.Simulation results indicated the robustness of the method,which is able to fill holes on complex areas such as the inner side of the nose without a prior model.This approach also effectively supplements the hole information,and the patched point cloud is closer to the original data.This method could be used across a wide range of applications requiring accurate facial recognition.
文摘This paper surveys state-of-the-art image features and descriptors for the task of 3D scan registration based on panoramic reflectance images.As modern terrestrial laser scanners digitize their environment in a spherical way,the sphere has to be projected to a two-dimensional image.To this end,we evaluate the equirectangular,the cylindrical,the Mercator,the rectilinear,the Pannini,the stereographic,and the z-axis projection.We show that the Mercator and the Pannini projection outperform the other projection methods.
文摘Multi-view laser radar （ladar） data registration in obscure environments is an important research field of obscured target detection from air to ground. There are few overlap regions of the observational data in different views because of the occluder, so the multi-view data registration is rather difficult. Through indepth analyses of the typical methods and problems, it is obtained that the sequence registration is more appropriate, but needs to improve the registration accuracy. On this basis, a multi-view data registration algorithm based on aggregating the adjacent frames, which are already registered, is proposed. It increases the overlap region between the pending registration frames by aggregation and further improves the registration accuracy. The experiment results show that the proposed algorithm can effectively register the multi-view ladar data in the obscure environment, and it also has a greater robustness and a higher registration accuracy compared with the sequence registration under the condition of equivalent operating efficiency.
基金supported by the National Natural Science Foundation of China under Grant[41971333,41930102,42371407]Priority Academic Programme Development of Jiangsu Higher Education Institutions under Grant[164320H116]The Priority Academic Program Development of Jiangsu Higher Education Institutions and the Deep-time Digital Earth(DDE)Big Science Program.
文摘Point clouds are widely used in Earth surface research but usually exhibit gaps of missing data.Previous point cloud restoration methods used in terrain modelling have not fully considered complex terrain characteristics,which can be summarised as the controlling role of topographic features in shaping terrain surfaces and the inherent similarities observed among these surfaces.This work introduces a novel method that integrates Topographic Features and Patch Matching(TFPM)into point cloud restoration processes for terrain modelling.The method mainly contains three steps.First,identifying gap boundary points.Second,topographic feature points are extracted and subsequently interpolated into the identified gaps.Third,searching other parts of the raw point cloud for patches resembling the gaps,and the identified patches are used as templates to restore the point cloud.The proposed method is benchmarked against three state-of-the-art point cloud restoration methods.The experimental results demonstrate that the TFPM method consistently exhibits superior accuracy in terrain modelling and analysis,as evidenced by low values of the root mean square error,average elevation difference,and average slope difference.This work endeavours to incorporate topographic features into point cloud restoration processes and can benefit future research related to terrain modelling and analysis.
