Recent advancements in artificial intelligence have transformed three-dimensional(3D)optical imaging and metrology,enabling high-resolution and high-precision 3D surface geometry measurements from one single fringe pa...Recent advancements in artificial intelligence have transformed three-dimensional(3D)optical imaging and metrology,enabling high-resolution and high-precision 3D surface geometry measurements from one single fringe pattern projection.However,the imaging speed of conventional fringe projection profilometry(FPP)remains limited by the native sensor refresh rates due to the inherent"one-to-one"synchronization mechanism between pattern projection and image acquisition in standard structured light techniques.Here,we present dual-frequency angular-multiplexed fringe projection profilometry(DFAMFPP),a deep learning-enabled 3D imaging technique that achieves high-speed,high-precision,and large-depth-range absolute 3D surface measurements at speeds 16 times faster than the sensor's native frame rate.By encoding multi-timeframe 3D information into a single multiplexed image using multiple pairs of dual-frequency fringes,high-accuracy absolute phase maps are reconstructed using specially trained two-stage number-theoretical-based deep neural networks.We validate the effectiveness of DFAMFPP through dynamic scene measurements,achieving 10,000 Hz 3D imaging of a running turbofan engine prototype with only a 625 Hz camera.By overcoming the sensor hardware bottleneck,DFAMFPP significantly advances high-speed and ultra-high-speed 3D imaging,opening new avenues for exploring dynamic processes across diverse scientific disciplines.展开更多
Grating fringe projection 3D measurement techniques are extensively applied in various fields.However,in high dynamic range scenarios with significant surface reflectivity variations,uneven greyscale distribution may ...Grating fringe projection 3D measurement techniques are extensively applied in various fields.However,in high dynamic range scenarios with significant surface reflectivity variations,uneven greyscale distribution may lead to phase errors and poor reconstruction results.To address this problem,an adaptive fringe projection method is introduced.The method involves projecting two sets of dark and light fringes onto the object,enabling the full-field projection intensity map to be generated adaptively based on greyscale analysis.First,dark fringes are projected onto the object to extend exposure time as long as possible without causing overexposure in the image.Subsequently,bright fringes are projected under the same exposure settings to detect overexposed pixels,and the greyscale distribution of these overexposed points from the previous dark fringe projection is analyzed to calculate the corresponding projection intensities.Finally,absolute phase information from orthogonal fringes is used for coordinate matching,enabling the generation of adaptive projection fringe patterns.Experiments on various high dynamic range objects show that compared to conventional fringe projection binocular reconstruction method,the proposed algorithm achieves complete reconstruction of high dynamic range surfaces and shows robust performance against phase calculation errors caused by overexposure and low modulation.展开更多
This paper discusses current research and development of fringe projection-based techniques. A system based on Fourier transform profilometry (FTP) is proposed for three-dimensional (3D) shape recovery. The system imp...This paper discusses current research and development of fringe projection-based techniques. A system based on Fourier transform profilometry (FTP) is proposed for three-dimensional (3D) shape recovery. The system improves the method of phase unwrapping to gain accurate 3D shapes of objects. The method uses a region-growing algorithm for the path prediction guided by the quality map to increase the recovering accuracy and provides a fast and simple tool for 3D shape recovery. The shape measurement and data recovery are integrated to offer a new method of 3D modelling. Examples are presented to verify the feasibility of the proposed method.展开更多
Single-shot high-speed 3D imaging is important for reconstructions of dynamic objects.For fringe projection profilometry(FPP),however,it is still challenging to recover accurate 3D shapes of isolated objects by a sing...Single-shot high-speed 3D imaging is important for reconstructions of dynamic objects.For fringe projection profilometry(FPP),however,it is still challenging to recover accurate 3D shapes of isolated objects by a single fringe image.In this paper,we demonstrate that the deep neural networks can be trained to directly recover the absolute phase from a unique fringe image that involves spatially multiplexed fringe patterns of different frequencies.The extracted phase is free from spectrum-aliasing problem which is hard to avoid for traditional spatial-multiplexing methods.Experiments on both static and dynamic scenes show that the proposed approach is robust to object motion and can obtain high-quality 3D reconstructions of isolated objects within a single fringe image.展开更多
Although the structured light system that uses digital fringe projection has been widely implemented in three-dimensional surface profile measurement, the measurement system is susceptible to non-linear error. In this...Although the structured light system that uses digital fringe projection has been widely implemented in three-dimensional surface profile measurement, the measurement system is susceptible to non-linear error. In this work, we propose a convenient look-up-table-based （LUT-based） method to compensate for the non-linear error in captured fringe patterns. Without extra calibration, this LUT-based method completely utilizes the captured fringe pattern by recording the full-field differences. Then, a phase compensation map is established to revise the measured phase. Experimental results demonstrate that this method works effectively.展开更多
We describe a 2D fringe projection method that involves projecting two groups of sharp comb fringes onto a free-flying hawk-moth from different directions and recording the images of distorted fringes by two high spee...We describe a 2D fringe projection method that involves projecting two groups of sharp comb fringes onto a free-flying hawk-moth from different directions and recording the images of distorted fringes by two high speed cameras from two orthogonal views. By calculating the 3D coordinates of the points on the hawk-moth and three-dimensional reconstruction of the wing, the flight trajectory, body attitude and wing kinematics including flapping angle, elevation angle, torsion angle, and camber deformation are obtained.展开更多
Fringe projection profilometry(FPP)has been widely applied to non-contact three-dimensional measurement in industries owing to its high accuracy and speed.The point cloud,which is a measurement result of the FPP syste...Fringe projection profilometry(FPP)has been widely applied to non-contact three-dimensional measurement in industries owing to its high accuracy and speed.The point cloud,which is a measurement result of the FPP system,typically contains a large number of invalid points caused by the background,ambient light,shadows,and object edge regions.Research on noisy point detection and elimination has been conducted over the past two decades.However,existing invalid point removal methods are based on image intensity analysis and are only applicable to simple measurement backgrounds that are purely dark.In this paper,we propose a novel invalid point removal framework that consists of two aspects:(1)A convolutional neural network(CNN)is designed to segment the foreground from the background of different intensity conditions in FPP measurement circumstances to remove background points and the most discrete points in background regions.(2)A two-step method based on the fringe image intensity threshold and a bilateral filter is proposed to eliminate the small number of discrete points remaining after background segmentation caused by shadows and edge areas on objects.Experimental results verify that the proposed framework(1)can remove background points intelligently and accurately in different types of complex circumstances,and(2)performs excellently in discrete point detection from object regions.展开更多
Phase unwrapping is one of the key roles in fringe projection three-dimensional(3D)measurement technology.We propose a new method to achieve phase unwrapping in camera array light filed fringe projection 3D measuremen...Phase unwrapping is one of the key roles in fringe projection three-dimensional(3D)measurement technology.We propose a new method to achieve phase unwrapping in camera array light filed fringe projection 3D measurement based on deep learning.A multi-stream convolutional neural network(CNN)is proposed to learn the mapping relationship between camera array light filed wrapped phases and fringe orders of the expected central view,and is used to predict the fringe order to achieve the phase unwrapping.Experiments are performed on the light field fringe projection data generated by the simulated camera array fringe projection measurement system in Blender and by the experimental 3×3 camera array light field fringe projection system.The performance of the proposed network with light field wrapped phases using multiple directions as network input data is studied,and the advantages of phase unwrapping based on deep learning in light filed fringe projection are demonstrated.展开更多
Acquisition of physical data with high precision is a key step in reverse engineering (RE). It is an important stimulative for the progress of reverse engineering with which various digitizing devices are invent ed,...Acquisition of physical data with high precision is a key step in reverse engineering (RE). It is an important stimulative for the progress of reverse engineering with which various digitizing devices are invent ed, developed and made applicable. This paper introduces a three dimensional opt ical measurement method based on digital fringe projection technique in RE to im prove the technique through its application. A practical example is presented an d the result demonstrates the applicability and feasibility of the measurement s ystem as well as the reliability and validity of relevant methods and algorithms .展开更多
In order to get the entire data in the optical measurement, a multi-view three-dimensional(3D) measurement method based on turntable is proposed. In the method, a turntable is used to rotate the object and obtain mult...In order to get the entire data in the optical measurement, a multi-view three-dimensional(3D) measurement method based on turntable is proposed. In the method, a turntable is used to rotate the object and obtain multi-view point cloud data, and then multi-view point cloud data are registered and integrated into a 3D model. The measurement results are compared with that of the sticking marked point method. Experimental results show that the measurement process of the proposed method is simpler, and the scanning speed and accuracy are improved.展开更多
Fringe projection profilometry(FPP)is a method that determines height by analyzing distortional fringes,which is widely used in high-accuracy 3D imaging.Now,one major reason limiting imaging speed in FPP is the projec...Fringe projection profilometry(FPP)is a method that determines height by analyzing distortional fringes,which is widely used in high-accuracy 3D imaging.Now,one major reason limiting imaging speed in FPP is the projection device;the capture speed of high-speed cameras far exceeds the projection frequency.Among various devices,an LED array can exceed the speed of a high-speed camera.However,non-sinusoidal fringe patterns in the LED array systems can arise from several factors that will reduce the accuracy,such as the spacing between adjacent LEDs,the inconsistency in brightness across different LEDs,and the residual high-order harmonics in binary defocusing projection.It is challenging to resolve by other methods.In this paper,we propose a method that creates a look-up table using system calibration data of phase-height models.Then we utilize the look-up table to compensate for the phase error during the reconstructing process.The foundation of the proposed method relies on the time-invariance of systematic error;any factor that impacts the sinusoidal characteristic would present as an anomaly in the unwrapped phase.Experiments have demonstrated that the root mean square errors(RMSEs)of the results yielded by the proposed method were reduced by over 90%compared to those yielded by the traditional method,reaching 20μm accuracy.This paper offers an alternative approach for high-speed and high-accuracy 3D imaging with an LED array and presents a workable solution for addressing complex errors from non-sinusoidal fringes.展开更多
Fringe projection profilometry,a powerful technique for three-dimensional(3D)imaging and measurement,has been revolutionized by deep learning,achieving speeds of up to 100,000 frames per second(fps)while preserving hi...Fringe projection profilometry,a powerful technique for three-dimensional(3D)imaging and measurement,has been revolutionized by deep learning,achieving speeds of up to 100,000 frames per second(fps)while preserving highresolution.This advancement expands its applications to high-speed transient scenarios,opening new possibilities for ultrafast 3D measurements.展开更多
Fringe projection profilometry(FPP)has been extensively studied in the field of three-dimensional(3D)measurement.Although FPP always uses high-frequency fringes to ensure high measurement accuracy,too many patterns ar...Fringe projection profilometry(FPP)has been extensively studied in the field of three-dimensional(3D)measurement.Although FPP always uses high-frequency fringes to ensure high measurement accuracy,too many patterns are projected to unwrap the phase,which affects the speed of 3D reconstruction.We propose a high-speed 3D shape measurement method using only three high-frequency inner shifting-phase patterns(70 periods),which satisfies both high precision and high measuring speed requirements.Besides,our proposed method obtains the wrapped phase and the fringe order simultaneously without any other information and constraints.The proposed method has successfully reconstructed moving objects with high speed at the camera's full frame rate(1700 frames per second).展开更多
Recent advances in imaging sensors and digital light projection technology have facilitated rapid progress in 3D optical sensing,enabling 3D surfaces of complexshaped objects to be captured with high resolution and ac...Recent advances in imaging sensors and digital light projection technology have facilitated rapid progress in 3D optical sensing,enabling 3D surfaces of complexshaped objects to be captured with high resolution and accuracy.Nevertheless,due to the inherent synchronous pattern projection and image acquisition mechanism,the temporal resolution of conventional structured light or fringe projection profilometry(FPP)based 3D imaging methods is still limited to the native detector frame rates.In this work,we demonstrate a new 3D imaging method,termed deep-learning-enabled multiplexed FPP(DLMFPP),that allows to achieve high-resolution and high-speed 3D imaging at near-one-order of magnitude-higher 3D frame rate with conventional low-speed cameras.By encoding temporal information in one multiplexed fringe pattern,DLMFPP harnesses deep neural networks embedded with Fourier transform,phase-shifting and ensemble learning to decompose the pattern and analyze separate fringes,furnishing a high signal-to-noise ratio and a ready-to-implement solution over conventional computational imaging techniques.We demonstrate this method by measuring different types of transient scenes,including rotating fan blades and bullet fired from a toy gun,at kHz using cameras of around 100 Hz.Experiential results establish that DLMFPP allows slow-scan cameras with their known advantages in terms of cost and spatial resolution to be used for high-speed 3D imaging tasks.展开更多
In 2019,the Event Horizon Telescope(EHT)released the first-ever image of a black hole event horizon.Astronomers are now aiming for higher angular resolutions of distant targets,like black holes,to understand more abou...In 2019,the Event Horizon Telescope(EHT)released the first-ever image of a black hole event horizon.Astronomers are now aiming for higher angular resolutions of distant targets,like black holes,to understand more about the fundamental laws of gravity that govern our universe.To achieve this higher resolution and increased sensitivity,larger radio telescopes are needed to operate at higher frequencies and in larger quantities.Projects like the next-generation Very Large Array(ngVLA)and the Square-Kilometer Array(SKA)require building hundreds of telescopes with diameters greater than 10 ms over the next decade.This has a twofold effect.Radio telescope surfaces need to be more accurate to operate at higher frequencies,and the logistics involved in maintaining a radio telescope need to be simplified to support them properly in large quantities.Both of these problems can be solved with improved methods for surface metrology that are faster and more accurate with a higher resolution.This leads to faster and more accurate panel alignment and,therefore,a more productive observatory.In this paper,we present the use of binocular fringe projection profilometry as a solution to this problem and demonstrate it by aligning two panels on a 3-m radio telescope dish.The measurement takes only 10 min and directly delivers feedback on the tip,tilt,and piston of each panel to create the ideal reflector shape.展开更多
In the three-dimensional(3D) contour measurement,the phase shift profilometry(PSP) method is the most widely used one.However,the measurement speed of PSP is very low because of the multiple projections.In order to im...In the three-dimensional(3D) contour measurement,the phase shift profilometry(PSP) method is the most widely used one.However,the measurement speed of PSP is very low because of the multiple projections.In order to improve the measurement speed,color grating stripes are used for measurement in this paper.During the measurement,only one color sinusoidal fringe is projected on the measured object.Therefore,the measurement speed is greatly improved.Since there is coupling or interference phenomenon between the adjacent color grating stripes,a color correction method is used to improve the measurement results.A method for correcting nonlinear error of measurement system is proposed in this paper,and the sinusoidal property of acquired image after correction is better than that before correction.Experimental results show that with these correction methods,the measurement errors can be reduced.Therefore,it can support a good foundation for the high-precision 3D reconstruction.展开更多
To measure breast basic dimension by using computer-aided projection fringe system.Methods A system has been developed for measuring breast basic dimension based on computer-aided projection fringe measurement and pro...To measure breast basic dimension by using computer-aided projection fringe system.Methods A system has been developed for measuring breast basic dimension based on computer-aided projection fringe measurement and programming software.Plastic manikins breast’s SN-N (sternal notch to nipple distance),N-ML (nipple to midline distance),N-N (internipple distance),MBW (base width of breast) and N-IMF (nipple to inframammary fold distance) are measured with this system.At the same time,these items are also measured with routine ruler.Results This study indicate that the system has some merits:① non-touching measurement;② it is very rapid,the patient measured need hold his breath only 0.5 second,and all the time it takes is about 2.5 minutes;③ the measurement’s sensitivity is as high as to 0.6 mm,which meets the clinic requirement entirely;④ the measurement’s accuracy of the system is not significantly when comparing to the routine ruler’s.Conclusion Computer-adided projection fringe system for measuring breast basic dimension is feasible and advanced.14 refs,1 fig.展开更多
The Fringe Projection Profilometry(FPP)system with a single exposure time or a single projection intensity is limited by the dynamic range of the camera,which can lead to overexposure and underexposure of the image,re...The Fringe Projection Profilometry(FPP)system with a single exposure time or a single projection intensity is limited by the dynamic range of the camera,which can lead to overexposure and underexposure of the image,resulting in point cloud loss or reduced accuracy.To address this issue,unlike the pixel modulation method of projectors,we utilize the characteristics of color projectors where the intensity of the three-channel LED can be controlled independently.We propose a method for separating the projector's three-channel light intensity,combined with a color camera,to achieve single exposure and multi-intensity image acquisition.Further,the crosstalk coefficient is applied to predict the three-channel reflectance of the measured object.By integrating clustering and channel mapping,we establish a pixel-level mapping model between the projector's three-channel current and the camera's three-channel image intensity,which realizes the optimal projection current prediction and the high dynamic range(HDR)image acquisition.The proposed method allows for high-precision three-dimensional(3D)data acquisition of HDR scenes with a single exposure.The effectiveness of this method has been validated through experiments with standard planes and standard steps,showing a significant reduction in mean absolute error(44.6%)compared to existing singleexposure HDR methods.Additionally,the number of images required for acquisition is significantly reduced(by 70.8%)compared to multi-exposure fusion methods.This proposed method has great potential in various FPP-related fields.展开更多
A fast 3D reconstruction method based on structured light to measure various parameters of the raceway groove is presented. Digital parallel grating stripes distributed with sine density are projected onto the raceway...A fast 3D reconstruction method based on structured light to measure various parameters of the raceway groove is presented. Digital parallel grating stripes distributed with sine density are projected onto the raceway groove by a DLP projector, and distorting of stripes is happened on the raceway. Simultaneously, aided by three-step phase-shifting approach, three images covered by different stripes are obtained by a high-resolution CCD camera at the same location, thus a more accuracy local topography can be obtained. And then the bearing is rotated on a high precision computer controlled rotational stage. Three images are also obtained as the former step at next planned location triggered by the motor. After one cycle, all images information is combined through the mosaics. As a result, the 3D information of raceway groove can be gained. Not only geometric properties but also surface flaws can be extracted by software. A preliminary hardware system has been built, with which some geometric parameters have been extracted from reconstructed local topography.展开更多
The knowledge of wing orientation and deformation during flapping flight is necessary for a complete aerodynamic analysis, but to date those kinematic features have not been simultaneously quantified for free-flying i...The knowledge of wing orientation and deformation during flapping flight is necessary for a complete aerodynamic analysis, but to date those kinematic features have not been simultaneously quantified for free-flying insects. A projected comb-fringe (PCF) method has been developed for measuring spanwise camber changes on free-flying dragonflies and on beating-flying dragonflies through the course of a wingbeat, which bases on projecting a fringe pattern over the whole measurement area and then measuring the wing deformation from the distorted fringe pattern. Experimental results demonstrate substantial camber changes both along the wingspan and through the course of a wingbeat. The ratio of camber deformation to chord length for hind wing is up to 0.11 at 75% spanwise with a flapping angle of -0.66 degree for a free-flying dragonfly.展开更多
基金supported by National Key Research and Development Program of China(2022YFB2804603,2022YFB2804605)National Natural Science Foundation of China(U21B2033)+4 种基金Fundamental Research Funds forthe Central Universities(2023102001,2024202002)National Key Laborato-ry of Shock Wave and Detonation Physics(JCKYS2024212111)China Post-doctoral Science Fund(2023T160318)Open Research Fund of JiangsuKey Laboratory of Spectral Imaging&Intelligent Sense(JSGP202105,JSGP202201)Postgraduate Research&Practice Innovation Program of Jiangsu Province(KYCX25_0695,SJCX25_0188)。
文摘Recent advancements in artificial intelligence have transformed three-dimensional(3D)optical imaging and metrology,enabling high-resolution and high-precision 3D surface geometry measurements from one single fringe pattern projection.However,the imaging speed of conventional fringe projection profilometry(FPP)remains limited by the native sensor refresh rates due to the inherent"one-to-one"synchronization mechanism between pattern projection and image acquisition in standard structured light techniques.Here,we present dual-frequency angular-multiplexed fringe projection profilometry(DFAMFPP),a deep learning-enabled 3D imaging technique that achieves high-speed,high-precision,and large-depth-range absolute 3D surface measurements at speeds 16 times faster than the sensor's native frame rate.By encoding multi-timeframe 3D information into a single multiplexed image using multiple pairs of dual-frequency fringes,high-accuracy absolute phase maps are reconstructed using specially trained two-stage number-theoretical-based deep neural networks.We validate the effectiveness of DFAMFPP through dynamic scene measurements,achieving 10,000 Hz 3D imaging of a running turbofan engine prototype with only a 625 Hz camera.By overcoming the sensor hardware bottleneck,DFAMFPP significantly advances high-speed and ultra-high-speed 3D imaging,opening new avenues for exploring dynamic processes across diverse scientific disciplines.
基金supported by the Science and Technology Program Project of Tianjin(No.24ZXZSSS00300).
文摘Grating fringe projection 3D measurement techniques are extensively applied in various fields.However,in high dynamic range scenarios with significant surface reflectivity variations,uneven greyscale distribution may lead to phase errors and poor reconstruction results.To address this problem,an adaptive fringe projection method is introduced.The method involves projecting two sets of dark and light fringes onto the object,enabling the full-field projection intensity map to be generated adaptively based on greyscale analysis.First,dark fringes are projected onto the object to extend exposure time as long as possible without causing overexposure in the image.Subsequently,bright fringes are projected under the same exposure settings to detect overexposed pixels,and the greyscale distribution of these overexposed points from the previous dark fringe projection is analyzed to calculate the corresponding projection intensities.Finally,absolute phase information from orthogonal fringes is used for coordinate matching,enabling the generation of adaptive projection fringe patterns.Experiments on various high dynamic range objects show that compared to conventional fringe projection binocular reconstruction method,the proposed algorithm achieves complete reconstruction of high dynamic range surfaces and shows robust performance against phase calculation errors caused by overexposure and low modulation.
基金Project (No. 59965003) supported by the National Natural ScienceFoundation of China
文摘This paper discusses current research and development of fringe projection-based techniques. A system based on Fourier transform profilometry (FTP) is proposed for three-dimensional (3D) shape recovery. The system improves the method of phase unwrapping to gain accurate 3D shapes of objects. The method uses a region-growing algorithm for the path prediction guided by the quality map to increase the recovering accuracy and provides a fast and simple tool for 3D shape recovery. The shape measurement and data recovery are integrated to offer a new method of 3D modelling. Examples are presented to verify the feasibility of the proposed method.
基金This work was supported by National Natural Science Foundation of China(62075096,62005121,U21B2033)Leading Technology of Jiangsu Basic Research Plan(BK20192003)+4 种基金“333 Engineering”Research Project of Jiangsu Province(BRA2016407)Jiangsu Provincial“One belt and one road”innovation cooperation project(BZ2020007)Fundamental Research Funds for the Central Universities(30921011208,30919011222,30920032101)Postgraduate Research&Practice Innovation Program of Jiangsu Province(KYCX21_0273)Open Research Fund of Jiangsu Key Laboratory of Spectral Imaging&Intelligent Sense(JSGP202105).
文摘Single-shot high-speed 3D imaging is important for reconstructions of dynamic objects.For fringe projection profilometry(FPP),however,it is still challenging to recover accurate 3D shapes of isolated objects by a single fringe image.In this paper,we demonstrate that the deep neural networks can be trained to directly recover the absolute phase from a unique fringe image that involves spatially multiplexed fringe patterns of different frequencies.The extracted phase is free from spectrum-aliasing problem which is hard to avoid for traditional spatial-multiplexing methods.Experiments on both static and dynamic scenes show that the proposed approach is robust to object motion and can obtain high-quality 3D reconstructions of isolated objects within a single fringe image.
基金the financial support provided by the National Natural Science Foundation of China(11472267 and 11372182)the National Basic Research Program of China(2012CB937504)
文摘Although the structured light system that uses digital fringe projection has been widely implemented in three-dimensional surface profile measurement, the measurement system is susceptible to non-linear error. In this work, we propose a convenient look-up-table-based （LUT-based） method to compensate for the non-linear error in captured fringe patterns. Without extra calibration, this LUT-based method completely utilizes the captured fringe pattern by recording the full-field differences. Then, a phase compensation map is established to revise the measured phase. Experimental results demonstrate that this method works effectively.
基金supported by the National Natural Science Foundation of China(10732030)
文摘We describe a 2D fringe projection method that involves projecting two groups of sharp comb fringes onto a free-flying hawk-moth from different directions and recording the images of distorted fringes by two high speed cameras from two orthogonal views. By calculating the 3D coordinates of the points on the hawk-moth and three-dimensional reconstruction of the wing, the flight trajectory, body attitude and wing kinematics including flapping angle, elevation angle, torsion angle, and camber deformation are obtained.
基金Supported by National Defense Basic Scientific Research Program of China(Grant No.JCKY2021602B032)。
文摘Fringe projection profilometry(FPP)has been widely applied to non-contact three-dimensional measurement in industries owing to its high accuracy and speed.The point cloud,which is a measurement result of the FPP system,typically contains a large number of invalid points caused by the background,ambient light,shadows,and object edge regions.Research on noisy point detection and elimination has been conducted over the past two decades.However,existing invalid point removal methods are based on image intensity analysis and are only applicable to simple measurement backgrounds that are purely dark.In this paper,we propose a novel invalid point removal framework that consists of two aspects:(1)A convolutional neural network(CNN)is designed to segment the foreground from the background of different intensity conditions in FPP measurement circumstances to remove background points and the most discrete points in background regions.(2)A two-step method based on the fringe image intensity threshold and a bilateral filter is proposed to eliminate the small number of discrete points remaining after background segmentation caused by shadows and edge areas on objects.Experimental results verify that the proposed framework(1)can remove background points intelligently and accurately in different types of complex circumstances,and(2)performs excellently in discrete point detection from object regions.
基金the National Natural Science Foundation of China(No.61905178)the Science&Technology Development Fund of Tianjin Education Commission for Higher Education(No.2019KJ021)the Natural Science Foundation of Tianjin(No.18JCQNJC71100)。
文摘Phase unwrapping is one of the key roles in fringe projection three-dimensional(3D)measurement technology.We propose a new method to achieve phase unwrapping in camera array light filed fringe projection 3D measurement based on deep learning.A multi-stream convolutional neural network(CNN)is proposed to learn the mapping relationship between camera array light filed wrapped phases and fringe orders of the expected central view,and is used to predict the fringe order to achieve the phase unwrapping.Experiments are performed on the light field fringe projection data generated by the simulated camera array fringe projection measurement system in Blender and by the experimental 3×3 camera array light field fringe projection system.The performance of the proposed network with light field wrapped phases using multiple directions as network input data is studied,and the advantages of phase unwrapping based on deep learning in light filed fringe projection are demonstrated.
基金Project supported by the Science Foundation of Shanghai Munici pal Commission of Science and Technology ( Grant No.011461059)
文摘Acquisition of physical data with high precision is a key step in reverse engineering (RE). It is an important stimulative for the progress of reverse engineering with which various digitizing devices are invent ed, developed and made applicable. This paper introduces a three dimensional opt ical measurement method based on digital fringe projection technique in RE to im prove the technique through its application. A practical example is presented an d the result demonstrates the applicability and feasibility of the measurement s ystem as well as the reliability and validity of relevant methods and algorithms .
基金supported by the National Natural Science Foundation of China(Nos.60808020 and 61078041)the Natural Science Foundation of Tianjin City(Nos.15JCYBJC51700 and 16JCYBJC15400)the National Science and Technology Support(No.2014BAH03F01)
文摘In order to get the entire data in the optical measurement, a multi-view three-dimensional(3D) measurement method based on turntable is proposed. In the method, a turntable is used to rotate the object and obtain multi-view point cloud data, and then multi-view point cloud data are registered and integrated into a 3D model. The measurement results are compared with that of the sticking marked point method. Experimental results show that the measurement process of the proposed method is simpler, and the scanning speed and accuracy are improved.
基金National Key Research and Development Program of China(2023YFB2806800)Open Research Projects of KLOMT(2022KLOMT02-02)。
文摘Fringe projection profilometry(FPP)is a method that determines height by analyzing distortional fringes,which is widely used in high-accuracy 3D imaging.Now,one major reason limiting imaging speed in FPP is the projection device;the capture speed of high-speed cameras far exceeds the projection frequency.Among various devices,an LED array can exceed the speed of a high-speed camera.However,non-sinusoidal fringe patterns in the LED array systems can arise from several factors that will reduce the accuracy,such as the spacing between adjacent LEDs,the inconsistency in brightness across different LEDs,and the residual high-order harmonics in binary defocusing projection.It is challenging to resolve by other methods.In this paper,we propose a method that creates a look-up table using system calibration data of phase-height models.Then we utilize the look-up table to compensate for the phase error during the reconstructing process.The foundation of the proposed method relies on the time-invariance of systematic error;any factor that impacts the sinusoidal characteristic would present as an anomaly in the unwrapped phase.Experiments have demonstrated that the root mean square errors(RMSEs)of the results yielded by the proposed method were reduced by over 90%compared to those yielded by the traditional method,reaching 20μm accuracy.This paper offers an alternative approach for high-speed and high-accuracy 3D imaging with an LED array and presents a workable solution for addressing complex errors from non-sinusoidal fringes.
文摘Fringe projection profilometry,a powerful technique for three-dimensional(3D)imaging and measurement,has been revolutionized by deep learning,achieving speeds of up to 100,000 frames per second(fps)while preserving highresolution.This advancement expands its applications to high-speed transient scenarios,opening new possibilities for ultrafast 3D measurements.
基金supported by the National Key Research and Development Program of China(No.2018YFB2001400)the Innovation Group Science Fund of Chongqing Natural Science Foundation(No.cstc2019jcyj-cxttX0003)。
文摘Fringe projection profilometry(FPP)has been extensively studied in the field of three-dimensional(3D)measurement.Although FPP always uses high-frequency fringes to ensure high measurement accuracy,too many patterns are projected to unwrap the phase,which affects the speed of 3D reconstruction.We propose a high-speed 3D shape measurement method using only three high-frequency inner shifting-phase patterns(70 periods),which satisfies both high precision and high measuring speed requirements.Besides,our proposed method obtains the wrapped phase and the fringe order simultaneously without any other information and constraints.The proposed method has successfully reconstructed moving objects with high speed at the camera's full frame rate(1700 frames per second).
基金supported by National Key Research and Development Program of China(2022YFB2804603)National Natural Science Foundation of China(62075096,62005121,U21B2033)+3 种基金Leading Technology of Jiangsu Basic Research Plan(BK20192003)“333 Engineering”Research Project of Jiangsu Province(BRA2016407)Fundamental Research Funds for the Central Universities(30921011208,30919011222,30920032101)Fundamental Research Funds for the Central Universities(2023102001,2024202002).
文摘Recent advances in imaging sensors and digital light projection technology have facilitated rapid progress in 3D optical sensing,enabling 3D surfaces of complexshaped objects to be captured with high resolution and accuracy.Nevertheless,due to the inherent synchronous pattern projection and image acquisition mechanism,the temporal resolution of conventional structured light or fringe projection profilometry(FPP)based 3D imaging methods is still limited to the native detector frame rates.In this work,we demonstrate a new 3D imaging method,termed deep-learning-enabled multiplexed FPP(DLMFPP),that allows to achieve high-resolution and high-speed 3D imaging at near-one-order of magnitude-higher 3D frame rate with conventional low-speed cameras.By encoding temporal information in one multiplexed fringe pattern,DLMFPP harnesses deep neural networks embedded with Fourier transform,phase-shifting and ensemble learning to decompose the pattern and analyze separate fringes,furnishing a high signal-to-noise ratio and a ready-to-implement solution over conventional computational imaging techniques.We demonstrate this method by measuring different types of transient scenes,including rotating fan blades and bullet fired from a toy gun,at kHz using cameras of around 100 Hz.Experiential results establish that DLMFPP allows slow-scan cameras with their known advantages in terms of cost and spatial resolution to be used for high-speed 3D imaging tasks.
基金funded by the National Science Foundation(NSF)Award 2009384.
文摘In 2019,the Event Horizon Telescope(EHT)released the first-ever image of a black hole event horizon.Astronomers are now aiming for higher angular resolutions of distant targets,like black holes,to understand more about the fundamental laws of gravity that govern our universe.To achieve this higher resolution and increased sensitivity,larger radio telescopes are needed to operate at higher frequencies and in larger quantities.Projects like the next-generation Very Large Array(ngVLA)and the Square-Kilometer Array(SKA)require building hundreds of telescopes with diameters greater than 10 ms over the next decade.This has a twofold effect.Radio telescope surfaces need to be more accurate to operate at higher frequencies,and the logistics involved in maintaining a radio telescope need to be simplified to support them properly in large quantities.Both of these problems can be solved with improved methods for surface metrology that are faster and more accurate with a higher resolution.This leads to faster and more accurate panel alignment and,therefore,a more productive observatory.In this paper,we present the use of binocular fringe projection profilometry as a solution to this problem and demonstrate it by aligning two panels on a 3-m radio telescope dish.The measurement takes only 10 min and directly delivers feedback on the tip,tilt,and piston of each panel to create the ideal reflector shape.
基金supported by the National Natural Science Foundation of China(Nos.60808020 and 61078041)the National Science and Technology Support(No.2014BAH03F01)+1 种基金the Tianjin Research Program of Application Foundation and Advanced Technology(No.10JCYBJC07200)the Technology Program of Tianjin Municipal Education Commission(No.20130324)
文摘In the three-dimensional(3D) contour measurement,the phase shift profilometry(PSP) method is the most widely used one.However,the measurement speed of PSP is very low because of the multiple projections.In order to improve the measurement speed,color grating stripes are used for measurement in this paper.During the measurement,only one color sinusoidal fringe is projected on the measured object.Therefore,the measurement speed is greatly improved.Since there is coupling or interference phenomenon between the adjacent color grating stripes,a color correction method is used to improve the measurement results.A method for correcting nonlinear error of measurement system is proposed in this paper,and the sinusoidal property of acquired image after correction is better than that before correction.Experimental results show that with these correction methods,the measurement errors can be reduced.Therefore,it can support a good foundation for the high-precision 3D reconstruction.
文摘To measure breast basic dimension by using computer-aided projection fringe system.Methods A system has been developed for measuring breast basic dimension based on computer-aided projection fringe measurement and programming software.Plastic manikins breast’s SN-N (sternal notch to nipple distance),N-ML (nipple to midline distance),N-N (internipple distance),MBW (base width of breast) and N-IMF (nipple to inframammary fold distance) are measured with this system.At the same time,these items are also measured with routine ruler.Results This study indicate that the system has some merits:① non-touching measurement;② it is very rapid,the patient measured need hold his breath only 0.5 second,and all the time it takes is about 2.5 minutes;③ the measurement’s sensitivity is as high as to 0.6 mm,which meets the clinic requirement entirely;④ the measurement’s accuracy of the system is not significantly when comparing to the routine ruler’s.Conclusion Computer-adided projection fringe system for measuring breast basic dimension is feasible and advanced.14 refs,1 fig.
文摘The Fringe Projection Profilometry(FPP)system with a single exposure time or a single projection intensity is limited by the dynamic range of the camera,which can lead to overexposure and underexposure of the image,resulting in point cloud loss or reduced accuracy.To address this issue,unlike the pixel modulation method of projectors,we utilize the characteristics of color projectors where the intensity of the three-channel LED can be controlled independently.We propose a method for separating the projector's three-channel light intensity,combined with a color camera,to achieve single exposure and multi-intensity image acquisition.Further,the crosstalk coefficient is applied to predict the three-channel reflectance of the measured object.By integrating clustering and channel mapping,we establish a pixel-level mapping model between the projector's three-channel current and the camera's three-channel image intensity,which realizes the optimal projection current prediction and the high dynamic range(HDR)image acquisition.The proposed method allows for high-precision three-dimensional(3D)data acquisition of HDR scenes with a single exposure.The effectiveness of this method has been validated through experiments with standard planes and standard steps,showing a significant reduction in mean absolute error(44.6%)compared to existing singleexposure HDR methods.Additionally,the number of images required for acquisition is significantly reduced(by 70.8%)compared to multi-exposure fusion methods.This proposed method has great potential in various FPP-related fields.
基金This project is supported by National Natural Science Foundation ofChina (No.50375047).
文摘A fast 3D reconstruction method based on structured light to measure various parameters of the raceway groove is presented. Digital parallel grating stripes distributed with sine density are projected onto the raceway groove by a DLP projector, and distorting of stripes is happened on the raceway. Simultaneously, aided by three-step phase-shifting approach, three images covered by different stripes are obtained by a high-resolution CCD camera at the same location, thus a more accuracy local topography can be obtained. And then the bearing is rotated on a high precision computer controlled rotational stage. Three images are also obtained as the former step at next planned location triggered by the motor. After one cycle, all images information is combined through the mosaics. As a result, the 3D information of raceway groove can be gained. Not only geometric properties but also surface flaws can be extracted by software. A preliminary hardware system has been built, with which some geometric parameters have been extracted from reconstructed local topography.
文摘The knowledge of wing orientation and deformation during flapping flight is necessary for a complete aerodynamic analysis, but to date those kinematic features have not been simultaneously quantified for free-flying insects. A projected comb-fringe (PCF) method has been developed for measuring spanwise camber changes on free-flying dragonflies and on beating-flying dragonflies through the course of a wingbeat, which bases on projecting a fringe pattern over the whole measurement area and then measuring the wing deformation from the distorted fringe pattern. Experimental results demonstrate substantial camber changes both along the wingspan and through the course of a wingbeat. The ratio of camber deformation to chord length for hind wing is up to 0.11 at 75% spanwise with a flapping angle of -0.66 degree for a free-flying dragonfly.
