In this work we consider the problem of shape reconstruction from an unorganized data set which has many important applications in medical imaging, scientific computing, reverse engineering and geometric modelling. Th...In this work we consider the problem of shape reconstruction from an unorganized data set which has many important applications in medical imaging, scientific computing, reverse engineering and geometric modelling. The reconstructed surface is obtained by continuously deforming an initial surface following the Partial Differential Equation (PDE)-based diffusion model derived by a minimal volume-like variational formulation. The evolution is driven both by the distance from the data set and by the curvature analytically computed by it. The distance function is computed by implicit local interpolants defined in terms of radial basis functions. Space discretization of the PDE model is obtained by finite co-volume schemes and semi-implicit approach is used in time/scale. The use of a level set method for the numerical computation of the surface reconstruction allows us to handle complex geometry and even changing topology,without the need of user-interaction. Numerical examples demonstrate the ability of the proposed method to produce high quality reconstructions. Moreover, we show the effectiveness of the new approach to solve hole filling problems and Boolean operations between different data sets.展开更多
Parametric curves such as Bézier and B-splines, originally developedfor the design of automobile bodies, are now also used in image processing andcomputer vision. For example, reconstructing an object shape in an...Parametric curves such as Bézier and B-splines, originally developedfor the design of automobile bodies, are now also used in image processing andcomputer vision. For example, reconstructing an object shape in an image,including different translations, scales, and orientations, can be performedusing these parametric curves. For this, Bézier and B-spline curves can be generatedusing a point set that belongs to the outer boundary of the object. Theresulting object shape can be used in computer vision fields, such as searchingand segmentation methods and training machine learning algorithms. Theprerequisite for reconstructing the shape with parametric curves is to obtainsequentially the points in the point set. In this study, a novel algorithm hasbeen developed that sequentially obtains the pixel locations constituting theouter boundary of the object. The proposed algorithm, unlike the methods inthe literature, is implemented using a filter containing weights and an outercircle surrounding the object. In a binary format image, the starting point ofthe tracing is determined using the outer circle, and the next tracing movementand the pixel to be labeled as the boundary point is found by the filter weights.Then, control points that define the curve shape are selected by reducing thenumber of sequential points. Thus, the Bézier and B-spline curve equationsdescribing the shape are obtained using these points. In addition, differenttranslations, scales, and rotations of the object shape are easily provided bychanging the positions of the control points. It has also been shown that themissing part of the object can be completed thanks to the parametric curves.展开更多
Depth measurement and three-dimensional(3D)imaging under complex reflection and transmission conditions are challenging and even impossible for traditional structured light techniques,owing to the precondition of poin...Depth measurement and three-dimensional(3D)imaging under complex reflection and transmission conditions are challenging and even impossible for traditional structured light techniques,owing to the precondition of point-to-point triangulation.Despite recent progress in addressing this problem,there is still no efficient and general solution.Herein,a Fourier dual-slice projection with depth-constrained localization is presented to separate and utilize different illumination and reflection components efficiently,which can significantly decrease the number of projection patterns in each sequence from thousands to fifteen.Subsequently,multi-scale parallel single-pixel imaging(MS-PSI)is proposed based on the established and proven position-invariant theorem,which breaks the local regional assumption and enables dynamic 3D reconstruction.Our methodology successfully unveils unseen-before capabilities such as(1)accurate depth measurement under interreflection and subsurface scattering conditions,(2)dynamic measurement of the time-varying high-dynamic-range scene and through thin volumetric scattering media at a rate of 333 frames per second;(3)two-layer 3D imaging of the semitransparent surface and the object hidden behind it.The experimental results confirm that the proposed method paves the way for dynamic 3D reconstruction under complex optical field reflection and transmission conditions,benefiting imaging and sensing applications in advanced manufacturing,autonomous driving,and biomedical imaging.展开更多
Tactile sensing enables high-precision 3D shape perception when vision is limited.However,tactilebased shape reconstruction remains a challenging problem.In this paper,a novel visuotactile sensor,GelStereo Palm 2.0,is...Tactile sensing enables high-precision 3D shape perception when vision is limited.However,tactilebased shape reconstruction remains a challenging problem.In this paper,a novel visuotactile sensor,GelStereo Palm 2.0,is proposed to better capture 3D contact geometry.Leveraging the dense tactile point cloud captured by GelStereo Palm 2.0,an active shape reconstruction pipeline is presented to achieve accurate and efficient 3D shape reconstruction on irregular surfaces.GelStereo Palm 2.0 achieves a spatial resolution of 1.5 mm and a reconstruction accuracy of 0.3 mm.The accuracy of the proposed active shape reconstruction pipeline reaches 2.3 mm within 18 explorations.The proposed method has potential applications in the shape reconstruction of transparent or underwater objects.展开更多
Depth from defocus(DFD),as a typical shape reconstruction method,has been widely researched in most recent years.However,all the existing DFD algorithms require at least two defocused images with different camera para...Depth from defocus(DFD),as a typical shape reconstruction method,has been widely researched in most recent years.However,all the existing DFD algorithms require at least two defocused images with different camera parameters.Unfortunately,in micro/nano manipulation,any change on visual sensor's parameters is absolutely forbidden.Therefore,a novel DFD method to reconstruct the shape of a nano grid on micro/nano scale is researched in this paper.First,the blurring imaging model is constructed with the relative blurring and the diffusion equation.Second,the relationship between depth and blurring is discussed from four aspects.Subsequently,depth measurement problem is transformed into an optimization issue which is solved using the gradient flow algorithm.Finally,experiment results and error analysis are conducted to show the feasibility and effectiveness of the proposed method.展开更多
Rapidly and accurately assessing the geometric characteristics of coarse aggregate particles is crucial for ensuring pavement performance in highway engineering.This article introduces an innovative system for the thr...Rapidly and accurately assessing the geometric characteristics of coarse aggregate particles is crucial for ensuring pavement performance in highway engineering.This article introduces an innovative system for the three-dimensional(3D)surface reconstruction of coarse aggregate particles using occlusion-free multi-view imaging.The system captures synchronized images of particles in free fall,employing a matte sphere and a nonlinear optimization approach to estimate the camera projection matrices.A pre-trained segmentation model is utilized to eliminate the background of the images.The Shape from Silhouettes(SfS)algorithm is then applied to generate 3D voxel data,followed by the Marching Cubes algorithm to construct the 3D surface contour.Validation against standard parts and diverse coarse aggregate particles confirms the method's high accuracy,with an average measurement precision of 0.434 mm and a significant increase in scanning and reconstruction efficiency.展开更多
We proposed a method for shape sensing using a few multicore fiber Bragg grating (FBG) sensors ina single-port continuum surgical robot (CSR). The traditional method of utilizing a forward kinematic model tocalculate t...We proposed a method for shape sensing using a few multicore fiber Bragg grating (FBG) sensors ina single-port continuum surgical robot (CSR). The traditional method of utilizing a forward kinematic model tocalculate the shape of a single-port CSR is limited by the accuracy of the model. If FBG sensors are used forshape sensing, their accuracy will be affected by their number, especially in long and flexible CSRs. A fusionmethod based on an extended Kalman filter (EKF) was proposed to solve this problem. Shape reconstructionwas performed using the CSR forward kinematic model and FBG sensors, and the two results were fused usingan EKF. The CSR reconstruction method adopted the incremental form of the forward kinematic model, whilethe FBG sensor method adopted the discrete arc-segment assumption method. The fusion method can eliminatethe inaccuracy of the kinematic model and obtain more accurate shape reconstruction results using only a smallnumber of FBG sensors. We validated our algorithm through experiments on multiple bending shapes underdifferent load conditions. The results show that our method significantly outperformed the traditional methodsin terms of robustness and effectiveness.展开更多
The field of vision-based human hand three-dimensional(3D)shape and pose estimation has attracted significant attention recently owing to its key role in various applications,such as natural human computer interaction...The field of vision-based human hand three-dimensional(3D)shape and pose estimation has attracted significant attention recently owing to its key role in various applications,such as natural human computer interactions.With the availability of large-scale annotated hand datasets and the rapid developments of deep neural networks(DNNs),numerous DNN-based data-driven methods have been proposed for accurate and rapid hand shape and pose estimation.Nonetheless,the existence of complicated hand articulation,depth and scale ambiguities,occlusions,and finger similarity remain challenging.In this study,we present a comprehensive survey of state-of-the-art 3D hand shape and pose estimation approaches using RGB-D cameras.Related RGB-D cameras,hand datasets,and a performance analysis are also discussed to provide a holistic view of recent achievements.We also discuss the research potential of this rapidly growing field.展开更多
Fluorescence molecular tomography(FMT) aims at tomographicallyresolving the fluorescent targets deeply inside small animal based on transmission boundary measurements. The image reconstruction of FMT isknown to be hig...Fluorescence molecular tomography(FMT) aims at tomographicallyresolving the fluorescent targets deeply inside small animal based on transmission boundary measurements. The image reconstruction of FMT isknown to be highlyill-posed, due to the highly scatteringnatureofbiologicaltissue.Hence,priorinformationisusuallyrequired for successful reconstruction. In this paper, a novel reconstruction method incorporating shape priors is proposed for 2D FMT. The fluorescent targets were assumed of round shape, which was practically appropriate for approximating various shapes inside diffusive medium. Compared to the traditional pixel based reconstruction, the number of unknowns was greatly reduced to a few control parameters of round shapes. A hybrid genetic algorithm was proposed to recover the shape parameters. The numerical experiments showed that the proposed method significantly improves the imaging accuracy, offering clearer target boundaries and better resolution. Comparison results also demonstrated that the hybridization of genetic algorithm and Newton-typesearchwaspivotalandimportantforrobustlyfindingthegloballyoptimalshape parameters.展开更多
Continuum manipulators have been applied in different surgical scenarios due to their dexterity and multi-DoF(degree of freedom)design compactness.To improve surgery safety,it is preferable to enable active compliance...Continuum manipulators have been applied in different surgical scenarios due to their dexterity and multi-DoF(degree of freedom)design compactness.To improve surgery safety,it is preferable to enable active compliance and force sensing abilities for a continuum manipulator.Existing works on active compliance and force sensing often rely on force sensors at the proximal or the distal ends,which inevitably increases the system complexity.In this paper,a shape reconstruction algorithm,a compliant motion controller,and a force estimation method are proposed successively based on the manipulator's tip pose via visual feedback.Four support vector regression(SVR)trainers are constructed and trained to compensate for the actuation residues,which are the differences between the actual actuation lengths outputs at the actuators and the ideal actuation lengths calculated from the estimated shape using the kinematics model,under no-load condition.Then,a compliant motion controller and a force estimation method are realized based on the current actuation residues,compared with the actuation residues under the no-load condition.In this way,no additional sensors are needed as an endoscopic camera is often available in a laparoscopic or endoscopic surgical system.The experiments were conducted on aφ3 mm-continuum manipulator to demonstrate the effectiveness of the proposed algorithms.展开更多
基金supported by PRIN-MIUR-Cofin 2006,project,by"Progetti Strategici EF2006"University of Bologna,and by University of Bologna"Funds for selected research topics"
文摘In this work we consider the problem of shape reconstruction from an unorganized data set which has many important applications in medical imaging, scientific computing, reverse engineering and geometric modelling. The reconstructed surface is obtained by continuously deforming an initial surface following the Partial Differential Equation (PDE)-based diffusion model derived by a minimal volume-like variational formulation. The evolution is driven both by the distance from the data set and by the curvature analytically computed by it. The distance function is computed by implicit local interpolants defined in terms of radial basis functions. Space discretization of the PDE model is obtained by finite co-volume schemes and semi-implicit approach is used in time/scale. The use of a level set method for the numerical computation of the surface reconstruction allows us to handle complex geometry and even changing topology,without the need of user-interaction. Numerical examples demonstrate the ability of the proposed method to produce high quality reconstructions. Moreover, we show the effectiveness of the new approach to solve hole filling problems and Boolean operations between different data sets.
文摘Parametric curves such as Bézier and B-splines, originally developedfor the design of automobile bodies, are now also used in image processing andcomputer vision. For example, reconstructing an object shape in an image,including different translations, scales, and orientations, can be performedusing these parametric curves. For this, Bézier and B-spline curves can be generatedusing a point set that belongs to the outer boundary of the object. Theresulting object shape can be used in computer vision fields, such as searchingand segmentation methods and training machine learning algorithms. Theprerequisite for reconstructing the shape with parametric curves is to obtainsequentially the points in the point set. In this study, a novel algorithm hasbeen developed that sequentially obtains the pixel locations constituting theouter boundary of the object. The proposed algorithm, unlike the methods inthe literature, is implemented using a filter containing weights and an outercircle surrounding the object. In a binary format image, the starting point ofthe tracing is determined using the outer circle, and the next tracing movementand the pixel to be labeled as the boundary point is found by the filter weights.Then, control points that define the curve shape are selected by reducing thenumber of sequential points. Thus, the Bézier and B-spline curve equationsdescribing the shape are obtained using these points. In addition, differenttranslations, scales, and rotations of the object shape are easily provided bychanging the positions of the control points. It has also been shown that themissing part of the object can be completed thanks to the parametric curves.
基金supported by the National Natural Science Foundation of China(62205226,62075143)the National Postdoctoral Program for Innovative Talents of China(BX2021199)+2 种基金the General Financial Grant from the China Postdoctoral Science Foundation(2022M722290)the Key Science and Technology Research and Development Program of Jiangxi Province(20224AAC01011)the Fundamental Research Funds for Central Universities(2022SCU12010).
文摘Depth measurement and three-dimensional(3D)imaging under complex reflection and transmission conditions are challenging and even impossible for traditional structured light techniques,owing to the precondition of point-to-point triangulation.Despite recent progress in addressing this problem,there is still no efficient and general solution.Herein,a Fourier dual-slice projection with depth-constrained localization is presented to separate and utilize different illumination and reflection components efficiently,which can significantly decrease the number of projection patterns in each sequence from thousands to fifteen.Subsequently,multi-scale parallel single-pixel imaging(MS-PSI)is proposed based on the established and proven position-invariant theorem,which breaks the local regional assumption and enables dynamic 3D reconstruction.Our methodology successfully unveils unseen-before capabilities such as(1)accurate depth measurement under interreflection and subsurface scattering conditions,(2)dynamic measurement of the time-varying high-dynamic-range scene and through thin volumetric scattering media at a rate of 333 frames per second;(3)two-layer 3D imaging of the semitransparent surface and the object hidden behind it.The experimental results confirm that the proposed method paves the way for dynamic 3D reconstruction under complex optical field reflection and transmission conditions,benefiting imaging and sensing applications in advanced manufacturing,autonomous driving,and biomedical imaging.
基金supported in part by the National Key Research and Development Program of China(2023YFB4705000)in part by the National Natural Science Foundation of(62303455,62273342,and 62122087)in part by Beijing Natural Science Foundation(L233006).
文摘Tactile sensing enables high-precision 3D shape perception when vision is limited.However,tactilebased shape reconstruction remains a challenging problem.In this paper,a novel visuotactile sensor,GelStereo Palm 2.0,is proposed to better capture 3D contact geometry.Leveraging the dense tactile point cloud captured by GelStereo Palm 2.0,an active shape reconstruction pipeline is presented to achieve accurate and efficient 3D shape reconstruction on irregular surfaces.GelStereo Palm 2.0 achieves a spatial resolution of 1.5 mm and a reconstruction accuracy of 0.3 mm.The accuracy of the proposed active shape reconstruction pipeline reaches 2.3 mm within 18 explorations.The proposed method has potential applications in the shape reconstruction of transparent or underwater objects.
基金supported by the CAS FEA international partnership program for creative research teams
文摘Depth from defocus(DFD),as a typical shape reconstruction method,has been widely researched in most recent years.However,all the existing DFD algorithms require at least two defocused images with different camera parameters.Unfortunately,in micro/nano manipulation,any change on visual sensor's parameters is absolutely forbidden.Therefore,a novel DFD method to reconstruct the shape of a nano grid on micro/nano scale is researched in this paper.First,the blurring imaging model is constructed with the relative blurring and the diffusion equation.Second,the relationship between depth and blurring is discussed from four aspects.Subsequently,depth measurement problem is transformed into an optimization issue which is solved using the gradient flow algorithm.Finally,experiment results and error analysis are conducted to show the feasibility and effectiveness of the proposed method.
基金Supported by the Key R&D Projects in Shaanxi Province(2022JBGS3-08)。
文摘Rapidly and accurately assessing the geometric characteristics of coarse aggregate particles is crucial for ensuring pavement performance in highway engineering.This article introduces an innovative system for the three-dimensional(3D)surface reconstruction of coarse aggregate particles using occlusion-free multi-view imaging.The system captures synchronized images of particles in free fall,employing a matte sphere and a nonlinear optimization approach to estimate the camera projection matrices.A pre-trained segmentation model is utilized to eliminate the background of the images.The Shape from Silhouettes(SfS)algorithm is then applied to generate 3D voxel data,followed by the Marching Cubes algorithm to construct the 3D surface contour.Validation against standard parts and diverse coarse aggregate particles confirms the method's high accuracy,with an average measurement precision of 0.434 mm and a significant increase in scanning and reconstruction efficiency.
基金the National Natural Science Foundation of China(Nos.61873257 and U20A20195)the Project of Natural Science Foundation of Liaoning Province(No.2021-MS-033)the Foundation of Millions of Talents Project of the Department of Human Resources and Social Security of Liaoning Province(No.2021921037)。
文摘We proposed a method for shape sensing using a few multicore fiber Bragg grating (FBG) sensors ina single-port continuum surgical robot (CSR). The traditional method of utilizing a forward kinematic model tocalculate the shape of a single-port CSR is limited by the accuracy of the model. If FBG sensors are used forshape sensing, their accuracy will be affected by their number, especially in long and flexible CSRs. A fusionmethod based on an extended Kalman filter (EKF) was proposed to solve this problem. Shape reconstructionwas performed using the CSR forward kinematic model and FBG sensors, and the two results were fused usingan EKF. The CSR reconstruction method adopted the incremental form of the forward kinematic model, whilethe FBG sensor method adopted the discrete arc-segment assumption method. The fusion method can eliminatethe inaccuracy of the kinematic model and obtain more accurate shape reconstruction results using only a smallnumber of FBG sensors. We validated our algorithm through experiments on multiple bending shapes underdifferent load conditions. The results show that our method significantly outperformed the traditional methodsin terms of robustness and effectiveness.
基金the National Key R&D Program of China(2018YFB1004600)the National Natural Science Foundation of China(61502187,61876211)the National Science Foundation Grant CNS(1951952).
文摘The field of vision-based human hand three-dimensional(3D)shape and pose estimation has attracted significant attention recently owing to its key role in various applications,such as natural human computer interactions.With the availability of large-scale annotated hand datasets and the rapid developments of deep neural networks(DNNs),numerous DNN-based data-driven methods have been proposed for accurate and rapid hand shape and pose estimation.Nonetheless,the existence of complicated hand articulation,depth and scale ambiguities,occlusions,and finger similarity remain challenging.In this study,we present a comprehensive survey of state-of-the-art 3D hand shape and pose estimation approaches using RGB-D cameras.Related RGB-D cameras,hand datasets,and a performance analysis are also discussed to provide a holistic view of recent achievements.We also discuss the research potential of this rapidly growing field.
基金State Key Laboratory of Software Development Environmentgrant number:SKLSDE-2011ZX-12+4 种基金the National Natural Science Foundation of Chinagrant number:61108084,61101008Research Fund for the Doctoral Program of Higher Education of Chinagrant number:20111102120039Key Laboratory for Biomechanics and Mechanobiology of Ministry of Education,the Fundamental Research Funds for the Central Universities
文摘Fluorescence molecular tomography(FMT) aims at tomographicallyresolving the fluorescent targets deeply inside small animal based on transmission boundary measurements. The image reconstruction of FMT isknown to be highlyill-posed, due to the highly scatteringnatureofbiologicaltissue.Hence,priorinformationisusuallyrequired for successful reconstruction. In this paper, a novel reconstruction method incorporating shape priors is proposed for 2D FMT. The fluorescent targets were assumed of round shape, which was practically appropriate for approximating various shapes inside diffusive medium. Compared to the traditional pixel based reconstruction, the number of unknowns was greatly reduced to a few control parameters of round shapes. A hybrid genetic algorithm was proposed to recover the shape parameters. The numerical experiments showed that the proposed method significantly improves the imaging accuracy, offering clearer target boundaries and better resolution. Comparison results also demonstrated that the hybridization of genetic algorithm and Newton-typesearchwaspivotalandimportantforrobustlyfindingthegloballyoptimalshape parameters.
基金supported by the National Key Research and Development Program of China(Grant No.2022YFB4700900)the National Natural Science Foundation of China(Grant No.51722507)。
文摘Continuum manipulators have been applied in different surgical scenarios due to their dexterity and multi-DoF(degree of freedom)design compactness.To improve surgery safety,it is preferable to enable active compliance and force sensing abilities for a continuum manipulator.Existing works on active compliance and force sensing often rely on force sensors at the proximal or the distal ends,which inevitably increases the system complexity.In this paper,a shape reconstruction algorithm,a compliant motion controller,and a force estimation method are proposed successively based on the manipulator's tip pose via visual feedback.Four support vector regression(SVR)trainers are constructed and trained to compensate for the actuation residues,which are the differences between the actual actuation lengths outputs at the actuators and the ideal actuation lengths calculated from the estimated shape using the kinematics model,under no-load condition.Then,a compliant motion controller and a force estimation method are realized based on the current actuation residues,compared with the actuation residues under the no-load condition.In this way,no additional sensors are needed as an endoscopic camera is often available in a laparoscopic or endoscopic surgical system.The experiments were conducted on aφ3 mm-continuum manipulator to demonstrate the effectiveness of the proposed algorithms.
