Background This study presents a neural hand reconstruction method for monocular 3D hand pose and shape estimation.Methods Alternate to directly representing hand with 3D data,a novel UV position map is used to repres...Background This study presents a neural hand reconstruction method for monocular 3D hand pose and shape estimation.Methods Alternate to directly representing hand with 3D data,a novel UV position map is used to represent a hand pose and shape with 2D data that maps 3D hand surface points to 2D image space.Furthermore,an encoder-decoder neural network is proposed to infer such UV position map from a single image.To train this network with inadequate ground truth training pairs,we propose a novel MANOReg module that employs MANO model as a prior shape to constrain high dimensional space of the UV position map.Results The quantitative and qualitative experiments demonstrate the effectiveness of our UV position map representation and MANOReg module.展开更多
Background Aiming at free-view exploration of complicated scenes,this paper presents a method for interpolating views among multi RGB cameras.Methods In this study,we combine the idea of cost volume,which represent 3 ...Background Aiming at free-view exploration of complicated scenes,this paper presents a method for interpolating views among multi RGB cameras.Methods In this study,we combine the idea of cost volume,which represent 3 D information,and 2 D semantic segmentation of the scene,to accomplish view synthesis of complicated scenes.We use the idea of cost volume to estimate the depth and confidence map of the scene,and use a multi-layer representation and resolution of the data to optimize the view synthesis of the main object.Results/Conclusions By applying different treatment methods on different layers of the volume,we can handle complicated scenes containing multiple persons and plentiful occlusions.We also propose the view-interpolation→multi-view reconstruction→view interpolation pipeline to iteratively optimize the result.We test our method on varying data of multi-view scenes and generate decent results.展开更多
Typical stereo algorithms treat disparity estimation and view synthesis as two sequential procedures.In this paper,we consider stereo matching and view synthesis as two complementary components,and present a novel ite...Typical stereo algorithms treat disparity estimation and view synthesis as two sequential procedures.In this paper,we consider stereo matching and view synthesis as two complementary components,and present a novel iterative refinement model for joint view synthesis and disparity refinement.To achieve the mutual promotion between view synthesis and disparity refinement,we apply two key strategies,disparity maps fusion and disparity-assisted plane sweep-based rendering(DAPSR).On the one hand,the disparity maps fusion strategy is applied to generate disparity map from synthesized view and input views.This strategy is able to detect and counteract disparity errors caused by potential artifacts from synthesized view.On the other hand,the DAPSR is used for view synthesis and updating,and is able to weaken the interpolation errors caused by outliers in the disparity maps.Experiments on Middlebury benchmarks demonstrate that by introducing the synthesized view,disparity errors due to large occluded region and large baseline are eliminated effectively and the synthesis quality is greatly improved.展开更多
文摘Background This study presents a neural hand reconstruction method for monocular 3D hand pose and shape estimation.Methods Alternate to directly representing hand with 3D data,a novel UV position map is used to represent a hand pose and shape with 2D data that maps 3D hand surface points to 2D image space.Furthermore,an encoder-decoder neural network is proposed to infer such UV position map from a single image.To train this network with inadequate ground truth training pairs,we propose a novel MANOReg module that employs MANO model as a prior shape to constrain high dimensional space of the UV position map.Results The quantitative and qualitative experiments demonstrate the effectiveness of our UV position map representation and MANOReg module.
文摘Background Aiming at free-view exploration of complicated scenes,this paper presents a method for interpolating views among multi RGB cameras.Methods In this study,we combine the idea of cost volume,which represent 3 D information,and 2 D semantic segmentation of the scene,to accomplish view synthesis of complicated scenes.We use the idea of cost volume to estimate the depth and confidence map of the scene,and use a multi-layer representation and resolution of the data to optimize the view synthesis of the main object.Results/Conclusions By applying different treatment methods on different layers of the volume,we can handle complicated scenes containing multiple persons and plentiful occlusions.We also propose the view-interpolation→multi-view reconstruction→view interpolation pipeline to iteratively optimize the result.We test our method on varying data of multi-view scenes and generate decent results.
基金supported by the National key foundation for exploring scientific instrument(2013YQ140517)the National Natural Science Foundation of China(Grant No.61522111)the Shenzhen Peacock Plan(KQTD20140630115140843).
文摘Typical stereo algorithms treat disparity estimation and view synthesis as two sequential procedures.In this paper,we consider stereo matching and view synthesis as two complementary components,and present a novel iterative refinement model for joint view synthesis and disparity refinement.To achieve the mutual promotion between view synthesis and disparity refinement,we apply two key strategies,disparity maps fusion and disparity-assisted plane sweep-based rendering(DAPSR).On the one hand,the disparity maps fusion strategy is applied to generate disparity map from synthesized view and input views.This strategy is able to detect and counteract disparity errors caused by potential artifacts from synthesized view.On the other hand,the DAPSR is used for view synthesis and updating,and is able to weaken the interpolation errors caused by outliers in the disparity maps.Experiments on Middlebury benchmarks demonstrate that by introducing the synthesized view,disparity errors due to large occluded region and large baseline are eliminated effectively and the synthesis quality is greatly improved.
