In this study, we present a new andinnovative framework for acquiring high-qualitySVBRDF maps. Our approach addresses the limitations of the current methods and proposes a newsolution. The core of our method is a simp...In this study, we present a new andinnovative framework for acquiring high-qualitySVBRDF maps. Our approach addresses the limitations of the current methods and proposes a newsolution. The core of our method is a simple hardwaresetup consisting of a consumer-level camera, LEDlights, and a carefully designed network that canaccurately obtain the high-quality SVBRDF propertiesof a nearly planar object. By capturing a flexiblenumber of images of an object, our network usesdifferent subnetworks to train different property mapsand employs appropriate loss functions for each ofthem. To further enhance the quality of the maps, weimproved the network structure by adding a novel skipconnection that connects the encoder and decoder withglobal features. Through extensive experimentation usingboth synthetic and real-world materials, our resultsdemonstrate that our method outperforms previousmethods and produces superior results. Furthermore,our proposed setup can also be used to acquire physicallybased rendering maps of special materials.展开更多
Learning-based approaches have made substantial progress in capturing spatially-varying bidirectional reflectance distribution functions(SVBRDFs)from a single image with unknown lighting and geometry.However,most exis...Learning-based approaches have made substantial progress in capturing spatially-varying bidirectional reflectance distribution functions(SVBRDFs)from a single image with unknown lighting and geometry.However,most existing networks only consider per-pixel losses which limit their capability to recover local features such as smooth glossy regions.A few generative adversarial networks use multiple discriminators for different parameter maps,increasing network complexity.We present a novel end-to-end generative adversarial network(GAN)to recover appearance from a single picture of a nearly-flat surface lit by flash.We use a single unified adversarial framework for each parameter map.An attention module guides the network to focus on details of the maps.Furthermore,the SVBRDF map loss is combined to prevent paying excess attention to specular highlights.We demonstrate and evaluate our method on both public datasets and real data.Quantitative analysis and visual comparisons indicate that our method achieves better results than the state-of-the-art in most cases.展开更多
This paper proposes a stable method for reconstructing spatially varying appearances (SVBRDFs) frommultiview images captured under casual lighting conditions. Unlike flat surface capture methods, ourscan be applied to...This paper proposes a stable method for reconstructing spatially varying appearances (SVBRDFs) frommultiview images captured under casual lighting conditions. Unlike flat surface capture methods, ourscan be applied to surfaces with complex silhouettes. The proposed method takes multiview images asinputs and outputs a unified SVBRDF estimation. We generated a large-scale dataset containing themultiview images, SVBRDFs, and lighting appearance of vast synthetic objects to train a two-streamhierarchical U-Net for SVBRDF estimation that is integrated into a differentiable rendering networkfor surface appearance reconstruction. In comparison with state-of-the-art approaches, our methodproduces SVBRDFs with lower biases for more casually captured images.展开更多
Creating realistic materials is essential in the construction of immersive virtual environments.While existing techniques for material capture and conditional generation rely on flash-lit photos,they often produce art...Creating realistic materials is essential in the construction of immersive virtual environments.While existing techniques for material capture and conditional generation rely on flash-lit photos,they often produce artifacts when the illumination mismatches the training data.In this study,we introduce DiffMat,a novel diffusion model that integrates the CLIP image encoder and a multi-layer,crossattention denoising backbone to generate latent materials from images under various illuminations.Using a pre-trained StyleGAN-based material generator,our method converts these latent materials into high-resolution SVBRDF textures,a process that enables a seamless fit into the standard physically based rendering pipeline,reducing the requirements for vast computational resources and expansive datasets.DiffMat surpasses existing generative methods in terms of material quality and variety,and shows adaptability to a broader spectrum of lighting conditions in reference images.展开更多
基金supported by the Nature Science Fund of Guangdong Province(No.2021A1515011849)the Key Area Research and Development of Guangdong Province(No.2022A0505050014).
文摘In this study, we present a new andinnovative framework for acquiring high-qualitySVBRDF maps. Our approach addresses the limitations of the current methods and proposes a newsolution. The core of our method is a simple hardwaresetup consisting of a consumer-level camera, LEDlights, and a carefully designed network that canaccurately obtain the high-quality SVBRDF propertiesof a nearly planar object. By capturing a flexiblenumber of images of an object, our network usesdifferent subnetworks to train different property mapsand employs appropriate loss functions for each ofthem. To further enhance the quality of the maps, weimproved the network structure by adding a novel skipconnection that connects the encoder and decoder withglobal features. Through extensive experimentation usingboth synthetic and real-world materials, our resultsdemonstrate that our method outperforms previousmethods and produces superior results. Furthermore,our proposed setup can also be used to acquire physicallybased rendering maps of special materials.
基金supported by the National Natural Science Foundation of China(No.61602416)Shaoxing Science and Technology Plan Project(No.2020B41006).
文摘Learning-based approaches have made substantial progress in capturing spatially-varying bidirectional reflectance distribution functions(SVBRDFs)from a single image with unknown lighting and geometry.However,most existing networks only consider per-pixel losses which limit their capability to recover local features such as smooth glossy regions.A few generative adversarial networks use multiple discriminators for different parameter maps,increasing network complexity.We present a novel end-to-end generative adversarial network(GAN)to recover appearance from a single picture of a nearly-flat surface lit by flash.We use a single unified adversarial framework for each parameter map.An attention module guides the network to focus on details of the maps.Furthermore,the SVBRDF map loss is combined to prevent paying excess attention to specular highlights.We demonstrate and evaluate our method on both public datasets and real data.Quantitative analysis and visual comparisons indicate that our method achieves better results than the state-of-the-art in most cases.
基金Grant-in-Aid for Scientific Research(A)JP21H04916 and the Research Grant of Keio Leading-edge Laboratory of Science&Technology.
文摘This paper proposes a stable method for reconstructing spatially varying appearances (SVBRDFs) frommultiview images captured under casual lighting conditions. Unlike flat surface capture methods, ourscan be applied to surfaces with complex silhouettes. The proposed method takes multiview images asinputs and outputs a unified SVBRDF estimation. We generated a large-scale dataset containing themultiview images, SVBRDFs, and lighting appearance of vast synthetic objects to train a two-streamhierarchical U-Net for SVBRDF estimation that is integrated into a differentiable rendering networkfor surface appearance reconstruction. In comparison with state-of-the-art approaches, our methodproduces SVBRDFs with lower biases for more casually captured images.
基金Grant-in-Aid for Scientific Research(A)JP21H04916 and the Research Grant of Keio Leading-edge Laboratory of Science and Technology,Japan.
文摘Creating realistic materials is essential in the construction of immersive virtual environments.While existing techniques for material capture and conditional generation rely on flash-lit photos,they often produce artifacts when the illumination mismatches the training data.In this study,we introduce DiffMat,a novel diffusion model that integrates the CLIP image encoder and a multi-layer,crossattention denoising backbone to generate latent materials from images under various illuminations.Using a pre-trained StyleGAN-based material generator,our method converts these latent materials into high-resolution SVBRDF textures,a process that enables a seamless fit into the standard physically based rendering pipeline,reducing the requirements for vast computational resources and expansive datasets.DiffMat surpasses existing generative methods in terms of material quality and variety,and shows adaptability to a broader spectrum of lighting conditions in reference images.
