Realistic human skin rendering has been a long-standing challenge in computer graphics.Recently,biophysical-based skin rendering has received increasing attention,as it provides a more realistic skin-rendering and a m...Realistic human skin rendering has been a long-standing challenge in computer graphics.Recently,biophysical-based skin rendering has received increasing attention,as it provides a more realistic skin-rendering and a more intuitive way to adjust the skin style.In this work,we present a novel heterogeneous biophysical-based volume rendering method for human skin that improves the realism of skin appearance while easily simulating various types of skin effects,including skin diseases,by modifying biological coefficient textures.Specifically,we introduce a two-layer skin representation by mesh deformation that explicitly models the epidermis and dermis with heterogeneous volumetric medium layers containing the corresponding spatially varying melanin and hemoglobin,respectively.Furthermore,to better facilitate skin acquisition,we introduced a learning-based framework that automatically estimates spatially varying biological coefficients from an albedo texture,enabling biophysical-based and intuitive editing,such as tanning,pathological vitiligo,and freckles.We illustrated the effects of multiple skinediting applications and demonstrated superior quality to the commonly used random walk skin-rendering method,with more convincing skin details regarding subsurface scattering.展开更多
Advances in mobile cameras have made it easier to capture ultra-high resolution(UHR)portraits.However,existing face reconstruction methods lack specific adaptations for UHR input(e.g.,4096×4096),leading to under-...Advances in mobile cameras have made it easier to capture ultra-high resolution(UHR)portraits.However,existing face reconstruction methods lack specific adaptations for UHR input(e.g.,4096×4096),leading to under-use of high-frequency details that are crucial for achieving photorealistic rendering.Our method supports 4096×4096 UHR input and utilizes a divide-and-conquer approach for end-to-end 4K albedo,micronormal,and specular texture reconstruction at the original resolution.We employ a two-stage strategy to capture both global distributions and local high-frequency details,effectively mitigating mosaic and seam artifacts common in patch-based prediction.Additionally,we innovatively apply hash encoding to facial U-V coordinates to boost the model’s ability to learn regional high-frequency feature distributions.Our method can be easily incorporated in stateof-the-art facial geometry reconstruction pipelines,significantly improving the texture reconstruction quality,facilitating artistic creation workflows.展开更多
Monte Carlo(MC)integration is used ubiquitously in realistic image synthesis because of its flexibility and generality.However,the integration has to balance estimator bias and variance,which causes visually distracti...Monte Carlo(MC)integration is used ubiquitously in realistic image synthesis because of its flexibility and generality.However,the integration has to balance estimator bias and variance,which causes visually distracting noise with low sample counts.Existing solutions fall into two categories,in-process sampling schemes and post-processing reconstruction schemes.This report summarizes recent trends in the post-processing reconstruction scheme.Recent years have seen increasing attention and significant progress in denoising MC rendering with deep learning,by training neural networks to reconstruct denoised rendering results from sparse MC samples.Many of these techniques show promising results in real-world applications,and this report aims to provide an assessment of these approaches for practitioners and researchers.展开更多
Photorealistic rendering of the virtual world is an important and classic problem in the field of computer graphics.With the development of GPU hardware and continuous research on computer graphics,representing and re...Photorealistic rendering of the virtual world is an important and classic problem in the field of computer graphics.With the development of GPU hardware and continuous research on computer graphics,representing and rendering virtual scenes has become easier and more efficient.However,there are still unresolved challenges in efficiently rendering global illumination effects.At the same time,machine learning and computer vision provide real-world image analysis and synthesis methods,which can be exploited by computer graphics rendering pipelines.Deep learning-enhanced rendering combines techniques from deep learning and computer vision into the traditional graphics rendering pipeline to enhance existing rasterization or Monte Carlo integration renderers.This state-of-the-art report summarizes recent studies of deep learning-enhanced rendering in the computer graphics community.Specifically,we focus on works of renderers represented using neural networks,whether the scene is represented by neural networks or traditional scene files.These works are either for general scenes or specific scenes,which are differentiated by the need to retrain the network for new scenes.展开更多
When the unmanned aerial vehicle(UAV)is applied to three-dimensional(3D)reconstruction of the offshore ship,it faces two problems:the battery capacity limitation of the UAV and the disturbance of the wind in the envir...When the unmanned aerial vehicle(UAV)is applied to three-dimensional(3D)reconstruction of the offshore ship,it faces two problems:the battery capacity limitation of the UAV and the disturbance of the wind in the environment.Wind disturbance is generally not considered in the path planning process of the existing UAV 3D reconstruction path planning research.Therefore,the planned path is only suitable for no-wind or light-wind scenarios.For the 3D reconstruction of ship targets,we propose a UAV path planning method that can satisfy both reconstruction efficiency and wind disturbance resistance requirements.Firstly,the concept of model surface complexity is proposed to generate a more efficient candidate view set.Secondly,the Min–Max strategy and a new viewpoint construction method are used to generate the initial path.Thirdly,combined with the wind field model,a method for generating a stable path against wind disturbance based on the idea of interval optimization is proposed.Experimental results demonstrate that our method can adaptively determine the number of sample points and viewpoints according to ship’s geometric characteristics and further reduce the number of viewpoints without significantly affecting the reconstruction quality;the path planned by our method is also stable against wind disturbance.展开更多
Gradient-domain rendering estimates finite difference gradients of image intensities and reconstructs the final result by solving a screened Poisson problem,which shows improvements over merely sampling pixel intensit...Gradient-domain rendering estimates finite difference gradients of image intensities and reconstructs the final result by solving a screened Poisson problem,which shows improvements over merely sampling pixel intensities.Adaptive sampling is another orthogonal research area that focuses on distributing samples adaptively in the primal domain.However,adaptive sampling in the gradient domain with low sampling budget has been less explored.Our idea is based on the observation that signals in the gradient domain are sparse,which provides more flexibility for adaptive sampling.We propose a deep-learning-based end-to-end sampling and reconstruction framework in gradient-domain rendering,enabling adaptive sampling gradient and the primal maps simultaneously.We conducted extensive experiments for evaluation and showed that our method produces better reconstruction quality than other methods in the test dataset.展开更多
基金supported by Key R&D Program of Zhejiang Province(No.2023C01039).
文摘Realistic human skin rendering has been a long-standing challenge in computer graphics.Recently,biophysical-based skin rendering has received increasing attention,as it provides a more realistic skin-rendering and a more intuitive way to adjust the skin style.In this work,we present a novel heterogeneous biophysical-based volume rendering method for human skin that improves the realism of skin appearance while easily simulating various types of skin effects,including skin diseases,by modifying biological coefficient textures.Specifically,we introduce a two-layer skin representation by mesh deformation that explicitly models the epidermis and dermis with heterogeneous volumetric medium layers containing the corresponding spatially varying melanin and hemoglobin,respectively.Furthermore,to better facilitate skin acquisition,we introduced a learning-based framework that automatically estimates spatially varying biological coefficients from an albedo texture,enabling biophysical-based and intuitive editing,such as tanning,pathological vitiligo,and freckles.We illustrated the effects of multiple skinediting applications and demonstrated superior quality to the commonly used random walk skin-rendering method,with more convincing skin details regarding subsurface scattering.
基金supported by the National Key R&D Program of China(2024YDLN0011)the Key R&D Program of Zhejiang Province(2023C01039).
文摘Advances in mobile cameras have made it easier to capture ultra-high resolution(UHR)portraits.However,existing face reconstruction methods lack specific adaptations for UHR input(e.g.,4096×4096),leading to under-use of high-frequency details that are crucial for achieving photorealistic rendering.Our method supports 4096×4096 UHR input and utilizes a divide-and-conquer approach for end-to-end 4K albedo,micronormal,and specular texture reconstruction at the original resolution.We employ a two-stage strategy to capture both global distributions and local high-frequency details,effectively mitigating mosaic and seam artifacts common in patch-based prediction.Additionally,we innovatively apply hash encoding to facial U-V coordinates to boost the model’s ability to learn regional high-frequency feature distributions.Our method can be easily incorporated in stateof-the-art facial geometry reconstruction pipelines,significantly improving the texture reconstruction quality,facilitating artistic creation workflows.
基金supported by National Research Foundation of Korea(NRF)grant(MSIT)(No.2019R1A2C3002833).
文摘Monte Carlo(MC)integration is used ubiquitously in realistic image synthesis because of its flexibility and generality.However,the integration has to balance estimator bias and variance,which causes visually distracting noise with low sample counts.Existing solutions fall into two categories,in-process sampling schemes and post-processing reconstruction schemes.This report summarizes recent trends in the post-processing reconstruction scheme.Recent years have seen increasing attention and significant progress in denoising MC rendering with deep learning,by training neural networks to reconstruct denoised rendering results from sparse MC samples.Many of these techniques show promising results in real-world applications,and this report aims to provide an assessment of these approaches for practitioners and researchers.
文摘Photorealistic rendering of the virtual world is an important and classic problem in the field of computer graphics.With the development of GPU hardware and continuous research on computer graphics,representing and rendering virtual scenes has become easier and more efficient.However,there are still unresolved challenges in efficiently rendering global illumination effects.At the same time,machine learning and computer vision provide real-world image analysis and synthesis methods,which can be exploited by computer graphics rendering pipelines.Deep learning-enhanced rendering combines techniques from deep learning and computer vision into the traditional graphics rendering pipeline to enhance existing rasterization or Monte Carlo integration renderers.This state-of-the-art report summarizes recent studies of deep learning-enhanced rendering in the computer graphics community.Specifically,we focus on works of renderers represented using neural networks,whether the scene is represented by neural networks or traditional scene files.These works are either for general scenes or specific scenes,which are differentiated by the need to retrain the network for new scenes.
基金supported by the National Natural Science Foundation of China[grant numbers 52071201 and 61602426]Special Funding for the Development of Science and Technology of Shanghai Ocean University[grant number A2-2006-21-200207]+3 种基金Fund of Hubei Key Laboratory of Inland Shipping Technology[grant number NHHY2019001]Open Project Program of the State Key Lab of CAD&CG(Zhejiang University)[grant number A2107]Open Subject of the State Key Laboratory of Engines(Tianjin University)[grant number K2019-14]Soybean Intelligent Computing Breeding and Application[grant number 2021PE0AC04].
文摘When the unmanned aerial vehicle(UAV)is applied to three-dimensional(3D)reconstruction of the offshore ship,it faces two problems:the battery capacity limitation of the UAV and the disturbance of the wind in the environment.Wind disturbance is generally not considered in the path planning process of the existing UAV 3D reconstruction path planning research.Therefore,the planned path is only suitable for no-wind or light-wind scenarios.For the 3D reconstruction of ship targets,we propose a UAV path planning method that can satisfy both reconstruction efficiency and wind disturbance resistance requirements.Firstly,the concept of model surface complexity is proposed to generate a more efficient candidate view set.Secondly,the Min–Max strategy and a new viewpoint construction method are used to generate the initial path.Thirdly,combined with the wind field model,a method for generating a stable path against wind disturbance based on the idea of interval optimization is proposed.Experimental results demonstrate that our method can adaptively determine the number of sample points and viewpoints according to ship’s geometric characteristics and further reduce the number of viewpoints without significantly affecting the reconstruction quality;the path planned by our method is also stable against wind disturbance.
基金supported by the Key R&D Program of Zhejiang Province(No.2023C01039).
文摘Gradient-domain rendering estimates finite difference gradients of image intensities and reconstructs the final result by solving a screened Poisson problem,which shows improvements over merely sampling pixel intensities.Adaptive sampling is another orthogonal research area that focuses on distributing samples adaptively in the primal domain.However,adaptive sampling in the gradient domain with low sampling budget has been less explored.Our idea is based on the observation that signals in the gradient domain are sparse,which provides more flexibility for adaptive sampling.We propose a deep-learning-based end-to-end sampling and reconstruction framework in gradient-domain rendering,enabling adaptive sampling gradient and the primal maps simultaneously.We conducted extensive experiments for evaluation and showed that our method produces better reconstruction quality than other methods in the test dataset.
