With the development of digital library technology, library books made of paper can be digital released and read, and Endangered Cultural Heritages can be preserved. Traditional library＇s contents and functions can b...With the development of digital library technology, library books made of paper can be digital released and read, and Endangered Cultural Heritages can be preserved. Traditional library＇s contents and functions can be greatly enhanced by digital technologies. For these new library objects, the primary key problem is precisely reconstructing their 3D models. When constructing complete 3D models, multiple color texture maps are often necessary. A commonly encountered problem uncounted during fusing of textures from multiple color images is color distortion. Each texture of a single 3D model may be obtained under possibly different lighting conditions and color response of the camera. To remove any visible seam and improve color consistency between the textures while avoiding color distortion, we propose a new efficient algorithm to relight all the texture images globally, spread residual light difference, and recolor each image by homogeneous transformation. A relative illumination model was adopted to obtain the relighting function. We choose lαβ color space with minimal correlation between channels for many natural scenes, for calculating the relighting result. Looking into two overlapped images A and B, we can pairwise relight B into A＇s luminosity condition in two steps. We first scale B＇s l channel by the lA/lB ratio of the overlapped region. We can assume A and B are in a same color plane now. Then a homogeneous transformation is applied to B＇s a and fl channels which moves B into A＇s hue and saturation condition. For multiple overlapped color textures, a patch based weighted global relighting method was proposed to minimize the total color difference. The pairwise relighting method was used between each two overlapped images, and the difference in every overlapped region after relighting was weighted and summed up to construct an energy value. We used Nelder-Mead method to find a minimal energy value and the relighting parameters for every image. After global relighting, textures become almost coherent. We simply blended the overlapped region along the texture border to remove small visual seams and get a final result. We illustrate our method by calibrating textures of a painted sculpture acquired with laser scanner. Experimental results were realistic and reliable and showed how this method can fuse multiple textures without color distortion.展开更多
In this paper, we proposed a new approach for face recognition with robust to illumination variation. The improved performance to various lights in recognition is obtained by a novel combination of multicondition reli...In this paper, we proposed a new approach for face recognition with robust to illumination variation. The improved performance to various lights in recognition is obtained by a novel combination of multicondition relighting and optimal feature selection. Multi-condition relighting provides a "coarse" compensation for the variable illumination, and then the optimal feature selection further refines the compensation, and additionally offers the robustness to shadow and highlight, by deemphasizing the local mismatches caused by imprecise lighting compensation, shadow or highlight on recognition. For evaluation, two databases with various illumination mismatches have been used. The results have demonstrated the improved robustness of the new methods.展开更多
Although many photorealistic relighting methods provide a way to change the illumination of objects in a digital photograph, it is currently difficult to relight digital illustrations having a cartoon shading style. T...Although many photorealistic relighting methods provide a way to change the illumination of objects in a digital photograph, it is currently difficult to relight digital illustrations having a cartoon shading style. The main difference between photorealistic and cartoon shading styles is that cartoon shading is characterized by soft color quantization and nonlinear color variations that cause noticeable reconstruction errors under a physical reflectance assumption, such as Lambertian reflection. To handle this non-photorealistic shading property, we focus on shading analysis of the most fundamental cartoon shading technique. Based on the color map shading representation, we propose a simple method to determine the input shading as that of a smooth shape with a nonlinear reflectance property. We have conducted simple ground-truth evaluations to compare our results to those obtained by other approaches.展开更多
An algorithm is presented for estimating the direction and strength of point light with the strength of ambient illumination. Existing approaches evaluate these illumination parameters directly in the high dimensional...An algorithm is presented for estimating the direction and strength of point light with the strength of ambient illumination. Existing approaches evaluate these illumination parameters directly in the high dimensional image space, while we estimate the parameters in two steps： first by projecting the image to an orthogonal linear subspace based on spherical harmonic basis functions and then by calculating the parameters in the low dimensional subspace. The test results using the CMU PIE database and Yale Database B show the stability and effectiveness of the method. The resulting illumination information can be used to synthesize more realistic relighting images and to recognize objects under variable illumination.展开更多
We propose a novel interactive lighting editing system for lighting a single indoor RGB image based on spherical harmonic lighting.It allows users to intuitively edit illumination and relight the complicated low-light...We propose a novel interactive lighting editing system for lighting a single indoor RGB image based on spherical harmonic lighting.It allows users to intuitively edit illumination and relight the complicated low-light indoor scene.Our method not only achieves plausible global relighting but also enhances the local details of the complicated scene according to the spatially-varying spherical harmonic lighting,which only requires a single RGB image along with a corresponding depth map.To this end,we first present a joint optimization algorithm,which is based on the geometric optimization of the depth map and intrinsic image decomposition avoiding texture-copy,for refining the depth map and obtaining the shading map.Then we propose a lighting estimation method based on spherical harmonic lighting,which not only achieves the global illumination estimation of the scene,but also further enhances local details of the complicated scene.Finally,we use a simple and intuitive interactive method to edit the environment lighting map to adjust lighting and relight the scene.Through extensive experimental results,we demonstrate that our proposed approach is simple and intuitive for relighting the low-light indoor scene,and achieve state-of-the-art results.展开更多
Lighting is crucial for portrait photography,yet the complex interactions between the skin and incident light are expensive to model computationally in graphics and difficult to reconstruct analytically via computer v...Lighting is crucial for portrait photography,yet the complex interactions between the skin and incident light are expensive to model computationally in graphics and difficult to reconstruct analytically via computer vision.Alternatively,to allow fast and controllable reflectance and lighting editing,we developed a physically based decomposition through deep learned priors from path-traced portrait images.Previous approaches that used simplified material models or low-frequency or low-dynamic-range lighting struggled to model specular reflections or relight directly without intermediate decomposition.However,we estimate the surface normal,skin albedo and roughness,and high-frequency HDRI maps,and propose an architecture to estimate both diffuse and specular reflectance components.In our experiments,we show that this approach can represent the true appearance function more effectively than simpler baseline methods,leading to better generalization and higher-quality editing.展开更多
We present a method for transferring lighting between photographs of a static scene. Our method takes as input a photo collection depicting a scene with varying viewpoints and lighting conditions.We cast lighting tran...We present a method for transferring lighting between photographs of a static scene. Our method takes as input a photo collection depicting a scene with varying viewpoints and lighting conditions.We cast lighting transfer as an edit propagation problem, where the transfer of local illumination across images is guided by sparse correspondences obtained through multi-view stereo. Instead of directly propagating color, we learn local color transforms from corresponding patches in pairs of images and propagate these transforms in an edge-aware manner to regions with no correspondences. Our color transforms model the large variability of appearance changes in local regions of the scene, and are robust to missing or inaccurate correspondences. The method is fully automatic and can transfer strong shadows between images. We show applications of our image relighting method for enhancing photographs, browsing photo collections with harmonized lighting, and generating synthetic time-lapse sequences.展开更多
基金Project supported by the National Basic Research Program (973) ofChina (No. 2002CB312106)
文摘With the development of digital library technology, library books made of paper can be digital released and read, and Endangered Cultural Heritages can be preserved. Traditional library＇s contents and functions can be greatly enhanced by digital technologies. For these new library objects, the primary key problem is precisely reconstructing their 3D models. When constructing complete 3D models, multiple color texture maps are often necessary. A commonly encountered problem uncounted during fusing of textures from multiple color images is color distortion. Each texture of a single 3D model may be obtained under possibly different lighting conditions and color response of the camera. To remove any visible seam and improve color consistency between the textures while avoiding color distortion, we propose a new efficient algorithm to relight all the texture images globally, spread residual light difference, and recolor each image by homogeneous transformation. A relative illumination model was adopted to obtain the relighting function. We choose lαβ color space with minimal correlation between channels for many natural scenes, for calculating the relighting result. Looking into two overlapped images A and B, we can pairwise relight B into A＇s luminosity condition in two steps. We first scale B＇s l channel by the lA/lB ratio of the overlapped region. We can assume A and B are in a same color plane now. Then a homogeneous transformation is applied to B＇s a and fl channels which moves B into A＇s hue and saturation condition. For multiple overlapped color textures, a patch based weighted global relighting method was proposed to minimize the total color difference. The pairwise relighting method was used between each two overlapped images, and the difference in every overlapped region after relighting was weighted and summed up to construct an energy value. We used Nelder-Mead method to find a minimal energy value and the relighting parameters for every image. After global relighting, textures become almost coherent. We simply blended the overlapped region along the texture border to remove small visual seams and get a final result. We illustrate our method by calibrating textures of a painted sculpture acquired with laser scanner. Experimental results were realistic and reliable and showed how this method can fuse multiple textures without color distortion.
文摘In this paper, we proposed a new approach for face recognition with robust to illumination variation. The improved performance to various lights in recognition is obtained by a novel combination of multicondition relighting and optimal feature selection. Multi-condition relighting provides a "coarse" compensation for the variable illumination, and then the optimal feature selection further refines the compensation, and additionally offers the robustness to shadow and highlight, by deemphasizing the local mismatches caused by imprecise lighting compensation, shadow or highlight on recognition. For evaluation, two databases with various illumination mismatches have been used. The results have demonstrated the improved robustness of the new methods.
基金supported in part by the Japan Science and Technology Agency CREST projectthe Japan Society for the Promotion of Science KAKENHI Grant No.JP15H05924
文摘Although many photorealistic relighting methods provide a way to change the illumination of objects in a digital photograph, it is currently difficult to relight digital illustrations having a cartoon shading style. The main difference between photorealistic and cartoon shading styles is that cartoon shading is characterized by soft color quantization and nonlinear color variations that cause noticeable reconstruction errors under a physical reflectance assumption, such as Lambertian reflection. To handle this non-photorealistic shading property, we focus on shading analysis of the most fundamental cartoon shading technique. Based on the color map shading representation, we propose a simple method to determine the input shading as that of a smooth shape with a nonlinear reflectance property. We have conducted simple ground-truth evaluations to compare our results to those obtained by other approaches.
基金the National Natural Science Foundation of China (No. 60273005)
文摘An algorithm is presented for estimating the direction and strength of point light with the strength of ambient illumination. Existing approaches evaluate these illumination parameters directly in the high dimensional image space, while we estimate the parameters in two steps： first by projecting the image to an orthogonal linear subspace based on spherical harmonic basis functions and then by calculating the parameters in the low dimensional subspace. The test results using the CMU PIE database and Yale Database B show the stability and effectiveness of the method. The resulting illumination information can be used to synthesize more realistic relighting images and to recognize objects under variable illumination.
基金supported by NSFC(No.61972298)Bingtuan Science and Technology Program(No.2019BC008).
文摘We propose a novel interactive lighting editing system for lighting a single indoor RGB image based on spherical harmonic lighting.It allows users to intuitively edit illumination and relight the complicated low-light indoor scene.Our method not only achieves plausible global relighting but also enhances the local details of the complicated scene according to the spatially-varying spherical harmonic lighting,which only requires a single RGB image along with a corresponding depth map.To this end,we first present a joint optimization algorithm,which is based on the geometric optimization of the depth map and intrinsic image decomposition avoiding texture-copy,for refining the depth map and obtaining the shading map.Then we propose a lighting estimation method based on spherical harmonic lighting,which not only achieves the global illumination estimation of the scene,but also further enhances local details of the complicated scene.Finally,we use a simple and intuitive interactive method to edit the environment lighting map to adjust lighting and relight the scene.Through extensive experimental results,we demonstrate that our proposed approach is simple and intuitive for relighting the low-light indoor scene,and achieve state-of-the-art results.
基金supported by NSF CAREER-2144956 and an Andy van Dam PhD Fellowship.
文摘Lighting is crucial for portrait photography,yet the complex interactions between the skin and incident light are expensive to model computationally in graphics and difficult to reconstruct analytically via computer vision.Alternatively,to allow fast and controllable reflectance and lighting editing,we developed a physically based decomposition through deep learned priors from path-traced portrait images.Previous approaches that used simplified material models or low-frequency or low-dynamic-range lighting struggled to model specular reflections or relight directly without intermediate decomposition.However,we estimate the surface normal,skin albedo and roughness,and high-frequency HDRI maps,and propose an architecture to estimate both diffuse and specular reflectance components.In our experiments,we show that this approach can represent the true appearance function more effectively than simpler baseline methods,leading to better generalization and higher-quality editing.
基金the National University of Singapore with support from the School of Computingsupported by the Being There Centre,a collaboration between Nanyang Technological University Singapore,Eidgenossische Technische Hochschule Zürich+2 种基金the University of North Carolina at Chapel Hillsupported by the Singapore National Research Foundation under its International Research Centre@Singapore Funding Initiativethe Interactive Digital Media Programme Office
文摘We present a method for transferring lighting between photographs of a static scene. Our method takes as input a photo collection depicting a scene with varying viewpoints and lighting conditions.We cast lighting transfer as an edit propagation problem, where the transfer of local illumination across images is guided by sparse correspondences obtained through multi-view stereo. Instead of directly propagating color, we learn local color transforms from corresponding patches in pairs of images and propagate these transforms in an edge-aware manner to regions with no correspondences. Our color transforms model the large variability of appearance changes in local regions of the scene, and are robust to missing or inaccurate correspondences. The method is fully automatic and can transfer strong shadows between images. We show applications of our image relighting method for enhancing photographs, browsing photo collections with harmonized lighting, and generating synthetic time-lapse sequences.
