Internal learning-based video inpainting methods have shown promising results by exploiting the intrinsic properties of the video to fill in the missing region without external dataset supervision.However,existing int...Internal learning-based video inpainting methods have shown promising results by exploiting the intrinsic properties of the video to fill in the missing region without external dataset supervision.However,existing internal learning-based video inpainting methods would produce inconsistent structures or blurry textures due to the insufficient utilisation of motion priors within the video sequence.In this paper,the authors propose a new internal learning-based video inpainting model called appearance consistency and motion coherence network(ACMC-Net),which can not only learn the recurrence of appearance prior but can also capture motion coherence prior to improve the quality of the inpainting results.In ACMC-Net,a transformer-based appearance network is developed to capture global context information within the video frame for representing appearance consistency accurately.Additionally,a novel motion coherence learning scheme is proposed to learn the motion prior in a video sequence effectively.Finally,the learnt internal appearance consistency and motion coherence are implicitly propagated to the missing regions to achieve inpainting well.Extensive experiments conducted on the DAVIS dataset show that the proposed model obtains the superior performance in terms of quantitative measurements and produces more visually plausible results compared with the state-of-the-art methods.展开更多
The Pressure Sensitive Paint Technique(PSP)has gained attention in recent years because of its significant benefits in measuring surface pressure on wind tunnel models.However,in the post-processing process of PSP ima...The Pressure Sensitive Paint Technique(PSP)has gained attention in recent years because of its significant benefits in measuring surface pressure on wind tunnel models.However,in the post-processing process of PSP images,issues such as pressure taps,paint peeling,and contamination can lead to the loss of pressure data on the image,which seriously affects the subsequent calculation and analysis of pressure distribution.Therefore,image inpainting is particularly important in the post-processing process of PSP images.Deep learning offers new methods for PSP image inpainting,but some basic characteristics of convolutional neural networks(CNNs)may limit their ability to handle restoration tasks.By contrast,the self-attention mechanism in the transformer can efficiently model nonlocal relationships among input features by generating adaptive attention scores.As a result,we propose an efficient transformer network model for the PSP image inpainting task,named multi-scale dilated attention transformer(D-former).The model utilizes the redundancy of global dependencies modeling in Vision Transformers(ViTs)to introducemulti-scale dilated attention(MDA),thismechanism effectivelymodels the interaction between localized and sparse patches within the shifted window,achieving a better balance between computational complexity and receptive field.As a result,D-former allows efficient modeling of long-range features while using fewer parameters and lower computational costs.The experiments on two public datasets and the PSP dataset indicate that the method in this article performs better compared to several advancedmethods.Through the verification of real wind tunnel tests,thismethod can accurately restore the luminescent intensity data of holes in PSP images,thereby improving the accuracy of full field pressure data,and has a promising future in practical applications.展开更多
Sparse representation has been highly successful in various tasks related to image processing and computer vision.For ancient mural image inpainting,traditional group sparse representation models usually lead to struc...Sparse representation has been highly successful in various tasks related to image processing and computer vision.For ancient mural image inpainting,traditional group sparse representation models usually lead to structure blur and line discontinuity due to the construction of similarity group solely based on the Euclidean distance and the randomness of dictionary initialization.To address the aforementioned issues,an improved curvature Gabor transform and group sparse representation(CGabor-GSR)model for ancient Dunhuang mural inpainting is proposed.To begin with,mutual information is introduced to weight the Euclidean distance,and then the weighted Euclidean distance acts as a new standard of similarity group.Subsequently,to mitigate the randomness of dictionary initialization,a curvature Gabor wavelet transform is proposed to extract the features and initialize the feature dictionary with dimension reduction based on principal component analysis(PCA).Ultimately,singular value decomposition(SVD)and split Bregman iteration(SBI)can be used to resolve the CGabor-GSR model to reconstruct the mural images.Experimental results on Dunhuang mural inpainting demonstrate tha the proposed CGabor-GSR achieves a better performance than compared algorithms in both objective and visual evaluation.展开更多
For the existing deep learning image restoration methods,the joint guidance of structure and texture information is not considered,which leads to structural disorder and texture blur in the restoration results.A gener...For the existing deep learning image restoration methods,the joint guidance of structure and texture information is not considered,which leads to structural disorder and texture blur in the restoration results.A generative adversarial mural inpainting algorithm based on structural and texture hybrid enhancement was proposed.Firstly,the structure guidance branch composed of dynamic convolution cascade was constructed to improve the expression ability of structure features,and the structure information was used to guide the encoder coding to enhance the edge contour information of the coding feature map.Then,the multi-granularity feature extraction module was designed to obtain the texture features of texture guided branches,and the multi-scale texture information was used to guide the decoder to reconstruct and repair,so as to improve the texture consistency of murals.Finally,skip connection was used to promote the feature sharing of structure and texture features,and the spectral-normalized PatchGAN discriminator was used to complete the mural restoration.The digital restoration experiment results of real Dunhuang murals showed that the proposed method was better than the comparison algorithms in both subjective and objective evaluation,and the restoration results were clearer and more natural.展开更多
Image inpainting refers to synthesizing missing content in an image based on known information to restore occluded or damaged regions,which is a typical manifestation of this trend.With the increasing complexity of im...Image inpainting refers to synthesizing missing content in an image based on known information to restore occluded or damaged regions,which is a typical manifestation of this trend.With the increasing complexity of image in tasks and the growth of data scale,existing deep learning methods still have some limitations.For example,they lack the ability to capture long-range dependencies and their performance in handling multi-scale image structures is suboptimal.To solve this problem,the paper proposes an image inpainting method based on the parallel dual-branch learnable Transformer network.The encoder of the proposed model generator consists of a dual-branch parallel structure with stacked CNN blocks and Transformer blocks,aiming to extract global and local feature information from images.Furthermore,a dual-branch fusion module is adopted to combine the features obtained from both branches.Additionally,a gated full-scale skip connection module is proposed to further enhance the coherence of the inpainting results and alleviate information loss.Finally,experimental results from the three public datasets demonstrate the superior performance of the proposed method.展开更多
Limited by the dynamic range of the detector,saturation artifacts usually occur in optical coherence tomography(OCT)imaging for high scattering media.The available methods are difficult to remove saturation artifacts ...Limited by the dynamic range of the detector,saturation artifacts usually occur in optical coherence tomography(OCT)imaging for high scattering media.The available methods are difficult to remove saturation artifacts and restore texture completely in OCT images.We proposed a deep learning-based inpainting method of saturation artifacts in this paper.The generation mechanism of saturation artifacts was analyzed,and experimental and simulated datasets were built based on the mechanism.Enhanced super-resolution generative adversarial networks were trained by the clear–saturated phantom image pairs.The perfect reconstructed results of experimental zebrafish and thyroid OCT images proved its feasibility,strong generalization,and robustness.展开更多
Recently,deep learning-based image inpainting methods have made great strides in reconstructing damaged regions.However,these methods often struggle to produce satisfactory results when dealing with missing images wit...Recently,deep learning-based image inpainting methods have made great strides in reconstructing damaged regions.However,these methods often struggle to produce satisfactory results when dealing with missing images with large holes,leading to distortions in the structure and blurring of textures.To address these problems,we combine the advantages of transformers and convolutions to propose an image inpainting method that incorporates edge priors and attention mechanisms.The proposed method aims to improve the results of inpainting large holes in images by enhancing the accuracy of structure restoration and the ability to recover texture details.This method divides the inpainting task into two phases:edge prediction and image inpainting.Specifically,in the edge prediction phase,a transformer architecture is designed to combine axial attention with standard self-attention.This design enhances the extraction capability of global structural features and location awareness.It also balances the complexity of self-attention operations,resulting in accurate prediction of the edge structure in the defective region.In the image inpainting phase,a multi-scale fusion attention module is introduced.This module makes full use of multi-level distant features and enhances local pixel continuity,thereby significantly improving the quality of image inpainting.To evaluate the performance of our method.comparative experiments are conducted on several datasets,including CelebA,Places2,and Facade.Quantitative experiments show that our method outperforms the other mainstream methods.Specifically,it improves Peak Signal-to-Noise Ratio(PSNR)and Structure Similarity Index Measure(SSIM)by 1.141~3.234 db and 0.083~0.235,respectively.Moreover,it reduces Learning Perceptual Image Patch Similarity(LPIPS)and Mean Absolute Error(MAE)by 0.0347~0.1753 and 0.0104~0.0402,respectively.Qualitative experiments reveal that our method excels at reconstructing images with complete structural information and clear texture details.Furthermore,our model exhibits impressive performance in terms of the number of parameters,memory cost,and testing time.展开更多
Image inpainting is a kind of use known area of information technology to repair the loss or damage to the area.Image feature extraction is the core of image restoration.Getting enough space for information and a larg...Image inpainting is a kind of use known area of information technology to repair the loss or damage to the area.Image feature extraction is the core of image restoration.Getting enough space for information and a larger receptive field is very important to realize high-precision image inpainting.However,in the process of feature extraction,it is difficult to meet the two requirements of obtaining sufficient spatial information and large receptive fields at the same time.In order to obtain more spatial information and a larger receptive field at the same time,we put forward a kind of image restoration based on space path and context path network.For the space path,we stack three convolution layers for 1/8 of the figure,the figure retained the rich spatial details.For the context path,we use the global average pooling layer,where the accept field is the maximum of the backbone network,and the pooling module can provide global context information for the maximum accept field.In order to better integrate the features extracted from the spatial and contextual paths,we study the fusion module of the two paths.Features fusionmodule first path output of the space and context path,and then through themass normalization to balance the scale of the characteristics,finally the characteristics of the pool will be connected into a feature vector and calculate the weight vector.Features of images in order to extract context information,we add attention to the context path refinement module.Attention modules respectively from channel dimension and space dimension to weighted images,in order to obtain more effective information.Experiments show that our method is better than the existing technology in the quality and quantity of themethod,and further to expand our network to other inpainting networks,in order to achieve consistent performance improvements.展开更多
A new algorithm is proposed for restoring disocclusion regions in depth-image-based rendering （DIBR） warped images. Current solutions include layered depth image （LDI）, pre-filtering methods, and post-processing m...A new algorithm is proposed for restoring disocclusion regions in depth-image-based rendering （DIBR） warped images. Current solutions include layered depth image （LDI）, pre-filtering methods, and post-processing methods. The LDI is complicated, and pre-filtering of depth images causes noticeable geometrical distortions in cases of large baseline warping. This paper presents a depth-aided inpainting method which inherits merits from Criminisi＇s inpainting algorithm. The proposed method features incorporation of a depth cue into texture estimation. The algorithm efficiently handles depth ambiguity by penalizing larger Lagrange multipliers of flling points closer to the warping position compared with the surrounding existing points. We perform morphological operations on depth images to accelerate the algorithm convergence, and adopt a luma-first strategy to adapt to various color sampling formats. Experiments on test multi-view sequence showed that our method has superiority in depth differentiation and geometrical loyalty in the restoration of warped images. Also, peak signal-to-noise ratio （PSNR） statistics on non-hole regions and whole image comparisons both compare favorably to those obtained by state of the art techniques.展开更多
The priority of the filled patch play a key role in the exemplar-based image inpainting, and it should be determined firstly to optimize the process of image inpainting. A modified image inpainting algorithm is propos...The priority of the filled patch play a key role in the exemplar-based image inpainting, and it should be determined firstly to optimize the process of image inpainting. A modified image inpainting algorithm is proposed by weighted-priority based on the Criminisi algorithm. The improved algorithm demonstrates better relationship between the data term and the confidence term for the optimization of the priority than the classical Criminisi algorithm. By comparing the effect of the inpainted images with different structure, conclusion can be drawn that the optimal priority should be chosen properly for different images with different structures.展开更多
Aiming at the problem of small area human occlusion in gait recognition,a method based on generating adversarial image inpainting network was proposed which can generate a context consistent image for gait occlusion a...Aiming at the problem of small area human occlusion in gait recognition,a method based on generating adversarial image inpainting network was proposed which can generate a context consistent image for gait occlusion area.In order to reduce the effect of noise on feature extraction,the stacked automatic encoder with robustness was used.In order to improve the ability of gait classification,the sparse coding was used to express and classify the gait features.Experiments results showed the effectiveness of the proposed method in comparison with other state-of-the-art methods on the public databases CASIA-B and TUM-GAID for gait recognition.展开更多
Images created from measurements made by wireline microresistivity imaging tools have longitudinal gaps when the well circumference exceeds the total width of the pad-mounted electrode arrays.The gap size depends on t...Images created from measurements made by wireline microresistivity imaging tools have longitudinal gaps when the well circumference exceeds the total width of the pad-mounted electrode arrays.The gap size depends on the tool design and borehole size,and the null data in these gaps negatively aff ect the quantitative evaluation of reservoirs.Images with linear and texture features obtained from microresistivity image logs have distinct dual fabric features because of logging principles and various geological phenomena.Linear image features usually include phenomena such as fractures,bedding,and unconformities.Contrarily,texture-based image features usually indicate phenomena such as vugs and rock matrices.According to the characteristics of this fabric-based binary image structure and guided by the practice of geological interpretation,an adaptive inpainting method for the blank gaps in microresistivity image logs is proposed.For images with linear features,a sinusoidal tracking inpainting algorithm based on an evaluation of the validity and continuity of pixel sets is used.Contrarily,the most similar target transplantation algorithm is applied to texture-based images.The results obtained for measured electrical imaging data showed that the full borehole image obtained by the proposed method,whether it was a linear structural image refl ecting fracture and bedding or texture-based image refl ecting the matrix and pore of rock,had substantially good inpainting quality with enhanced visual connectivity.The proposed method was eff ective for inpainting electrical image logs with large gaps and high angle fractures with high heterogeneity.Moreover,ladder and block artifacts were rare,and the inpainting marks were not obvious.In addition,detailed full borehole images obtained by the proposed method will provide an essential basis for interpreting geological phenomena and reservoir parameters.展开更多
In the exemplar-based image inpainting approach,there are usually two major problems:the unreasonable calculation of priority and only considering the color features in the patch lookup strategy.In this paper,we propo...In the exemplar-based image inpainting approach,there are usually two major problems:the unreasonable calculation of priority and only considering the color features in the patch lookup strategy.In this paper,we propose an image inpainting approach based on the structural tensor edge intensity model.First,we use the progressive scanning inpainting method to avoid the image filling order being affected by the priority function.Then,we use the edge intensity model to build the patches similarity function for correctly identifying the local image structure.Finally,the balance operator is used to restrict the excessive propagation of structural information to ensure the correct structural reconstruction.The experimental results show that the our approach is comparable and even superior to some state-of-the-art inpainting algorithms.展开更多
Inpainting has been continuously studied in the field of computer vision.As artificial intelligence technology developed,deep learning technology was introduced in inpainting research,helping to improve performance.Cu...Inpainting has been continuously studied in the field of computer vision.As artificial intelligence technology developed,deep learning technology was introduced in inpainting research,helping to improve performance.Currently,the input target of an inpainting algorithm using deep learning has been studied from a single image to a video.However,deep learning-based inpainting technology for panoramic images has not been actively studied.We propose a 360-degree panoramic image inpainting method using generative adversarial networks(GANs).The proposed network inputs a 360-degree equirectangular format panoramic image converts it into a cube map format,which has relatively little distortion and uses it as a training network.Since the cube map format is used,the correlation of the six sides of the cube map should be considered.Therefore,all faces of the cube map are used as input for the whole discriminative network,and each face of the cube map is used as input for the slice discriminative network to determine the authenticity of the generated image.The proposed network performed qualitatively better than existing single-image inpainting algorithms and baseline algorithms.展开更多
A simple and effective image inpainting method is proposed in this paper, which is proved to be suitable for different kinds of target regions with shapes from little scraps to large unseemly objects in a wide range o...A simple and effective image inpainting method is proposed in this paper, which is proved to be suitable for different kinds of target regions with shapes from little scraps to large unseemly objects in a wide range of images. It is an important improvement upon the traditional image inpainting techniques. By introducing a new bijeetive-mapping term into the matching cost function, the artificial repetition problem in the final inpainting image is practically solved. In addition, by adopting an inpainting error map, not only the target pixels are refined gradually during the inpainting process but also the overlapped target patches are combined more seamlessly than previous method. Finally, the inpainting time is dramatically decreased by using a new acceleration method in the matching process.展开更多
We propose a layered image inpainting scheme based on image decomposition. The damaged image is first decomposed into three layers： cartoon, edge, and texture. The cartoon and edge layers are repaired using an adapti...We propose a layered image inpainting scheme based on image decomposition. The damaged image is first decomposed into three layers： cartoon, edge, and texture. The cartoon and edge layers are repaired using an adaptive offset operator that can fill-in damaged image blocks while preserving sharpness of edges. The missing information in the texture layer is generated with a texture synthesis method. By using discrete cosine transform （DCT） in image decomposition and trading between resolution and computation complexity in texture synthesis, the processing time is kept at a reasonable level.展开更多
The depth information of the scene indicates the distance between the object and the camera,and depth extraction is a key technology in 3D video system.The emergence of Kinect makes the high resolution depth map captu...The depth information of the scene indicates the distance between the object and the camera,and depth extraction is a key technology in 3D video system.The emergence of Kinect makes the high resolution depth map capturing possible.However,the depth map captured by Kinect can not be directly used due to the existing holes and noises,which needs to be repaired.We propose a texture combined inpainting algorithm in this paper.Firstly,the foreground is segmented combined with the color characteristics of the texture image to repair the foreground of the depth map.Secondly,region growing is used to determine the match region of the hole in the depth map,and to accurately position the match region according to the texture information.Then the match region is weighted to fill the hole.Finally,a Gaussian filter is used to remove the noise in the depth map.Experimental results show that the proposed method can effectively repair the holes existing in the original depth map and get an accurate and smooth depth map,which can be used to render a virtual image with good quality.展开更多
The classical TV (Total Variation) model has been applied to gray texture image denoising and inpainting previously based on the non local operators, but such model can not be directly used to color texture image inpa...The classical TV (Total Variation) model has been applied to gray texture image denoising and inpainting previously based on the non local operators, but such model can not be directly used to color texture image inpainting due to coupling of different image layers in color images. In order to solve the inpainting problem for color texture images effectively, we propose a non local CTV (Color Total Variation) model. Technically, the proposed model is an extension of local TV model for gray images but we take account of the coupling of different layers in color images and make use of concepts of the non-local operators. As the coupling of different layers for color images in the proposed model will in-crease computational complexity, we also design a fast Split Bregman algorithm. Finally, some numerical experiments are conducted to validate the performance of the proposed model and its algorithm.展开更多
基金Shenzhen Science and Technology Programme,Grant/Award Number:JCYJ202308071208000012023 Shenzhen sustainable supporting funds for colleges and universities,Grant/Award Number:20231121165240001Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology,Grant/Award Number:2024B1212010006。
文摘Internal learning-based video inpainting methods have shown promising results by exploiting the intrinsic properties of the video to fill in the missing region without external dataset supervision.However,existing internal learning-based video inpainting methods would produce inconsistent structures or blurry textures due to the insufficient utilisation of motion priors within the video sequence.In this paper,the authors propose a new internal learning-based video inpainting model called appearance consistency and motion coherence network(ACMC-Net),which can not only learn the recurrence of appearance prior but can also capture motion coherence prior to improve the quality of the inpainting results.In ACMC-Net,a transformer-based appearance network is developed to capture global context information within the video frame for representing appearance consistency accurately.Additionally,a novel motion coherence learning scheme is proposed to learn the motion prior in a video sequence effectively.Finally,the learnt internal appearance consistency and motion coherence are implicitly propagated to the missing regions to achieve inpainting well.Extensive experiments conducted on the DAVIS dataset show that the proposed model obtains the superior performance in terms of quantitative measurements and produces more visually plausible results compared with the state-of-the-art methods.
基金partly supported by the National Natural Science Foundation of China under Grant 12202476,author Chunhua Wei,https://www.nsfc.gov.cn/.
文摘The Pressure Sensitive Paint Technique(PSP)has gained attention in recent years because of its significant benefits in measuring surface pressure on wind tunnel models.However,in the post-processing process of PSP images,issues such as pressure taps,paint peeling,and contamination can lead to the loss of pressure data on the image,which seriously affects the subsequent calculation and analysis of pressure distribution.Therefore,image inpainting is particularly important in the post-processing process of PSP images.Deep learning offers new methods for PSP image inpainting,but some basic characteristics of convolutional neural networks(CNNs)may limit their ability to handle restoration tasks.By contrast,the self-attention mechanism in the transformer can efficiently model nonlocal relationships among input features by generating adaptive attention scores.As a result,we propose an efficient transformer network model for the PSP image inpainting task,named multi-scale dilated attention transformer(D-former).The model utilizes the redundancy of global dependencies modeling in Vision Transformers(ViTs)to introducemulti-scale dilated attention(MDA),thismechanism effectivelymodels the interaction between localized and sparse patches within the shifted window,achieving a better balance between computational complexity and receptive field.As a result,D-former allows efficient modeling of long-range features while using fewer parameters and lower computational costs.The experiments on two public datasets and the PSP dataset indicate that the method in this article performs better compared to several advancedmethods.Through the verification of real wind tunnel tests,thismethod can accurately restore the luminescent intensity data of holes in PSP images,thereby improving the accuracy of full field pressure data,and has a promising future in practical applications.
基金supported by National Natural Science Foundation of China(No.61963023)Humanities and Social Sciences Youth Foundation of Ministry of Education(No.19YJC760012)Lanzhou Jiaotong University Basic Top-Notch Personnel Project(No.2022JC36).
文摘Sparse representation has been highly successful in various tasks related to image processing and computer vision.For ancient mural image inpainting,traditional group sparse representation models usually lead to structure blur and line discontinuity due to the construction of similarity group solely based on the Euclidean distance and the randomness of dictionary initialization.To address the aforementioned issues,an improved curvature Gabor transform and group sparse representation(CGabor-GSR)model for ancient Dunhuang mural inpainting is proposed.To begin with,mutual information is introduced to weight the Euclidean distance,and then the weighted Euclidean distance acts as a new standard of similarity group.Subsequently,to mitigate the randomness of dictionary initialization,a curvature Gabor wavelet transform is proposed to extract the features and initialize the feature dictionary with dimension reduction based on principal component analysis(PCA).Ultimately,singular value decomposition(SVD)and split Bregman iteration(SBI)can be used to resolve the CGabor-GSR model to reconstruct the mural images.Experimental results on Dunhuang mural inpainting demonstrate tha the proposed CGabor-GSR achieves a better performance than compared algorithms in both objective and visual evaluation.
基金supported by Ministry of Education in China Project of Humanities and Social Sciences(No.19YJC760012)Star of Innovation Project for Outstanding Graduate Students in Gansu Province(No.2022CXZX-546)。
文摘For the existing deep learning image restoration methods,the joint guidance of structure and texture information is not considered,which leads to structural disorder and texture blur in the restoration results.A generative adversarial mural inpainting algorithm based on structural and texture hybrid enhancement was proposed.Firstly,the structure guidance branch composed of dynamic convolution cascade was constructed to improve the expression ability of structure features,and the structure information was used to guide the encoder coding to enhance the edge contour information of the coding feature map.Then,the multi-granularity feature extraction module was designed to obtain the texture features of texture guided branches,and the multi-scale texture information was used to guide the decoder to reconstruct and repair,so as to improve the texture consistency of murals.Finally,skip connection was used to promote the feature sharing of structure and texture features,and the spectral-normalized PatchGAN discriminator was used to complete the mural restoration.The digital restoration experiment results of real Dunhuang murals showed that the proposed method was better than the comparison algorithms in both subjective and objective evaluation,and the restoration results were clearer and more natural.
基金supported by Scientific Research Fund of Hunan Provincial Natural Science Foundation under Grant 20231J60257Hunan Provincial Engineering Research Center for Intelligent Rehabilitation Robotics and Assistive Equipment under Grant 2025SH501Inha University and Design of a Conflict Detection and Validation Tool under Grant HX2024123.
文摘Image inpainting refers to synthesizing missing content in an image based on known information to restore occluded or damaged regions,which is a typical manifestation of this trend.With the increasing complexity of image in tasks and the growth of data scale,existing deep learning methods still have some limitations.For example,they lack the ability to capture long-range dependencies and their performance in handling multi-scale image structures is suboptimal.To solve this problem,the paper proposes an image inpainting method based on the parallel dual-branch learnable Transformer network.The encoder of the proposed model generator consists of a dual-branch parallel structure with stacked CNN blocks and Transformer blocks,aiming to extract global and local feature information from images.Furthermore,a dual-branch fusion module is adopted to combine the features obtained from both branches.Additionally,a gated full-scale skip connection module is proposed to further enhance the coherence of the inpainting results and alleviate information loss.Finally,experimental results from the three public datasets demonstrate the superior performance of the proposed method.
基金supported by the National Natural Science Foundation of China(62375144 and 61875092)Tianjin Foundation of Natural Science(21JCYBJC00260)Beijing-Tianjin-Hebei Basic Research Cooperation Special Program(19JCZDJC65300).
文摘Limited by the dynamic range of the detector,saturation artifacts usually occur in optical coherence tomography(OCT)imaging for high scattering media.The available methods are difficult to remove saturation artifacts and restore texture completely in OCT images.We proposed a deep learning-based inpainting method of saturation artifacts in this paper.The generation mechanism of saturation artifacts was analyzed,and experimental and simulated datasets were built based on the mechanism.Enhanced super-resolution generative adversarial networks were trained by the clear–saturated phantom image pairs.The perfect reconstructed results of experimental zebrafish and thyroid OCT images proved its feasibility,strong generalization,and robustness.
基金supported in part by the National Natural Science Foundation of China under Grant 62062061/in part by the Major Project Cultivation Fund of Xizang Minzu University under Grant 324112300447.
文摘Recently,deep learning-based image inpainting methods have made great strides in reconstructing damaged regions.However,these methods often struggle to produce satisfactory results when dealing with missing images with large holes,leading to distortions in the structure and blurring of textures.To address these problems,we combine the advantages of transformers and convolutions to propose an image inpainting method that incorporates edge priors and attention mechanisms.The proposed method aims to improve the results of inpainting large holes in images by enhancing the accuracy of structure restoration and the ability to recover texture details.This method divides the inpainting task into two phases:edge prediction and image inpainting.Specifically,in the edge prediction phase,a transformer architecture is designed to combine axial attention with standard self-attention.This design enhances the extraction capability of global structural features and location awareness.It also balances the complexity of self-attention operations,resulting in accurate prediction of the edge structure in the defective region.In the image inpainting phase,a multi-scale fusion attention module is introduced.This module makes full use of multi-level distant features and enhances local pixel continuity,thereby significantly improving the quality of image inpainting.To evaluate the performance of our method.comparative experiments are conducted on several datasets,including CelebA,Places2,and Facade.Quantitative experiments show that our method outperforms the other mainstream methods.Specifically,it improves Peak Signal-to-Noise Ratio(PSNR)and Structure Similarity Index Measure(SSIM)by 1.141~3.234 db and 0.083~0.235,respectively.Moreover,it reduces Learning Perceptual Image Patch Similarity(LPIPS)and Mean Absolute Error(MAE)by 0.0347~0.1753 and 0.0104~0.0402,respectively.Qualitative experiments reveal that our method excels at reconstructing images with complete structural information and clear texture details.Furthermore,our model exhibits impressive performance in terms of the number of parameters,memory cost,and testing time.
基金supported by the National Natural Science Foundation of China under Grants 62172059 and 62072055Scientific Research Fund of Hunan Provincial Education Department of China under Grant 22A0200.
文摘Image inpainting is a kind of use known area of information technology to repair the loss or damage to the area.Image feature extraction is the core of image restoration.Getting enough space for information and a larger receptive field is very important to realize high-precision image inpainting.However,in the process of feature extraction,it is difficult to meet the two requirements of obtaining sufficient spatial information and large receptive fields at the same time.In order to obtain more spatial information and a larger receptive field at the same time,we put forward a kind of image restoration based on space path and context path network.For the space path,we stack three convolution layers for 1/8 of the figure,the figure retained the rich spatial details.For the context path,we use the global average pooling layer,where the accept field is the maximum of the backbone network,and the pooling module can provide global context information for the maximum accept field.In order to better integrate the features extracted from the spatial and contextual paths,we study the fusion module of the two paths.Features fusionmodule first path output of the space and context path,and then through themass normalization to balance the scale of the characteristics,finally the characteristics of the pool will be connected into a feature vector and calculate the weight vector.Features of images in order to extract context information,we add attention to the context path refinement module.Attention modules respectively from channel dimension and space dimension to weighted images,in order to obtain more effective information.Experiments show that our method is better than the existing technology in the quality and quantity of themethod,and further to expand our network to other inpainting networks,in order to achieve consistent performance improvements.
基金Project supported by the National Natural Science Foundation of China (No 60802013)the Natural Science Foundation of Zhe-jiang Province, China (No Y106574)
文摘A new algorithm is proposed for restoring disocclusion regions in depth-image-based rendering （DIBR） warped images. Current solutions include layered depth image （LDI）, pre-filtering methods, and post-processing methods. The LDI is complicated, and pre-filtering of depth images causes noticeable geometrical distortions in cases of large baseline warping. This paper presents a depth-aided inpainting method which inherits merits from Criminisi＇s inpainting algorithm. The proposed method features incorporation of a depth cue into texture estimation. The algorithm efficiently handles depth ambiguity by penalizing larger Lagrange multipliers of flling points closer to the warping position compared with the surrounding existing points. We perform morphological operations on depth images to accelerate the algorithm convergence, and adopt a luma-first strategy to adapt to various color sampling formats. Experiments on test multi-view sequence showed that our method has superiority in depth differentiation and geometrical loyalty in the restoration of warped images. Also, peak signal-to-noise ratio （PSNR） statistics on non-hole regions and whole image comparisons both compare favorably to those obtained by state of the art techniques.
基金Supported by the National Natural Science Foundation of China (No. 60972106)Postdoctoral Science Foundation (No. 20090450750)the Science Foundation of Tianjin(No. 11JCYBJC00900)
文摘The priority of the filled patch play a key role in the exemplar-based image inpainting, and it should be determined firstly to optimize the process of image inpainting. A modified image inpainting algorithm is proposed by weighted-priority based on the Criminisi algorithm. The improved algorithm demonstrates better relationship between the data term and the confidence term for the optimization of the priority than the classical Criminisi algorithm. By comparing the effect of the inpainted images with different structure, conclusion can be drawn that the optimal priority should be chosen properly for different images with different structures.
基金Project(51678075) supported by the National Natural Science Foundation of ChinaProject(2017GK2271) supported by Hunan Provincial Science and Technology Department,China
文摘Aiming at the problem of small area human occlusion in gait recognition,a method based on generating adversarial image inpainting network was proposed which can generate a context consistent image for gait occlusion area.In order to reduce the effect of noise on feature extraction,the stacked automatic encoder with robustness was used.In order to improve the ability of gait classification,the sparse coding was used to express and classify the gait features.Experiments results showed the effectiveness of the proposed method in comparison with other state-of-the-art methods on the public databases CASIA-B and TUM-GAID for gait recognition.
基金This work was supported by Initial Scientifi c Research Fund for Doctor of Xinjiang University(No.620321016)Gansu Provincial Natural Science Foundation of China(No.17JR5RA313)Key Laboratory of Petroleum Resource Research of Chinese Academy of Science Foundation(No.KFJJ2016-02).
文摘Images created from measurements made by wireline microresistivity imaging tools have longitudinal gaps when the well circumference exceeds the total width of the pad-mounted electrode arrays.The gap size depends on the tool design and borehole size,and the null data in these gaps negatively aff ect the quantitative evaluation of reservoirs.Images with linear and texture features obtained from microresistivity image logs have distinct dual fabric features because of logging principles and various geological phenomena.Linear image features usually include phenomena such as fractures,bedding,and unconformities.Contrarily,texture-based image features usually indicate phenomena such as vugs and rock matrices.According to the characteristics of this fabric-based binary image structure and guided by the practice of geological interpretation,an adaptive inpainting method for the blank gaps in microresistivity image logs is proposed.For images with linear features,a sinusoidal tracking inpainting algorithm based on an evaluation of the validity and continuity of pixel sets is used.Contrarily,the most similar target transplantation algorithm is applied to texture-based images.The results obtained for measured electrical imaging data showed that the full borehole image obtained by the proposed method,whether it was a linear structural image refl ecting fracture and bedding or texture-based image refl ecting the matrix and pore of rock,had substantially good inpainting quality with enhanced visual connectivity.The proposed method was eff ective for inpainting electrical image logs with large gaps and high angle fractures with high heterogeneity.Moreover,ladder and block artifacts were rare,and the inpainting marks were not obvious.In addition,detailed full borehole images obtained by the proposed method will provide an essential basis for interpreting geological phenomena and reservoir parameters.
基金This work was supported by National Science Foundation of China(Nos.61401150,61602157 and 61872311)Key Science and Technology Program of Henan Province(Nos.182102210053 and 202102210167)Excellent Young Teachers Program of Henan Polytechnic University(No.2019XQG-02).
文摘In the exemplar-based image inpainting approach,there are usually two major problems:the unreasonable calculation of priority and only considering the color features in the patch lookup strategy.In this paper,we propose an image inpainting approach based on the structural tensor edge intensity model.First,we use the progressive scanning inpainting method to avoid the image filling order being affected by the priority function.Then,we use the edge intensity model to build the patches similarity function for correctly identifying the local image structure.Finally,the balance operator is used to restrict the excessive propagation of structural information to ensure the correct structural reconstruction.The experimental results show that the our approach is comparable and even superior to some state-of-the-art inpainting algorithms.
基金Korea Electric Power Corporation(Grant No.R18XA02).
文摘Inpainting has been continuously studied in the field of computer vision.As artificial intelligence technology developed,deep learning technology was introduced in inpainting research,helping to improve performance.Currently,the input target of an inpainting algorithm using deep learning has been studied from a single image to a video.However,deep learning-based inpainting technology for panoramic images has not been actively studied.We propose a 360-degree panoramic image inpainting method using generative adversarial networks(GANs).The proposed network inputs a 360-degree equirectangular format panoramic image converts it into a cube map format,which has relatively little distortion and uses it as a training network.Since the cube map format is used,the correlation of the six sides of the cube map should be considered.Therefore,all faces of the cube map are used as input for the whole discriminative network,and each face of the cube map is used as input for the slice discriminative network to determine the authenticity of the generated image.The proposed network performed qualitatively better than existing single-image inpainting algorithms and baseline algorithms.
基金Supported by the National Natural Science Foundation of China （No. 60403044, No. 60373070） and partly funded by Microsoft Research Asia： Project 2004-Image-01.
文摘A simple and effective image inpainting method is proposed in this paper, which is proved to be suitable for different kinds of target regions with shapes from little scraps to large unseemly objects in a wide range of images. It is an important improvement upon the traditional image inpainting techniques. By introducing a new bijeetive-mapping term into the matching cost function, the artificial repetition problem in the final inpainting image is practically solved. In addition, by adopting an inpainting error map, not only the target pixels are refined gradually during the inpainting process but also the overlapped target patches are combined more seamlessly than previous method. Finally, the inpainting time is dramatically decreased by using a new acceleration method in the matching process.
基金Project supported by the Shanghai Leading Academic Discipline Project(Grant No.T0102)
文摘We propose a layered image inpainting scheme based on image decomposition. The damaged image is first decomposed into three layers： cartoon, edge, and texture. The cartoon and edge layers are repaired using an adaptive offset operator that can fill-in damaged image blocks while preserving sharpness of edges. The missing information in the texture layer is generated with a texture synthesis method. By using discrete cosine transform （DCT） in image decomposition and trading between resolution and computation complexity in texture synthesis, the processing time is kept at a reasonable level.
基金Supported by the Key Project of National Natural Science Foundation of China(Nos.60832003 and 61172096)major Project of Shanghai Science and Technology Committee(No.10510500500)the Major Innovation Project of Shanghai Municipal Education Commission
文摘The depth information of the scene indicates the distance between the object and the camera,and depth extraction is a key technology in 3D video system.The emergence of Kinect makes the high resolution depth map capturing possible.However,the depth map captured by Kinect can not be directly used due to the existing holes and noises,which needs to be repaired.We propose a texture combined inpainting algorithm in this paper.Firstly,the foreground is segmented combined with the color characteristics of the texture image to repair the foreground of the depth map.Secondly,region growing is used to determine the match region of the hole in the depth map,and to accurately position the match region according to the texture information.Then the match region is weighted to fill the hole.Finally,a Gaussian filter is used to remove the noise in the depth map.Experimental results show that the proposed method can effectively repair the holes existing in the original depth map and get an accurate and smooth depth map,which can be used to render a virtual image with good quality.
文摘The classical TV (Total Variation) model has been applied to gray texture image denoising and inpainting previously based on the non local operators, but such model can not be directly used to color texture image inpainting due to coupling of different image layers in color images. In order to solve the inpainting problem for color texture images effectively, we propose a non local CTV (Color Total Variation) model. Technically, the proposed model is an extension of local TV model for gray images but we take account of the coupling of different layers in color images and make use of concepts of the non-local operators. As the coupling of different layers for color images in the proposed model will in-crease computational complexity, we also design a fast Split Bregman algorithm. Finally, some numerical experiments are conducted to validate the performance of the proposed model and its algorithm.
