摘要
图像组合是图像处理中一个重要操作,然而组合图像中前景区域与背景区域的外观不协调使得组合图像看起来不真实。图像协调化是图像组合中极其重要的一个环节,其目的是调整组合图像前景区域的外观使其与背景区域一致,从而让组合图像在视觉上看起来真实。然而,现有方法只考虑了组合图像前景与背景之间的外观差异,忽略了图像局部的亮度变化差异,这使得图像整体的光照不协调。为此,该文提出一个新的多尺度特征校准模块(MFCM)学习不同尺度的感受野之间细微的特征差异。基于所提模块,该文进一步设计了一个新的编码器学习组合图像中前景与背景的外观差异和局部亮度变化,然后利用解码器重构出图像,并通过一个对前景区域归一化的回归损失指导网络学习调整前景区域的外观。在广泛使用的iHarmony4数据集上进行实验验证,结果表明该方法的效果超过了目前最优的方法,验证了该方法的有效性。
Image composition is an important operation in image processing,but the inharmonious appearance between the foreground region and background makes the composite image look unrealistic.Image harmonization is a very important step in image compositing,and targets at adjusting the appearances of foreground to make it consistent with background,improving the visual quality of output image.However,previous approaches only consider the appearance difference between the foreground and the background of the composite image,and neglect the local brightness change of the image,making the illumination of the whole image inharmonious.In order to solve the problem,in this work,a novel module named Multi-scale Feature Calibration Module(MFCM)is proposed to learn the subtle feature differences between multiple scales of receptive field.Based on the proposed MFCM,a novel encoder is designed further to learn the illumination and brightness change in composite image,followed by a decoder is used to reconstruct image.The foreground normalized regression loss is utilized to instruct the network to learn and adjust the appearances of the foreground.The proposed method is validated on a widely used iHarmony4 dataset.The results show that the proposed method achieves the state of the art and demonstrate the effectiveness of the proposed method.
作者
高陈强
谢承娟
杨烽
赵悦
李鹏程
GAO Chenqiang;XIE Chengjuan;YANG Feng;ZHAO Yue;LI Pengcheng(School of Communication and Information Engineering,Chongqing University of Posts and Telecommunications,Chongqing 400065,China;Chongqing Key Laboratory of Signal and Information Processing,Chongqing 400065,China)
出处
《电子与信息学报》
EI
CSCD
北大核心
2022年第4期1495-1502,共8页
Journal of Electronics & Information Technology
基金
国家自然科学基金(62176035,61906025)
重庆市科委自然科学基金项目(cstc2020jcyj-msxmX0835,cstc2021jcyj-bsh0155)
重庆市教委科学技术研究项目(KJQN201900607,KJZD-K202100606,KJQN202000647,KJQN202100646)。
关键词
图像处理
图像协调化
深度学习
多尺度特征校准
Image processing
Image harmonization
Deep learning
Multi-scale Feature Calibration(MFC)
