摘要
基于草图的图像检索以手绘草图为输入检索相应的自然图像,让使用者即使没有精准的相似自然图像也可以自行绘制后检索。边缘图通常作为中间模态引入,以减小草图和自然图像之间的域差异,但现有方法忽视了边缘图与自然图像之间的内在联系。基于自然图像及其边缘图应有相近重点区域的设想,提出了基于跨域联合空间注意网络的深度学习模型。该模型从边缘图与自然图像的融合特征中获取两者共用的空间注意掩模,结合损失函数和辅助分类器进行端到端的训练。与现有代表性的基于草图的图像检索方法相比,所提方法能有效地提取草图和自然图像的特征,在Sketchy和TU-Berlin数据集上的平均精度均值(mAP)分别达0.933和0.799,优于大部分代表性方法。
Sketch-based image retrieval uses hand-drawn sketches as input to retrieve corresponding natural images,allowing users to draw and find desired natural images when no accurate query images are available. Edge maps are commonly used as an intermediate modality to bridge the domain gap between sketches and natural images. However,existing methods ignore the inherent relationship between edge maps and natural images. Based on the assumption that natural images and their corresponding edge maps have similar key regions, this paper proposes a deep learning model based on a cross-domain spatial co-attention network. The proposed model derives the shared spatial attention mask from the fused feature of the edge map and natural image, and it combines the loss function and auxiliary classifier for end-to-end training. When compared with existing representative sketch-based image retrieval methods, the proposed method can effectively extract the features of sketches and natural images, with mean average precision(mAP) values of 0. 933 and0. 799 on the Sketchy and TU-Berlin datasets, respectively, outperforming most representative methods.
作者
于凌志
张熙凡
Yu Lingzhi;Zhang Xifan(School of Electrical and Information Engineering,Tianjin University,Tianjin 300072,China)
出处
《激光与光电子学进展》
CSCD
北大核心
2022年第22期272-277,共6页
Laser & Optoelectronics Progress
关键词
深度学习
图像检索
注意力机制
跨域检索
deep learning
image retrieval
attention mechanism
cross-domain retrieval
