摘要
监督方法在跨模态检索中已有不少成果,是比较热门的方法。然而,这类方法过于依赖标记的数据,没有充分利用无标签数据所包含的丰富信息。为了解决这一问题,人们开始研究无监督方法,但是仅依靠未标记数据的效果并不理想。对此,提出了基于GAN的语义对齐网络半监督跨模态哈希方法(GAN-SASCH)。该模型基于生成对抗网络,结合了语义对齐的概念。生成对抗网络分为两个模块,分别是生成器和判别器,生成器学习拟合未标记数据的相关性分布并生成虚假的数据样本,判别器则用于判断数据对样本是来自数据集还是生成器。通过这两个模块之间展开极大极小的对抗博弈游戏,不断提升生成对抗网络的性能。语义对齐能充分利用不同模态之间的相互作用和对称性,统一不同模态的相似性信息,有效地指导哈希代码的学习过程。除此之外,还引入了自适应学习优化参数以提升模型性能。在NUS-WIDE和MIRFLICKR25K数据集上,对比了所提方法与9种相关前沿方法,使用MAP与PR图两种评价指标验证了所提方法的有效性。
Supervised methods have achieved a lot of results in cross-modal retrieval and have become popular methods.However,these methods rely too much on labeled data and do not make full use of the rich information contained in unlabeled data.To solve this problem,unsupervised methods have been studied,but when relying solely on unlabeled data,the results are not ideal.Therefore,this paper proposes a semi-supervised cross-modal hashing method for semantic alignment networks based on GAN(GAN-SASCH).This model is based on generative adversarial networks that incorporate the concept of semantic alignment.The generative adversarial network is divided into two modules.The generator learns to fit the correlation distribution of the unlabeled data and generates a spurious data sample,and the discriminator is used to determine whether the data pair sample comes from the dataset or the generator.By developing a very small adversarial game between these two modules,the performance of the generative adversarial network is continuously improved.Semantic alignment can make full use of the interaction and symmetry between different modalities,unify the similarity information of different modalities,and effectively guide the learning process of hash code.In this paper,adaptive learning optimization parameters are also introduced to improve the performance of the model.On NUS-WIDE and MIRFLICKR25K datasets,we compare the proposed method with 9 related frontier methods,and verify the effectiveness of the proposed method by using two evaluation indicators,MAP and PR map.
作者
刘华咏
朱婷
LIU Huayong;ZHU Ting(School of Computer Science,Central China Normal University,Wuhan 430079,China)
出处
《计算机科学》
北大核心
2025年第6期159-166,共8页
Computer Science
基金
教育部人文社会科学研究项目(21YJA870005)。
关键词
跨模态哈希
生成对抗网络
语义对齐
半监督
自适应学习
Cross-modal hash
Generative adversarial network
Semantic alignment
Semi-supervised
Adaptive learning
