摘要
为解决传统的遥感图像描述算法对图像多尺度信息利用不充分的问题,本文提出了结合多尺度特征增强与记忆引导Transformer的遥感图像描述生成算法(MFE-MGT).首先,利用预训练的视觉特征提取器提取图像特征,并将卷积神经网络中浅层与深层的特征进行拼接;其次,通过多尺度特征增强模块获得融合增强后的图像特征,以更好地捕捉多尺度特征;接着,将融合增强后的视觉特征输入记忆引导Transformer的编码器进行编码聚合;最后,通过Transformer记忆解码器生成图像描述.模型采用RSICD数据集进行训练,实验结果表明,MFE-MGT在多个评价指标上的表现均优于当前主流的遥感图像描述生成算法,能够准确的描述图像内容.
To address the issues of forgetting and underutilization of multi-scale information in traditional remote sensing image captioning algorithms,this paper proposes a remote sensing image description generation algorithm combining Multi-Scale Feature Enhancement with Memory-guided Transformer(MFE-MGT).Firstly,visual features are extracted using a pre-trained visual feature extractor,and features from shallow and deep layers of convolutional neural networks are concatenated.Secondly,a multi-scale feature enhancement module is employed to obtain fused and enhanced image features,better capturing multi-scale characteristics.Next,the fused enhanced visual features are fed into the encoder of a Memory-guided Transformer for encoding aggregation.Finally,image descriptions are generated through a Transformer memory decoder.The model is trained on the RSICD dataset,and experimental results demonstrate that MFE-MGT outperforms current mainstream remote sensing image description generation algorithms across multiple evaluation metrics,accurately describing image content.
作者
姚志远
桑国明
张益嘉
YAO Zhiyuan;SANG Guoming;ZHANG Yijia(College of Information Science and Technology,Dalian Maritime University,Dalian 116026,China)
出处
《小型微型计算机系统》
北大核心
2025年第8期1978-1985,共8页
Journal of Chinese Computer Systems
基金
国家自然科学基金项目(62072070)资助。
