摘要
近几年的研究表明视觉注意力机制是提升深层卷积神经网络性能的有效途径.然而,现有的视觉注意力方法更多地致力于建模所有卷积通道之间的相关性,在一定程度上限制了模型的计算效率.此外,这些方法尚未明确考虑相关性建模过程中不确定性带来的影响,缺少对注意力机制在泛化能力和稳定性方面的探索.为解决上述问题,提出了一种多分支局部通道注意力模块(Multi-Branch Local Channel Attention,MBLCA).通过建模通道之间的局部相关性学习各个通道的权重,提升了模型的计算效率.并采用蒙特卡洛(Monte Carlo,MC)Dropout近似的深度贝叶斯学习方法对局部通道注意力模块进行不确定性建模,从而得到一个多分支的局部通道注意力模块.提出的MBLCA模块可以灵活地应用于各种深层卷积神经网络架构中,与同类型的工作相比,嵌入MBLCA模块的ResNet-50网络结构在ImageNet-1K和MS COCO数据集上分别取得了2.58%的分类精度提升和1.9%的AP提升.
Recent researches demonstrate that attention mechanism is an effective way to improve performance of deep convolution neural networks(CNNs).However,most of existing attention methods more dedicate to modeling the correlation between all channels,which limits the computational efficiency of the model.In addition,these methods have not considered the impact of uncertainty in the correlation modeling process,and lack the exploration of the generalization ability and stability of the attention mechanism.A multi-branch local channel attention(MBLCA)module is proposed to handle above issues.MBLCA learns channel attention by capturing correlation across channels in a local range instead of global ones,improving the computational efficiency,and models the uncertainty of local channel attention by deep Bayesian learning,which is approximated by Monte Carlo(MC)Dropout,leading a multi-branch structure.The proposed MBLCA can be flexibly adopted to various deep CNN architectures.For example,ResNet-50with the MBLCA module has achieved2.58%improvement in classification accuracy and1.9%improvement in average precise on the ImageNet-1K and MS COCO datasets against state-of-the-art counterparts.
作者
伍邦谷
张苏林
石红
朱鹏飞
王旗龙
胡清华
WU Bang-gu;ZHANG Su-lin;SHI Hong;ZHU Peng-fei;WANG Qi-long;HU Qing-hua(College of Intelligence and Computing,Tianjin University,Tianjin 300350,China;Automotive Data of China Co.,Ltd.,Tianjin 300380,China)
出处
《电子学报》
EI
CAS
CSCD
北大核心
2022年第2期374-382,共9页
Acta Electronica Sinica
基金
国家自然科学基金(No.61806140,No.61925602,No.61876127,No.61732011)。
关键词
通道注意力机制
不确定性
多分支结构
深层卷积神经网络
channel attention mechanism
uncertainty
multi-branch structure
deep convolutional neural networks
