Image captioning models typically operate with a fixed vocabulary,but captioning is an open-vocabulary problem.Existing work addresses the image captioning of out-of-vocabulary words by labeling it as unknown in a dic...Image captioning models typically operate with a fixed vocabulary,but captioning is an open-vocabulary problem.Existing work addresses the image captioning of out-of-vocabulary words by labeling it as unknown in a dictionary.In addition,recurrent neural network(RNN)and its variants used in the caption task have become a bottleneck for their generation quality and training time cost.To address these 2 essential problems,a simpler but more effective approach is proposed for generating open-vocabulary caption,long short-term memory(LSTM)unit is replaced with transformer as decoder for better caption quality and less training time.The effectiveness of different word segmentation vocabulary and generation improvement of transformer over LSTM is discussed and it is proved that the improved models achieve state-of-the-art performance for the MSCOCO2014 image captioning tasks over a back-off dictionary baseline model.展开更多
基金the National Natural Science Foundation of China(No.61877002)Beijing Municipal Commission of Education(PXM2019014213000007).
文摘Image captioning models typically operate with a fixed vocabulary,but captioning is an open-vocabulary problem.Existing work addresses the image captioning of out-of-vocabulary words by labeling it as unknown in a dictionary.In addition,recurrent neural network(RNN)and its variants used in the caption task have become a bottleneck for their generation quality and training time cost.To address these 2 essential problems,a simpler but more effective approach is proposed for generating open-vocabulary caption,long short-term memory(LSTM)unit is replaced with transformer as decoder for better caption quality and less training time.The effectiveness of different word segmentation vocabulary and generation improvement of transformer over LSTM is discussed and it is proved that the improved models achieve state-of-the-art performance for the MSCOCO2014 image captioning tasks over a back-off dictionary baseline model.
