摘要
近年来,跨模态研究吸引了越来越多学者的关注,尤其是连接视觉和语言的相关课题。该文针对跨视觉和语言模态研究中的核心任务——图像描述生成,进行文献综述。该文从基于视觉的文本生成框架、基于视觉的文本生成研究中的关键问题、图像描述生成模型的性能评价和图像描述生成模型的主要发展过程四个方面对相关文献进行介绍和总结。最后,该文给出了几个未来的重点研究方向,包括跨视觉和语言模态的特征对齐、自动化评价指标的设计以及多样化图像描述生成。
In recent years, increasing attention has been attracted to the research field related to cross-modality, especially vision and language. This survey focuses on the task of image captioning and summarizes literatures from four aspects, including the overall architecture, some key questions for cross-modality research, the evaluation of image captioning and the state-of-the-art approaches to image captioning. In conclusion, we suggest three directions for future research, i.e., cross-modality representation, automatic evaluation metrics and diverse text generation.
作者
魏忠钰
范智昊
王瑞泽
承怡菁
赵王榕
黄萱菁
WEI Zhongyu;FAN Zhihao;WANG Ruize;CHENG Yijing;ZHAO Wangrong;HUANG Xuanjing(School of Data Science,Fudan University,Shanghai 200433,China;Academy for Engineering and Technology,Fudan University,Shanghai 200433,China;School of Computer Science and Technology,Fudan University,Shanghai 200433,China)
出处
《中文信息学报》
CSCD
北大核心
2020年第7期19-29,共11页
Journal of Chinese Information Processing
基金
国家自然科学基金(71991471)
国家社会科学基金(20ZDA060)
上海市科学技术委员会(18DZ1201000,17JC1420200)
