摘要
对深度学习领域的研究进行综合评述,并对其进一步发展方向进行分析。首先分析围绕注意力机制的深度学习技术最新研究成果,以及在自然语言处理领域取得突破性进展的巨型预训练模型的特点与发展路径;随后概述开源深度学习市场的火热局面及其对技术升级的推动作用;最后分别从香农定律、冯·诺依曼架构、摩尔定律三个角度探讨深度学习技术的未来发展方向。综述表明,注意力机制和预训练范式在当前计算机视觉和自然语言处理等深度学习重点应用领域中取得长足技术突破,开源深度学习市场的兴起有效推动产学研用各领域深度学习技术落地,在今后很长一段时间里,深度学习依然具有很广阔的发展空间。
This paper makes a comprehensive survey of the research in the field of deep learning(DL),and analyzes its further development directions.This paper firstly overviews the latest research achievements of DL technology about attention mechanism,as well as the characteristics and development directions of giant pre-training model which has made a breakthrough in the domain of Natural Language Processing(NLP).Secondly,it summarizes the hot situation of open-source DL market and its role in promoting the corresponding technology upgrading.Finally,this paper discusses the future development directions of DL technology from three aspects:Shannon’s Law,Von Neumann’s architecture and Moore’s Law.The survey shows that,attention mechanism and pre-training paradigm have made great technological breakthroughs in the key application fields of deep learning,such as Computer Vision(CV)and NLP.Besides,the rise of open-source DL market has effectively promoted the implementation of DL technology in various fields of industry,research and applications.For a long time,DL still has a broad development space.
作者
王亚珅
WANG Yashen(National Engineering Laboratory for Risk Perception and Prevention(RPP),China Academy of Electronics and Information Technology of CETC,Beijing 100041,China)
出处
《无人系统技术》
2021年第2期1-7,共7页
Unmanned Systems Technology
基金
国家自然科学基金(U19B2026)
中国电科新一代人工智能专项行动计划项目(AI20191125008)
全国一体化国家大数据中心先导工程(17111002,20500908)。
关键词
深度学习
注意力机制
预训练
神经网络
开源
人工智能
Deep Learning
Attention Mechanism
Pre-Training
Neural Network
Open-Source
Artificial Intelligence
