A non-photorealistic rendering technique is a method to show various effects different from those of realistic image generation.Of the various techniques,flow-based image abstraction displays the shape and color featu...A non-photorealistic rendering technique is a method to show various effects different from those of realistic image generation.Of the various techniques,flow-based image abstraction displays the shape and color features well and performs a stylistic visual abstraction.But real-time rendering is impossible when CPU is used because it applies various filtering and iteration methods.In this paper,we present real-time processing methods of video abstraction using open open computing language(OpenCL),technique of general-purpose computing on graphics processing units(GPGPU).Through the acceleration of general-purpose computing(GPU),16 frame-per-second(FPS)or greater is shown to process video abstraction.展开更多
The exponential growth of video content has driven significant advancements in video summarization techniques in recent years.Breakthroughs in deep learning have been particularly transformative,enabling more effectiv...The exponential growth of video content has driven significant advancements in video summarization techniques in recent years.Breakthroughs in deep learning have been particularly transformative,enabling more effective detection of key information and creating new possibilities for video synopsis.To summarize recent progress and accelerate research in this field,this paper provides a comprehensive review of deep learning-based video summarization methods developed over the past decade.We begin by examining the research landscape of video abstraction technologies and identifying core challenges in video summarization.Subsequently,we systematically analyze prevailing deep learning frameworks and methodologies employed in current video summarization systems,offering researchers a clear roadmap of the field's evelution.Unlike previous review works,we first classify research papers based on the structural hierarchy of the video(from frame-level to shot-level to video-level),then further categorize them according to the summary backbone model(feature extraction and spatiotemporal modeling).This approach provides a more systematic and hierarchical organization of the documents.Following this comprehensive review,we summarize the benchmark datasets and evaluation metrics commonly employed in the field.Finally,we analyze persistent challenges and propose insightful directions for future research,providing a forward-looking perspective on video summarization technologies.This systematic literature review is of great reference value to new researchers exploring the fields of deep learning and video summarization.展开更多
针对现有视频摘要算法以及摘要评价方法未能充分考虑工业智能终端所感知的视频数据特点以及工业智能感知相关应用需求,改写了代表性与多样性两种评价约束,基于此,结合DWConv(Depthwise Convolution)与ConvLSTM(Convolutional Long Short...针对现有视频摘要算法以及摘要评价方法未能充分考虑工业智能终端所感知的视频数据特点以及工业智能感知相关应用需求,改写了代表性与多样性两种评价约束,基于此,结合DWConv(Depthwise Convolution)与ConvLSTM(Convolutional Long Short-Term Memory)设计了一种混合双向多层的工业视频摘要方案。该方案由全局粗粒度特征提取、局部细粒度特征提取、反馈更新以及以查询为驱动的特征融合这4部分组成。为应对工业数据高冗余性、感知的视频噪声大等特点,围绕着ConvLSTM与注意力机制搭建全局特征提取模块;为充分提取视频数据的时空特性,结合注意力机制与DB-DWConvLSTM构建局部特征提取模块;针对工业数据具有的周期性与局部稳定性,借助残差网络思想,设计了融合DWConv反馈模块;为了更加凸显关键帧特征,便于更好的筛选关键帧,研究以查询驱动的特征融合模块。为验证方案的有效性与可行性,将该方案在TVSum与SumMe两个数据集上进行分析验证。实验结果表明:该方案在交叉验证、消融实验以及对比分析中都有着较好的性能。展开更多
文摘A non-photorealistic rendering technique is a method to show various effects different from those of realistic image generation.Of the various techniques,flow-based image abstraction displays the shape and color features well and performs a stylistic visual abstraction.But real-time rendering is impossible when CPU is used because it applies various filtering and iteration methods.In this paper,we present real-time processing methods of video abstraction using open open computing language(OpenCL),technique of general-purpose computing on graphics processing units(GPGPU).Through the acceleration of general-purpose computing(GPU),16 frame-per-second(FPS)or greater is shown to process video abstraction.
基金supported by UKRI(EP/Z000025/1)Horizon Europe Programme under the MSCA grant for the ACMod project(101130271)。
文摘The exponential growth of video content has driven significant advancements in video summarization techniques in recent years.Breakthroughs in deep learning have been particularly transformative,enabling more effective detection of key information and creating new possibilities for video synopsis.To summarize recent progress and accelerate research in this field,this paper provides a comprehensive review of deep learning-based video summarization methods developed over the past decade.We begin by examining the research landscape of video abstraction technologies and identifying core challenges in video summarization.Subsequently,we systematically analyze prevailing deep learning frameworks and methodologies employed in current video summarization systems,offering researchers a clear roadmap of the field's evelution.Unlike previous review works,we first classify research papers based on the structural hierarchy of the video(from frame-level to shot-level to video-level),then further categorize them according to the summary backbone model(feature extraction and spatiotemporal modeling).This approach provides a more systematic and hierarchical organization of the documents.Following this comprehensive review,we summarize the benchmark datasets and evaluation metrics commonly employed in the field.Finally,we analyze persistent challenges and propose insightful directions for future research,providing a forward-looking perspective on video summarization technologies.This systematic literature review is of great reference value to new researchers exploring the fields of deep learning and video summarization.
文摘针对现有视频摘要算法以及摘要评价方法未能充分考虑工业智能终端所感知的视频数据特点以及工业智能感知相关应用需求,改写了代表性与多样性两种评价约束,基于此,结合DWConv(Depthwise Convolution)与ConvLSTM(Convolutional Long Short-Term Memory)设计了一种混合双向多层的工业视频摘要方案。该方案由全局粗粒度特征提取、局部细粒度特征提取、反馈更新以及以查询为驱动的特征融合这4部分组成。为应对工业数据高冗余性、感知的视频噪声大等特点,围绕着ConvLSTM与注意力机制搭建全局特征提取模块;为充分提取视频数据的时空特性,结合注意力机制与DB-DWConvLSTM构建局部特征提取模块;针对工业数据具有的周期性与局部稳定性,借助残差网络思想,设计了融合DWConv反馈模块;为了更加凸显关键帧特征,便于更好的筛选关键帧,研究以查询驱动的特征融合模块。为验证方案的有效性与可行性,将该方案在TVSum与SumMe两个数据集上进行分析验证。实验结果表明:该方案在交叉验证、消融实验以及对比分析中都有着较好的性能。
