The Joint Video Experts Team(JVET)has announced the latest generation of the Versatile Video Coding(VVC,H.266)standard.The in-loop filter in VVC inherits the De-Blocking Filter(DBF)and Sample Adaptive Offset(SAO)of Hi...The Joint Video Experts Team(JVET)has announced the latest generation of the Versatile Video Coding(VVC,H.266)standard.The in-loop filter in VVC inherits the De-Blocking Filter(DBF)and Sample Adaptive Offset(SAO)of High Efficiency Video Coding(HEVC,H.265),and adds the Adaptive Loop Filter(ALF)to minimize the error between the original sample and the decoded sample.However,for chaotic moving video encoding with low bitrates,serious blocking artifacts still remain after in-loop filtering due to the severe quantization distortion of texture details.To tackle this problem,this paper proposes a Convolutional Neural Network(CNN)based VVC in-loop filter for chaotic moving video encoding with low bitrates.First,a blur-aware attention network is designed to perceive the blurring effect and to restore texture details.Then,a deep in-loop filtering method is proposed based on the blur-aware network to replace the VVC in-loop filter.Finally,experimental results show that the proposed method could averagely save 8.3%of bit consumption at similar subjective quality.Meanwhile,under close bit rate consumption,the proposed method could reconstruct more texture information,thereby significantly reducing the blocking artifacts and improving the visual quality.展开更多
Versatile video coding(H.266/VVC),which was newly released by the Joint Video Exploration Team(JVET),introduces quad-tree plus multitype tree(QTMT)partition structure on the basis of quad-tree(QT)partition structure i...Versatile video coding(H.266/VVC),which was newly released by the Joint Video Exploration Team(JVET),introduces quad-tree plus multitype tree(QTMT)partition structure on the basis of quad-tree(QT)partition structure in High Efficiency Video Coding(H.265/HEVC).More complicated coding unit(CU)partitioning processes in H.266/VVC significantly improve video compression efficiency,but greatly increase the computational complexity compared.The ultra-high encoding complexity has obstructed its real-time applications.In order to solve this problem,a CU partition algorithm using convolutional neural network(CNN)is proposed in this paper to speed up the H.266/VVC CU partition process.Firstly,64×64 CU is divided into smooth texture CU,mildly complex texture CU and complex texture CU according to the CU texture characteristics.Second,CU texture complexity classification convolutional neural network(CUTCC-CNN)is proposed to classify CUs.Finally,according to the classification results,the encoder is guided to skip different RDO search process.And optimal CU partition results will be determined.Experimental results show that the proposed method reduces the average coding time by 32.2%with only 0.55%BD-BR loss compared with VTM 10.2.展开更多
基金supported by National Natural Science Foundation of China under grant U20A20157,61771082,62271096 and 61871062the General Program of Chonqing Natural Science Foundation under grant cstc2021jcyj-msxm X0032+2 种基金the Natural Science Foundation of Chongqing,China(cstc2020jcyj-zdxm X0024)the Science and Technology Research Program of Chongqing Municipal Education Commission under grant KJQN202300632the University Innovation Research Group of Chongqing(CXQT20017)。
文摘The Joint Video Experts Team(JVET)has announced the latest generation of the Versatile Video Coding(VVC,H.266)standard.The in-loop filter in VVC inherits the De-Blocking Filter(DBF)and Sample Adaptive Offset(SAO)of High Efficiency Video Coding(HEVC,H.265),and adds the Adaptive Loop Filter(ALF)to minimize the error between the original sample and the decoded sample.However,for chaotic moving video encoding with low bitrates,serious blocking artifacts still remain after in-loop filtering due to the severe quantization distortion of texture details.To tackle this problem,this paper proposes a Convolutional Neural Network(CNN)based VVC in-loop filter for chaotic moving video encoding with low bitrates.First,a blur-aware attention network is designed to perceive the blurring effect and to restore texture details.Then,a deep in-loop filtering method is proposed based on the blur-aware network to replace the VVC in-loop filter.Finally,experimental results show that the proposed method could averagely save 8.3%of bit consumption at similar subjective quality.Meanwhile,under close bit rate consumption,the proposed method could reconstruct more texture information,thereby significantly reducing the blocking artifacts and improving the visual quality.
文摘现有的基于卷积神经网络(convolutional neural network,CNN)的环路滤波器倾向于将多个网络应用于不同的量化参数(quantization parameter,QP),消耗训练模型中的大量资源,并增加内存负担。针对这一问题,提出一种基于CNN的QP自适应环路滤波器。首先,设计一个轻量级分类网络,按照滤波难易程度将编码树单元(coding tree unit,CTU)划分为难、中、易3类;然后,构建3个融合了特征信息增强融合模块的基于CNN的滤波网络,以满足不同QP下的3类CTU滤波需求。将所提出的环路滤波器集成到多功能视频编码(versatile video coding,VVC)标准H.266/VVC的测试软件VTM 6.0中,替换原有的去块效应滤波器(deblocking filter,DBF)、样本自适应偏移(sample adaptive offset,SAO)滤波器和自适应环路滤波器。实验结果表明,该方法平均降低了3.14%的比特率差值(Bjøntegaard delta bit rate,BD-BR),与其他基于CNN的环路滤波器相比,显著提高了压缩效率,并减少了压缩伪影。
基金This paper is supported by the following funds:The National Key Research and Development Program of China(2018YFF01010100)Basic Research Program of Qinghai Province under Grants No.2021-ZJ-704,The Beijing Natural Science Foundation(4212001)Advanced information network Beijing laboratory(PXM2019_014204_500029).
文摘Versatile video coding(H.266/VVC),which was newly released by the Joint Video Exploration Team(JVET),introduces quad-tree plus multitype tree(QTMT)partition structure on the basis of quad-tree(QT)partition structure in High Efficiency Video Coding(H.265/HEVC).More complicated coding unit(CU)partitioning processes in H.266/VVC significantly improve video compression efficiency,but greatly increase the computational complexity compared.The ultra-high encoding complexity has obstructed its real-time applications.In order to solve this problem,a CU partition algorithm using convolutional neural network(CNN)is proposed in this paper to speed up the H.266/VVC CU partition process.Firstly,64×64 CU is divided into smooth texture CU,mildly complex texture CU and complex texture CU according to the CU texture characteristics.Second,CU texture complexity classification convolutional neural network(CUTCC-CNN)is proposed to classify CUs.Finally,according to the classification results,the encoder is guided to skip different RDO search process.And optimal CU partition results will be determined.Experimental results show that the proposed method reduces the average coding time by 32.2%with only 0.55%BD-BR loss compared with VTM 10.2.
