Current mainstream unsupervised video object segmentation(UVOS) approaches typically incorporate optical flow as motion information to locate the primary objects in coherent video frames. However, they fuse appearance...Current mainstream unsupervised video object segmentation(UVOS) approaches typically incorporate optical flow as motion information to locate the primary objects in coherent video frames. However, they fuse appearance and motion information without evaluating the quality of the optical flow. When poor-quality optical flow is used for the interaction with the appearance information, it introduces significant noise and leads to a decline in overall performance. To alleviate this issue, we first employ a quality evaluation module(QEM) to evaluate the optical flow. Then, we select high-quality optical flow as motion cues to fuse with the appearance information, which can prevent poor-quality optical flow from diverting the network's attention. Moreover, we design an appearance-guided fusion module(AGFM) to better integrate appearance and motion information. Extensive experiments on several widely utilized datasets, including DAVIS-16, FBMS-59, and You Tube-Objects, demonstrate that the proposed method outperforms existing methods.展开更多
This paper presented an object-based fast motion estimation (ME) algorithm for object-based texture coding in moving picture experts group four (MPEG-4), which takes full advantage of the shape information of video ob...This paper presented an object-based fast motion estimation (ME) algorithm for object-based texture coding in moving picture experts group four (MPEG-4), which takes full advantage of the shape information of video object. Compared with the full search (FS) algorithm, the proposed algorithm can significantly speed the ME process. The speed of ME using the proposed algorithm is faster than that using new three-step search (NTSS), four-step search (4SS), diamond search (DS), and block-based gradient descent search (BBGDS) algorithms with similar motion compensation (MC) errors. The proposed algorithm can be combined with other fast ME algorithm to make the ME process faster.展开更多
A novel moving objects segmentation method is proposed in this paper. A modified three dimensional recursive search （3DRS） algorithm is used in order to obtain motion information accurately. A motion feature descrip...A novel moving objects segmentation method is proposed in this paper. A modified three dimensional recursive search （3DRS） algorithm is used in order to obtain motion information accurately. A motion feature descriptor （MFD） is designed to describe motion feature of each block in a picture based on motion intensity, motion in occlusion areas, and motion correlation among neighbouring blocks. Then, a fuzzy C-means clustering algorithm （FCM） is implemented based on those MFDs so as to segment moving objects. Moreover, a new parameter named as gathering degree is used to distinguish foreground moving objects and background motion. Experimental results demonstrate the effectiveness of the proposed method.展开更多
This paper proposes a motion-based region growing segmentation scheme for the object-based video coding, which segments an image into homogeneous regions characterized by a coherent motion. It adopts a block matching ...This paper proposes a motion-based region growing segmentation scheme for the object-based video coding, which segments an image into homogeneous regions characterized by a coherent motion. It adopts a block matching algorithm to estimate motion vectors and uses morphological tools such as open-close by reconstruction and the region-growing version of the watershed algorithm for spatial segmentation to improve the temporal segmentation. In order to determine the reliable motion vectors, this paper also proposes a change detection algorithm and a multi-candidate pro- screening motion estimation method. Preliminary simulation results demonstrate that the proposed scheme is feasible. The main advantage of the scheme is its low computational load.展开更多
现有无监督视频目标分割(Unsupervised Video Object Segmentation,UVOS)方法多采用像素级密集匹配策略,通过对齐融合多帧之间或单帧与光流之间的信息来提升模型性能.然而,在遮挡、相机抖动、运动模糊等挑战性场景中,光流估计误差易产...现有无监督视频目标分割(Unsupervised Video Object Segmentation,UVOS)方法多采用像素级密集匹配策略,通过对齐融合多帧之间或单帧与光流之间的信息来提升模型性能.然而,在遮挡、相机抖动、运动模糊等挑战性场景中,光流估计误差易产生大量错误匹配,导致融合后的时空表征易过拟合运动噪声.为此,本文提出一种运动提示引导的自适应学习UVOS框架.通过设计一种无监督光流提示生成算法,将光流编码的密集运动信息转换为稀疏点和框提示,借助提示学习引导分割一切模型(Segment Anything Model,SAM)通过本文设计的两个轻量级适配器来自适应学习,从而获得更为鲁棒的时空表征,增强模型的抗噪能力.为获得有效的提示,设计了一种无监督运动提示生成算法.该算法基于光流特征计算一系列统计量,筛选出显著区域,再利用运动边缘信息去除伪显著区域的干扰,并设定自适应阈值进行过滤,生成提示显著运动目标所在区域的点和框坐标.为提升SAM在下游UVOS任务中的泛化性,提出一种自适应表征学习SAM模型.通过设计两个轻量级特征适配器,从SAM的通用知识库中自适应学习与下游UVOS任务相关的知识,以准确地粗定位目标.针对SAM基于纯Transformer架构在细节处理上的不足,基于卷积神经网络(Convolutional Neural Networks,CNN)架构设计了表观聚焦细化模块.由SAM得到的定位注意力图渐进式地引导细化过程,使模型的注意力从全局粗定位聚焦到局部细化,最终得到更加精确的分割掩码.本文方法在DAVIS16(DAVIS 2016)、FBMS(Financial and Business Management System)和YTOBJ(YouTube-OBJects)三个主流数据集上进行了充分验证.结果表明:本文方法在区域相似度指标上较当前先进方法分别提升了1.8%、1.6%和2.6%,充分表明了本文方法的有效性.展开更多
The past two decades witnessed a broad-increase in web technology and on-line gaming.Enhancing the broadband confinements is viewed as one of the most significant variables that prompted new gaming technology.The imme...The past two decades witnessed a broad-increase in web technology and on-line gaming.Enhancing the broadband confinements is viewed as one of the most significant variables that prompted new gaming technology.The immense utilization of web applications and games additionally prompted growth in the handled devices and moving the limited gaming experience from user devices to online cloud servers.As internet capabilities are enhanced new ways of gaming are being used to improve the gaming experience.In cloud-based video gaming,game engines are hosted in cloud gaming data centers,and compressed gaming scenes are rendered to the players over the internet with updated controls.In such systems,the task of transferring games and video compression imposes huge computational complexity is required on cloud servers.The basic problems in cloud gaming in particular are high encoding time,latency,and low frame rates which require a new methodology for a better solution.To improve the bandwidth issue in cloud games,the compression of video sequences requires an alternative mechanism to improve gaming adaption without input delay.In this paper,the proposed improved methodology is used for automatic unnecessary scene detection,scene removing and bit rate reduction using an adaptive algorithm for object detection in a game scene.As a result,simulations showed without much impact on the players’quality experience,the selective object encoding method and object adaption technique decrease the network latency issue,reduce the game streaming bitrate at a remarkable scale on different games.The proposed algorithm was evaluated for three video game scenes.In this paper,achieved 14.6%decrease in encoding and 45.6%decrease in bit rate for the first video game scene.展开更多
This paper proposes a motion-based region growing segmentation scheme, which incorporatesluminance and motion information simultaneously and uses morphological tools such as open-close byreconstruction and the region-...This paper proposes a motion-based region growing segmentation scheme, which incorporatesluminance and motion information simultaneously and uses morphological tools such as open-close byreconstruction and the region-growing version of the watershed algorithm. The main advantage of this scheme is thatthe resultant objects ore characterized by a coherent motion and foe moving object boundaries are precisely located.Simulation results demonstrate the effiency of the Proposed scheme.展开更多
基金supported by the National Natural Science Foundation of China (No.61872189)。
文摘Current mainstream unsupervised video object segmentation(UVOS) approaches typically incorporate optical flow as motion information to locate the primary objects in coherent video frames. However, they fuse appearance and motion information without evaluating the quality of the optical flow. When poor-quality optical flow is used for the interaction with the appearance information, it introduces significant noise and leads to a decline in overall performance. To alleviate this issue, we first employ a quality evaluation module(QEM) to evaluate the optical flow. Then, we select high-quality optical flow as motion cues to fuse with the appearance information, which can prevent poor-quality optical flow from diverting the network's attention. Moreover, we design an appearance-guided fusion module(AGFM) to better integrate appearance and motion information. Extensive experiments on several widely utilized datasets, including DAVIS-16, FBMS-59, and You Tube-Objects, demonstrate that the proposed method outperforms existing methods.
基金National High Technology Research and De-velopment Program of China (863 Program)(No.2003AA103810)
文摘This paper presented an object-based fast motion estimation (ME) algorithm for object-based texture coding in moving picture experts group four (MPEG-4), which takes full advantage of the shape information of video object. Compared with the full search (FS) algorithm, the proposed algorithm can significantly speed the ME process. The speed of ME using the proposed algorithm is faster than that using new three-step search (NTSS), four-step search (4SS), diamond search (DS), and block-based gradient descent search (BBGDS) algorithms with similar motion compensation (MC) errors. The proposed algorithm can be combined with other fast ME algorithm to make the ME process faster.
基金Supported by the National Natural Science Foundation of China （No. 60772134, 60902081, 60902052） the 111 Project （No.B08038） the Fundamental Research Funds for the Central Universities（No.72105457）.
文摘A novel moving objects segmentation method is proposed in this paper. A modified three dimensional recursive search （3DRS） algorithm is used in order to obtain motion information accurately. A motion feature descriptor （MFD） is designed to describe motion feature of each block in a picture based on motion intensity, motion in occlusion areas, and motion correlation among neighbouring blocks. Then, a fuzzy C-means clustering algorithm （FCM） is implemented based on those MFDs so as to segment moving objects. Moreover, a new parameter named as gathering degree is used to distinguish foreground moving objects and background motion. Experimental results demonstrate the effectiveness of the proposed method.
文摘This paper proposes a motion-based region growing segmentation scheme for the object-based video coding, which segments an image into homogeneous regions characterized by a coherent motion. It adopts a block matching algorithm to estimate motion vectors and uses morphological tools such as open-close by reconstruction and the region-growing version of the watershed algorithm for spatial segmentation to improve the temporal segmentation. In order to determine the reliable motion vectors, this paper also proposes a change detection algorithm and a multi-candidate pro- screening motion estimation method. Preliminary simulation results demonstrate that the proposed scheme is feasible. The main advantage of the scheme is its low computational load.
文摘现有无监督视频目标分割(Unsupervised Video Object Segmentation,UVOS)方法多采用像素级密集匹配策略,通过对齐融合多帧之间或单帧与光流之间的信息来提升模型性能.然而,在遮挡、相机抖动、运动模糊等挑战性场景中,光流估计误差易产生大量错误匹配,导致融合后的时空表征易过拟合运动噪声.为此,本文提出一种运动提示引导的自适应学习UVOS框架.通过设计一种无监督光流提示生成算法,将光流编码的密集运动信息转换为稀疏点和框提示,借助提示学习引导分割一切模型(Segment Anything Model,SAM)通过本文设计的两个轻量级适配器来自适应学习,从而获得更为鲁棒的时空表征,增强模型的抗噪能力.为获得有效的提示,设计了一种无监督运动提示生成算法.该算法基于光流特征计算一系列统计量,筛选出显著区域,再利用运动边缘信息去除伪显著区域的干扰,并设定自适应阈值进行过滤,生成提示显著运动目标所在区域的点和框坐标.为提升SAM在下游UVOS任务中的泛化性,提出一种自适应表征学习SAM模型.通过设计两个轻量级特征适配器,从SAM的通用知识库中自适应学习与下游UVOS任务相关的知识,以准确地粗定位目标.针对SAM基于纯Transformer架构在细节处理上的不足,基于卷积神经网络(Convolutional Neural Networks,CNN)架构设计了表观聚焦细化模块.由SAM得到的定位注意力图渐进式地引导细化过程,使模型的注意力从全局粗定位聚焦到局部细化,最终得到更加精确的分割掩码.本文方法在DAVIS16(DAVIS 2016)、FBMS(Financial and Business Management System)和YTOBJ(YouTube-OBJects)三个主流数据集上进行了充分验证.结果表明:本文方法在区域相似度指标上较当前先进方法分别提升了1.8%、1.6%和2.6%,充分表明了本文方法的有效性.
文摘The past two decades witnessed a broad-increase in web technology and on-line gaming.Enhancing the broadband confinements is viewed as one of the most significant variables that prompted new gaming technology.The immense utilization of web applications and games additionally prompted growth in the handled devices and moving the limited gaming experience from user devices to online cloud servers.As internet capabilities are enhanced new ways of gaming are being used to improve the gaming experience.In cloud-based video gaming,game engines are hosted in cloud gaming data centers,and compressed gaming scenes are rendered to the players over the internet with updated controls.In such systems,the task of transferring games and video compression imposes huge computational complexity is required on cloud servers.The basic problems in cloud gaming in particular are high encoding time,latency,and low frame rates which require a new methodology for a better solution.To improve the bandwidth issue in cloud games,the compression of video sequences requires an alternative mechanism to improve gaming adaption without input delay.In this paper,the proposed improved methodology is used for automatic unnecessary scene detection,scene removing and bit rate reduction using an adaptive algorithm for object detection in a game scene.As a result,simulations showed without much impact on the players’quality experience,the selective object encoding method and object adaption technique decrease the network latency issue,reduce the game streaming bitrate at a remarkable scale on different games.The proposed algorithm was evaluated for three video game scenes.In this paper,achieved 14.6%decrease in encoding and 45.6%decrease in bit rate for the first video game scene.
文摘This paper proposes a motion-based region growing segmentation scheme, which incorporatesluminance and motion information simultaneously and uses morphological tools such as open-close byreconstruction and the region-growing version of the watershed algorithm. The main advantage of this scheme is thatthe resultant objects ore characterized by a coherent motion and foe moving object boundaries are precisely located.Simulation results demonstrate the effiency of the Proposed scheme.
