In this paper we present a novel automatic background substitution approach for live video. The objective of background substitution is to extract the foreground from the input video and then combine it with a new bac...In this paper we present a novel automatic background substitution approach for live video. The objective of background substitution is to extract the foreground from the input video and then combine it with a new background. In this paper, we use a color line model to improve the Gaussian mixture model in the background cut method to obtain a binary foreground segmentation result that is less sensitive to brightness differences. Based on the high quality binary segmentation results, we can automatically create a reliable trimap for alpha matting to refine the segmentation boundary. To make the composition result more realistic, an automatic foreground color adjustment step is added to make the foreground look consistent with the new background. Compared to previous approaches, our method can produce higher quality binary segmentation results, and to the best of our knowledge, this is the first time such an automatic and integrated background substitution system has been proposed which can run in real time, which makes it practical for everyday applications.展开更多
This paper presents a multi-task gradual inference model,MTGINet,for automatic portrait matting.It handles the subtasks of automatic portrait matting,namely portrait–transition–background trimap segmentation and tra...This paper presents a multi-task gradual inference model,MTGINet,for automatic portrait matting.It handles the subtasks of automatic portrait matting,namely portrait–transition–background trimap segmentation and transition region matting,with a single encoder–decoder structure.First,we enrich the highest stage of features from the encoder with portrait shape context via a shape context aggregation(SCA)module for trimap segmentation.Then,we fuse the SCA-enhanced features with detailed clues from the encoder for transition-region-aware alpha matting.The gradual inference model naturally allows sufficient interaction between the subtasks via forward computation and backwards propagation during training,and therefore achieves high accuracy while maintaining low complexity.In addition,considering the discrepancies in feature requirements across subtasks,we adapt the features from the encoders before reusing them via a feature rectification module.In addition to the MTGINet model,we have constructed a new large-scale dataset,HPM-17K,for half-body portrait matting.It consists of 16,967 images with diverse backgrounds.Comparative experiments with existing deep models on the public P3M-10K dataset and our HPM-17K dataset demonstrate that the proposed model exhibits state-of-the-art performance.展开更多
基金supported by the National HighTech R&D Program of China (Project No. 2012AA011903)the National Natural Science Foundation of China (Project No. 61373069)+1 种基金the Research Grant of Beijing Higher Institution Engineering Research CenterTsinghua–Tencent Joint Laboratory for Internet Innovation Technology
文摘In this paper we present a novel automatic background substitution approach for live video. The objective of background substitution is to extract the foreground from the input video and then combine it with a new background. In this paper, we use a color line model to improve the Gaussian mixture model in the background cut method to obtain a binary foreground segmentation result that is less sensitive to brightness differences. Based on the high quality binary segmentation results, we can automatically create a reliable trimap for alpha matting to refine the segmentation boundary. To make the composition result more realistic, an automatic foreground color adjustment step is added to make the foreground look consistent with the new background. Compared to previous approaches, our method can produce higher quality binary segmentation results, and to the best of our knowledge, this is the first time such an automatic and integrated background substitution system has been proposed which can run in real time, which makes it practical for everyday applications.
基金supported by the National Natural Science Foundation of China(Nos.62176010 and 61771026).
文摘This paper presents a multi-task gradual inference model,MTGINet,for automatic portrait matting.It handles the subtasks of automatic portrait matting,namely portrait–transition–background trimap segmentation and transition region matting,with a single encoder–decoder structure.First,we enrich the highest stage of features from the encoder with portrait shape context via a shape context aggregation(SCA)module for trimap segmentation.Then,we fuse the SCA-enhanced features with detailed clues from the encoder for transition-region-aware alpha matting.The gradual inference model naturally allows sufficient interaction between the subtasks via forward computation and backwards propagation during training,and therefore achieves high accuracy while maintaining low complexity.In addition,considering the discrepancies in feature requirements across subtasks,we adapt the features from the encoders before reusing them via a feature rectification module.In addition to the MTGINet model,we have constructed a new large-scale dataset,HPM-17K,for half-body portrait matting.It consists of 16,967 images with diverse backgrounds.Comparative experiments with existing deep models on the public P3M-10K dataset and our HPM-17K dataset demonstrate that the proposed model exhibits state-of-the-art performance.
