UNSUPERVISED DEEP BACKGROUND MATTING USING DEEP MATTE PRIOR
Abstract : Background matting is a recently developed image matting approach, with applications to image and video editing. It refers to estimating both the alpha matte and foreground from a pair of images with and without foreground objects. Recent work has applied deep learning to background matting, with very promising performance achieved. However, existing deep models are supervised which require a large dataset with ground truth alpha mattes for training. To avoid the cost of data collection and possible bias in training data, this paper proposes a datasetfree unsupervised deep learning-based approach for background mattingObserving that the local smoothness of alpha matte can be well characterized by the untrained network prior called deep matte prior, we model the foreground and alpha matte using the priors encoded by two generative convolutional neural networks. To avoid possible overfitting during unsupervised learning, a twostage learning scheme is developed which contains projectionbased training and Bayesian post refinement. An alpha-mattedriven initialization scheme is also developed for performance boost.. Even without calling external training data, the proposed approach provides competitive performance to recent supervised learning-based methods in the experiments.
Motivated by the cost and possible bias introduced by the prerequisite on training datasets of a supervised deep learningbased method, this paper aims at developing an unsupervised deep learning-based approach for background matting which does not require any external training sample. In other words, no ground truth alpha matte will be called for training, and the proposed approach only takes an image pair ( ¯I, B¯) as the input and directly learns to estimate the corresponding F and a. Such a training-data-free approach has its great benefits in practice. In addition, it can also be used for generating foreground objects and alpha mattes that are close to the ground truths for boosting the supervised training.
Notice that for each image pixel, there are three unknowns in the matting problem (1). Therefore, it is a highly ill-posed problem, and the main concern in image matting is about how to resolve the solution ambiguity. Most existing methods introduce additional external information (e.g. a trimap input by the user) or constraints (e.g. a green-screen environment) for reducing the solution ambiguity. The trimap-based methods require manual annotations from users, which can be laborintensive in the batch processing of many images. The methods assuming a green-screen environment has quite limited applicability, as such an assumption does not hold true for many scenarios, especially in urban areas
? However, existing deep models are supervised which require a large dataset with ground truth alpha mattes for training. To avoid the cost of data collection and possible bias in training data, this paper proposes a datasetfree unsupervised deep learning-based approach for background matting. Observing that the local smoothness of alpha matte can be well characterized by the untrained network prior called deep matte prior, we model the foreground and alpha matte using the priors encoded by two generative convolutional neural networks. To avoid possible overfitting during unsupervised learning, a twostage learning scheme is developed which contains projectionbased training and Bayesian post refinement.
? Based on the deep image prior and deep matte prior, we adopt two generative CNNs to model the foreground and alpha matte respectively, and the predictions from these two CNNs will then be used to reconstruct the background given by the input image pair. While the deep image/matte priors partially addressed the possible overfitting arising from the solution ambiguity, a straightforward training by the standard procedure will still suffer from the overfitting to undesired solutions,. In addition, it can also be used for generating foreground objects and alpha mattes that are close to the ground truths for boosting the supervised training.
|