Deep Ladder-Suppression Network for Unsupervised Domain Adaptation
Abstract : Unsupervised domain adaptation (UDA) aims at learning a classifier for an unlabeled target domain by transferring knowledge from a labeled source domain with a related but different distribution. Most existing approaches learn domain-invariant features by adapting the entire information of the images. However, forcing adaptation of domain-specific variations undermines the effectiveness of the learned features. To address this problem, we propose a novel, yet elegant module, called the deep ladder-suppression network (DLSN), which is designed to better learn the cross-domain shared content by suppressing domain-specific variations. Our proposed DLSN is an autoencoder with lateral connections from the encoder to the decoder. By this design, the domain-specific details, which are only necessary for reconstructing the unlabeled target data, are directly fed to the decoder to complete the reconstruction task, relieving the pressure of learning domain-specific variations at the later layers of the shared encoder. As a result, DLSN allows the shared encoder to focus on learning cross-domain shared content and ignores the domain-specific variations. Notably, the proposed DLSN can be used as a standard module to be integrated with various existing UDA frameworks to further boost performance. Without whistles and bells, extensive experimental results on four gold-standard domain adaptation datasets, for example: 1) Digits; 2) Office31; 3) Office-Home; and 4) VisDA-C, demonstrate that the proposed DLSN can consistently and significantly improve the performance of various popular UDA frameworks.
? Existing methods cannot fully solve the domain shift problem without data sharing. Collecting and labeling more data in the target domain is straightforward, but it is expensive and likely under scrutiny.
? However, existing works are inapplicable to face recognition under privacy constraints because they require sharing sensitive face images between two domains.
? Existing research on UDA for face recognition mainly leverages some of these methods: Sohn et al. learned domain-invariant features through domain adversarial discriminator.
? Although unsupervised domain adaptation methods reduce domain gaps effectively, they mostly assume data is shared between domains.
? Neural networks where used for classification or regression problems with only supervised data.
? Recurrent Neural Networks (RNN) are a kind of neural network models used for these problems.
? The very big problem of these approaches is the computation cost due to the computation of the second order term that increases quadratically with the number of parameters.
? Gradient vanishing is a problem that appears in very deep architectures in part due to the derivative of certain activation functions.
? These problems appear when lots of neurons activate as a zero because the zeros are propagated through the network and so it is the derivative.
• We evaluate the effectiveness of BiAT on three benchmark datasets and experimental results demonstrate the proposed method achieves the state-of-the-art.
• The MME method they proposed maximize the entropy of unlabeled target data to optimize classifier, and minimize the entropy with respect to the feature extractor to cluster features.
• We firstly propose Adaptive Adversarial Training (AAT), a novel adversarial training notion for specific SSDA scenarios that generates adaptive adversarial examples from the source to target domain.
• We propose a uniform Bidirectional Adversarial Training (BiAT) network to perform AT, AAT, and E-VAT jointly.
? The use of unsupervised data in addition to supervised data in training neural networks has improved the performance of this classification paradigm.
? We have already talked about how can we improve the performance of a neural network with techniques that affect directly the optimization algorithm and with techniques that improve generalization.
? The big contribution of semi-supervised learning is the ability of using the features that supervised learning finds suitable to refine which features of the input data, at different levels, are helpfull for the discriminant task performance.
? One of the keys that cannot let us conclude if adversarial noise improve the performance of the ladder network is the high variability of the hyperparemeters of this model.
|