Multiview Graph Restricted Boltzmann Machines
Abstract : Recently, the restricted Boltzmann machine (RBM) has aroused considerable interest in the multiview learning field. Although effectiveness is observed, like many existing multiview learning models, multiview RBM ignores the local manifold structure of multiview data. In this article, we first propose a novel graph RBM model, which preserves the data manifold structure and is amenable to Gibbs sampling. Then, we develop a multiview graph RBM model on the basis of the graph RBM, which performs local structural learning and multiview representation learning simultaneously. The proposed multiview model has the following merits: 1) it preserves the data manifold structure for multiview classification and 2) it performs view-consistent representation learning and view-specific representation learning simultaneously. The experimental results show that the proposed multiview model outperforms other state-of-the-art multiview classification algorithms.
? Graphs are built using multiple methods from multiple corpora and existing lexical databases such as WordNet.
? Comprehensive experiments of our proposed model in document modeling with a variety of evaluations: graph construction methods from in-domain, out-domain, and existing databases.
? To exploit prior structures, we use the CPAN-WordNet:Similarity package to query the similarity measures for every pair of words.
? As the analytical solution of the gradient cannot be computed, the existing training algorithms are mainly based on the sampling method.
? The entire RBM network is a bipartite graph, in which connections only exist between the visible and hidden units.
? We aim to demonstrate that these structural constraints will improve the modeling qualities by preventing overfitting and enhancing factor coherence – the issues largely ignored in RBMs.
? However, the posterior computation is extremely hard due to the so-called “double intractability” problem in undirected graphical models – intractability of both the data likelihood and the posterior.
? To prevent this problem, we drop all edges associated with the nodes that have more than 20 outlinks.
? Our purpose is to automatically learn and understand the research topics, issues and the evolution of ASD research literature.
• We demonstrate that the proposed technique improves the group coherence, facilitates visualization, provides means for estimation of intrinsic dimensionality, reduces overfitting, and possibly leads to better classification accuracy.
• Focusing on document modeling, we perform extensive experiments to evaluate the efficacy of the proposed framework.
• We propose a principled way to introduce domain structures into RBMs, focusing on an important class of constraints that arise from a network of features - pairwise correlation, pairwise smoothness, and groupwise smoothness.
• To assess factor coherence, we adopted the measure proposed, which has good agreement with human’s judgment.
? In order to test the performance of the algorithms, the proposed algorithms are compared with state-of-the-art classification algorithms, the multi-view Gaussian process with posterior consistency (MvGP) and consensus and complementarity based maximum entropy discrimination (MED-2C).
? There are many efficient multi-view algorithms for classification, such as multi-view Gaussian process with posterior Consistency (MvGP) [10] and consensus and complementarity based maximum entropy discrimination (MED-2C).
? In the PCRBM, the negative of the distance between two conditional probabilities is used to measure the consistency between two conditional probabilities.
|