Cross-Lingual Knowledge Transferring by Structural Correspondence and Space Transfer
Abstract : The cross-lingual sentiment analysis (CLSA) aims to leverage label-rich resources in the source language to improve the models of a resource-scarce domain in the target language, where monolingual approaches based on machine learning usually suffer from the unavailability of sentiment knowledge. Recently, the transfer learning paradigm that can transfer sentiment knowledge from resource-rich languages, for example, English, to resource-poor languages, for example, Chinese, has gained particular interest. Along this line, in this article, we propose semisupervised learning with SCL and space transfer (ssSCL-ST), a semisupervised transfer learning approach that makes use of structural correspondence learning as well as space transfer for cross-lingual sentiment analysis. The key idea behind ssSCL-ST, at a high level, is to explore the intrinsic sentiment knowledge in the target-lingual domain and to reduce the loss of valuable knowledge due to the knowledge transfer via semisupervised learning. ssSCL-ST also features in pivot set extension and space transfer, which helps to enhance the efficiency of knowledge transfer and improve the classification accuracy in the target language domain. Extensive experimental results demonstrate the superiority of ssSCL-ST to the state-of-the-art approaches without using any parallel corpora.
? This demonstrates that roughly incorporating existing meta learning algorithms into the CLT problem may not work well.
? However, existing mPLM-based methods focus on designing costly model pre-training while ignoring equally crucial downstream adaptation.
? On the other hand, existing adaptation approaches for mPLM behave as a black box without explicitly identifying intrinsic language relations.
? Most existing meta-learners lack the ability to handle tasks lying in different distributions, especially tasks for heterogeneous languages.
? However, single-source CLT methods would incur the risk of negative transfer when there exists a large language shift.
? This creates a problem, as embeddings are not in the same vector space; they are trained from multiple disparate corpora.
? The problem with this approach is that the translations generated by Google Translate are not necessarily in the target language domain space.
? They initialized the optimization algorithm with a convex relaxation traditionally used for the graph-isomorphism or graph-matching problem.
? They investigated the impact of language similarity, among other factors, on BLI performance and reported poor performance for English–Finnish and English–Estonian pairs.
? Another observation from their results is the impact of batch size on performance.
• A novel MGL method is proposed to learn to cross-lingual transfer (L2CLT) for taskaware adaptation of mPLM by leveraging previous CLT experiences.
• The proposed MGL method can potentially applied to more cross-lingual natural language understanding (XLU) tasks and be generalized to learn to learn for domain adaptation multi-task learning problems, etc.
• To address the issues, we propose meta graph learning (MGL), a meta learning framework to learn how to cross-lingual transfer for mPLM.
• We propose a meta graph learning (MGL) method to further guide the versatile multilingual representations to be task-aware for downstream CLT tasks.
? The Transformer has demonstrated superior performance in modelling long-term dependencies in the text, compared to the RNN architecture.
? In a typical deep learning algorithm, a model is trained to learn patterns from training data to efficiently classify and predict unseen data.
? With NLP, transfer learning is anticipated to be a useful option in the development of efficient models, given the noisy, diversity, and unstructured characteristics of text data.
? FastText is an open-source library, designed by the Facebook research team for learning efficient word representations and classification of text/documents.
? It is evident that for the same corpus, vectors will manifest disparately each time a different vector training algorithm is used.
|