Zero-shot Cross-lingual Transfer Learning with Multiple Source and Target Languages for Information Extraction: Language Selection and
Nghia Trung Ngo
Committee: Thien Huu Nguyen (chair), Humphrey Shi, Thanh Hong Nguyen
Directed Research Project(Aug 2023)
Keywords: Information Extraction, Zero-shot, Cross-lingual, Multi-lingual, Transfer Learning, Unsupervised Domain Adaptation, Data Selection, Adversarial Learning, Linguistic Relations

Previous works on multi-lingual Information Extraction (IE) are restricted to high-resource languages and the single-transfer (one-to-one) setting. Consequently, these studies offer limited insights for the realistic goal of developing a multi-lingual IE system that can generalize to as many languages as possible. Our paper aims to fill this gap by providing a detailed analysis of Cross-Lingual Multi-Transferability (many-to-many transfer learning) for recent IE corpora that encompass a diverse set of languages. We first establish the correlation between single-transfer performance and various linguistic-based distances, leading to the formulation of a combined distance metric that is both highly correlated and robust across different tasks and model scales. We then explore zero-shot many-to-many transfer settings, whereby the language clustering based on the combined metric provides guidance to achieve the optimal cost-performance trade-off. Finally, a relational-transfer setting is introduced to incorporate multi-lingual unlabeled data through adversarial training, leveraging the relations induced from the linguistic distance. Experimental results on two practical multi-lingual IE tasks show our method significantly improves upon baselines across tasks and languages simultaneously, implying potential for multi-lingual generalization with reasonable labor cost for additional but guided data collection from relevant languages.