A Systematic Analysis of Subwords and Cross-Lingual Transfer in Multilingual Translation

Meyer, Francois and Buys, Jan (2024) A Systematic Analysis of Subwords and Cross-Lingual Transfer in Multilingual Translation, Proceedings of Findings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2024), Mexico City, Mexico.

[thumbnail of 2024.findings-naacl.141.pdf] Text
2024.findings-naacl.141.pdf

Download (209kB)

Abstract

Multilingual modelling can improve machine translation for low-resource languages, partly through shared subword representations. This paper studies the role of subword segmentation in cross-lingual transfer. We systematically compare the efficacy of several subword methods in promoting synergy and preventing interference across different linguistic typologies. Our findings show that subword regularisation boosts synergy in multilingual modelling, whereas BPE more effectively facilitates transfer during cross-lingual fine-tuning. Notably, our results suggest that differences in orthographic word boundary conventions (the morphological granularity of written words) may impede cross-lingual transfer more significantly than linguistic unrelatedness. Our study confirms that decisions around subword modelling can be key to optimising the benefits of multilingual modelling.

Item Type: Conference paper
Subjects: Computing methodologies > Artificial intelligence > Natural language processing
Computing methodologies > Artificial intelligence > Natural language processing > Machine translation
Computing methodologies > Artificial intelligence > Natural language processing > Natural language generation
Computing methodologies > Artificial intelligence > Natural language processing > Phonology / morphology
Date Deposited: 08 Aug 2024 08:49
Last Modified: 08 Aug 2024 08:49
URI: https://pubs.cs.uct.ac.za/id/eprint/1670

Actions (login required)

View Item View Item