Volume 4, Issue 2, December 2020, Page: 42-49
Spanish-Turkish Low-Resource Machine Translation: Unsupervised Learning vs Round-Tripping
Tianyi Xu, Department of Computer Science, Tulane University of Louisiana, New Orleans, United States
Ozge Ilkim Ozbek, Department of Linguistics, Tulane University of Louisiana, New Orleans, United States
Shannon Marks, Department of Linguistics, Tulane University of Louisiana, New Orleans, United States
Sri Korrapati, Department of Linguistics, Tulane University of Louisiana, New Orleans, United States
Benyamin Ahmadnia, Department of Computer Science, Tulane University of Louisiana, New Orleans, United States; Department of Linguistics, University of California, Davis, United States
Received: May 28, 2020;       Accepted: Jun. 18, 2020;       Published: Jul. 23, 2020
DOI: 10.11648/j.ajai.20200402.11      View  40      Downloads  18
Abstract
The quality of data-driven Machine Translation (MT) strongly depends on the quantity as well as the quality of the training dataset. However, collecting a large set of training parallel texts is not easy in practice. Although various approaches have already been proposed to overcome this issue, the lack of large parallel corpora still poses a major practical problem for many language pairs. Since monolingual data plays an important role in boosting fluency for Neural MT (NMT) models, this paper investigates and compares the performance of two learning-based translation approaches for Spanish-Turkish translation as a low-resource setting in case we only have access to large sets of monolingual data in each language; 1) Unsupervised Learning approach, and 2) Round-Tripping approach. Either approach completely removes the need for bilingual data or enables us to train the NMT system relying on monolingual data only. We utilize an Attention-based NMT (Attentional NMT) model, which leverages a careful initialization of the parameters, the denoising effect of language models, and the automatic generation of bilingual data. Our experimental results demonstrate that the Unsupervised Learning approach outperforms the Round-Tripping approach in Spanish-Turkish translation and vice versa. These results confirm that the Unsupervised Learning approach is still a reliable learning-based translation technique for Spanish-Turkish low-resource NMT.
Keywords
Computational Linguistics, Natural Language Processing, Neural Machine Translation, Low-Resource Languages, Unsupervised Learning, Round-Tripping
To cite this article
Tianyi Xu, Ozge Ilkim Ozbek, Shannon Marks, Sri Korrapati, Benyamin Ahmadnia, Spanish-Turkish Low-Resource Machine Translation: Unsupervised Learning vs Round-Tripping, American Journal of Artificial Intelligence. Special Issue: Machine Translation for Low-Resource Languages . Vol. 4, No. 2, 2020, pp. 42-49. doi: 10.11648/j.ajai.20200402.11
Copyright
Copyright © 2020 Authors retain the copyright of this article.
This article is an open access article distributed under the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/) which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Reference
[1]
Lample G., Conneau A., Denoyer L., Ranzato M., Unsupervised machine translation using monolingual corpora only, Proceedings of the International Conference on Learning Representations, 2018.
[2]
Bahdanau D., Cho K., Bengio Y., Neural machine translation by jointly learning to align and translate, Proceedings of the International Conference on Learning Representations, 2015.
[3]
Ahmadnia B., Haffari G., Serrano J., Round-trip training approach for bilingually low-resource statistical machine translation systems, International Journal of Artificial Intelligence, 2019, 17 (1): 167-185.
[4]
Ahmadnia B., Dorr B. J., Augmenting Neural Machine Translation through Round-Trip Training Approach, Open Computer Science, 2019, 9 (1): 268-278.
[5]
Ahmadnia B., Haffari G., Serrano J., Statistical Machine Translation for Bilingually Low-Resource Scenarios: A Round-Tripping Approach, Proceedings of the 3rd IEEE International Conference on Machine Learning and Natural Language Processing, 2018, 261-265.
[6]
He D., Xia Y., Qin T., Wang L., Yu N., Liu T., Ma W., Dual learning for machine translation, Proceedings of the 30th Conference on Neural Information Processing Systems, 2016.
[7]
Wu H., Wang H., Pivot language approach for phrase-based statistical machine translation, Proceedings of ACL: the 45th Annual Meeting of the Association of Computational Linguistics, 2007, 856-863.
[8]
Ahmadnia B., Serrano J., Direct translation vs. pivot language translation for Persian-Spanish low-resourced statistical machine translation system, Proceedings of the 18th International Conference on Artificial Intelligence and Computer Science, 2016.
[9]
Ahmadnia B., Serrano J., Haffari G., Persian-Spanish low-resource statistical machine translation through English as pivot language, Proceedings of Recent Advances in Natural Language Processing, 2017, 24-30.
[10]
Ahmadnia B., Serrano J., Employing pivot language technique through statistical and neural machine translation frameworks: The case of under-resourced Persian-Spanish language pair, International Journal on Natural Language Computing, 2017, 6 (5): 37-47.
[11]
Ahmadnia B., Serrano, Haffari G., Balouchzahi NM., Direct-bridge combination scenario for Persian-Spanish low-resource statistical machine translation, Proceedings of Artificial Intelligence and Natural Language, 2018, 67-78.
[12]
Firat O., Sankaran B., Al-Onaizan Y., Yarman Vural F. T., Cho K., Effective approaches to attention-based neural machine translation, Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2016, 268–277.
[13]
Chen, Y., Liu Y., Cheng Y., Li V., A teacher-student framework for zero- resource neural machine translation, Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017, 1925-1935.
[14]
Sennrich R., Haddow B., Birch A., Improving neural machine translation models with monolingual data, Proceedings of the 54th Annual Meeting of Association for Computational Linguistics, 2016.
[15]
Luong T., Sutskever I., Le Q. V., Vinyals O., Zaremba W., Addressing the rare word problem in neural machine translation, Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, 2015, 11–19.
[16]
Koehn P., Knowles R., Six Challenges for Machine Translation, Proceedings of the First Workshop on Neural Machine Translation, 2017.
[17]
Sennrich R., Zhang B., Revisiting Low-Resource Neural Machine Translation: A Case Study, Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, 211-221.
[18]
Lample G., Ott M., Conneau A., Denoyer L., Ranzato M., Phrase-based & neural unsupervised machine translation, Proceedings of EMNLP, 2018, 5039-5049.
[19]
Imankulova, A., Sato, T., Komachi, M., Improving Low-Resource Neural Machine Translation with Filtered Pseudo-parallel Corpus, Proceedings of the 4th Workshop on Asian Translation, 2017, 70-78.
[20]
Fadaee M., Bisazza A., Monz C., Data augmentation for low-resource neural machine translation, Association for Computational Linguistics, 2019.
[21]
Artetxe, M., Labaka, G., Agirre, E., & Cho, K. 2017. Unsupervised neural machine translation. arXiv preprint arXiv: 1710.11041.
[22]
Artetxe M., Labaka G., Agirre E., An effective approach to unsupervised machine translation, Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, 194-203.
[23]
Hochreiter S., Schmidhuber J., Long short-term memory, Neural Computation, 1997, 9 (8), 1735–1780.
[24]
Luong T., Pham H., Manning C. D., Effective approaches to attention-based neural machine translation, Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2015, 1412–1421.
[25]
Tiedemann J., Parallel data, tools and interfaces in OPUS, Proceedings of the 8th International Conference on Language Resources and Evaluation, 2012.
[26]
Koehn P., Europarl: A parallel corpus for statistical machine translation, Proceedings of the 10th Machine Translation Summit, 2005, 79-86.
[27]
Sennrich R., Haddow B., Birch A., Neural machine translation of rare words with subword units, Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016b, 1715-1725.
[28]
Mikolov T., Sutskever I., Chen K., Corrado G., Dean J., Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems 26, 2013, 3111-3119.
[29]
Mi H., Wang Z., Ittycheriah A., Supervised attentions for neural machine translation, Proceedings of the International Conference on Empirical Methods in Natural Language Processing, 2016, 2283-2288.
[30]
Cohn T., Huang C. D. V., Vymolova E., Yao K., Dyer C., Haffari G., Incorporating structural alignment biases into an attentional neural translation model, Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics Human Language Technologies, 2016, 876-885.
[31]
Sutskever I., Vinyals O., le Q. V., Sequence to sequence learn- ing with neural networks, Proceedings of Advances in Neural Information Processing Systems, 2014, 3104-3112.
[32]
Papineni K., Roukos S., Ward T., Zhu W. J., BLEU: A method for automatic evaluation of machine translation, Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, 2001, 311-318.
[33]
Ahmadnia B., Kordjamshidi P., Haffari G., Neural machine translation advised by statistical machine translation: The case of Farsi-Spanish bilingually low-resource scenario, Proceedings of the 17th IEEE International Conference on Machine Learning and Applications, 2018, 1209-1213.
Browse journals by subject