Electronic Theses and Dissertation

Fenomena code-switching (CS) antara bahasa Indonesia, Arab, dan Inggris semakin umum ditemukan dalam komunikasi multibahasa di Indonesia. Namun, sistem Automatic Speech Recognition (ASR) monolingual masih mengalami penurunan performa akibat peralihan bahasa dalam satu ujaran, perbedaan sistem penulisan Latin dan Arab, serta keterbatasan korpus CS trilingual. Penelitian ini bertujuan mengembangkan sistem ASR end-to-end (E2E) trilingual Indonesia–Arab–Inggris yang mampu mengenali tuturan monolingual dan code-switching. Dataset penelitian disusun melalui integrasi 17 sumber data yang terdiri atas korpus monolingual bahasa Indonesia, Arab, dan Inggris serta korpus code-switching sintetis, menghasilkan total 174,82 jam audio dan 83.964 ucapan. Sistem dikembangkan menggunakan ESPnet toolkit dengan arsitektur Conformer encoder–Transformer decoder, fungsi kehilangan CTC+attention, dan representasi subkata Byte-Pair Encoding (BPE). Penelitian dilakukan melalui tiga tahap eksperimen, yaitu perbandingan model monolingual dan trilingual, integrasi language model OPT-350M melalui shallow fusion, serta integrasi data CS menggunakan pendekatan retraining dan fine-tuning. Evaluasi menggunakan metrik Character Error Rate (CER) dan Word Error Rate (WER). Model exp3_finetune_lm menghasilkan performa terbaik pada data monolingual dengan CER 3,9–6,5% dan WER 11,7–18,2%, sedangkan model exp3_retrain_lm memperoleh performa terbaik pada data code-switching dengan CER 24,0% dan WER 49,5%. Hasil penelitian menunjukkan bahwa integrasi language model lebih efektif untuk meningkatkan pengenalan monolingual, sedangkan penambahan data CS secara eksplisit melalui retraining lebih efektif untuk pengenalan ujaran campuran. Penelitian ini menyediakan kerangka pengembangan ASR trilingual untuk bahasa dengan sumber daya terbatas dan skenario code-switching.

Code-switching (CS) between Indonesian, Arabic, and English has become increasingly common in multilingual communication in Indonesia. However, conventional monolingual Automatic Speech Recognition (ASR) systems still experience significant performance degradation due to language switching within an utterance, differences between Latin and Arabic writing systems, and the limited availability of trilingual CS corpora. This study aims to develop an end-to-end (E2E) trilingual ASR system for Indonesian, Arabic, and English that can recognize both monolingual and code-switching speech. The dataset was constructed by integrating 17 data sources consisting of Indonesian, Arabic, and English monolingual corpora and a synthetic code-switching corpus, resulting in a total of 174,82 hours of audio and 83.964 utterances. The system was developed using the ESPnet toolkit with a Conformer encoder–Transformer decoder architecture, a joint CTC-attention loss function, and Byte-Pair Encoding (BPE) subword representations. The study consisted of three experimental stages: comparison between monolingual and trilingual models, integration of the OPT-350M language model through shallow fusion, and incorporation of code-switching data using retraining and fine-tuning approaches. Evaluation was conducted using Character Error Rate (CER) and Word Error Rate (WER). The exp3_finetune_lm model achieved the best performance on monolingual data with CER of 3,9–6,5% and WER of 11,7–18,2%, while the exp3_retrain_lm model achieved the best performance on code-switching data with CER of 24,0% and WER of 49,5%. The results indicate that language model integration is more effective for improving monolingual recognition, whereas explicit integration of code-switching data through retraining is more effective for recognizing mixed-language speech. This study provides a framework for developing trilingual ASR systems for low-resource languages and code-switching scenarios.