Electronic Theses and Dissertation

Automatic speech recognition (asr) trilingual adalah sistem yang mentranskripsikan ucapan tiga bahasa sekaligus. penelitian ini mengembangkan sistem asr trilingual indonesia-arab-inggris melalui fine-tuning model whisper small untuk menangani fenomena code-switching. whisper small dipilih karena kemampuan multibahasanya dari 680.000 jam pre-training. korpus latih hasil kurasi yang diberi nama inagris (indonesia–arab–inggris) dikompilasi dari empat belas sumber data publik (antara lain common voice, fleurs, librispeech, clartts, titml-idn), rekaman code-switching nyata dari podcast hari minggoean dan homostoria serta korpus qcri/escwa, dan augmentasi code-switching sintetis, dengan total 173,86 jam audio dan 83.388 utterance. selain itu, dihimpun langsung himpunan uji independen inf23_cs berisi 571 ujaran code-switching trilingual nyata dari 57 penutur sebagai tolok ukur generalisasi. pelatihan menggunakan strategi curriculum learning dua fase, yaitu fase 1 membangun fondasi monolingual (56.010 utterance) dan fase 2 memperluas ke code-switching (67.167 utterance). evaluasi pada test set (8.163 utterance) menunjukkan penurunan wer yang signifikan dibandingkan model zero-shot. wer bahasa indonesia turun dari 23,93% menjadi 7,78%, bahasa arab dari 38,05% menjadi 15,82%, bahasa inggris dari 7,56% menjadi 3,94%, dan segmen code-switching dari 124,71% menjadi 36,45%. penurunan serupa terlihat pada dua metrik pelengkap, dengan mer keseluruhan turun dari 36,50% menjadi 13,80% dan cer dari 19,67% menjadi 5,79%. model trilingual terbukti setara dengan tiga model monolingual khusus, dengan selisih wer di bawah 2 persen pada semua bahasa, dan mampu mengenali ujaran code-switching. penelitian ini menghasilkan model asr trilingual indonesia-arab-inggris beserta korpus dan tolok ukur evaluasinya.

Trilingual Automatic Speech Recognition (ASR) transcribes speech in three languages simultaneously. This research develops a trilingual Indonesian-Arabic-English ASR system by fine-tuning the Whisper Small model to handle code-switching. Whisper Small was selected for its multilingual capability from 680,000 hours of pre-training. The curated training corpus, named INAGRIS (Indonesian–Arabic–English), was compiled from fourteen publicly available sources (including Common Voice, FLEURS, LibriSpeech, ClArTTS, TITML-IDN), real code-switching recordings from the Hari Minggoean and Homostoria podcasts and the QCRI/ESCWA corpus, and synthetic code-switching augmentation, totaling 173.86 hours of audio and 83,388 utterances. In addition, an independent test set INF23_CS containing 571 genuine trilingual code-switching utterances from 57 speakers was directly collected as a generalization benchmark. Training used a two-phase curriculum learning strategy, where Phase 1 built a monolingual foundation (56,010 utterances) and Phase 2 extended the capability to code-switching (67,167 utterances). Evaluation on a test set of 8,163 utterances shows significant WER reductions compared to the zero-shot model. WER for Indonesian dropped from 23.93% to 7.78%, Arabic from 38.05% to 15.82%, English from 7.56% to 3.94%, and code-switching segments from 124.71% to 36.45%. Similar reductions were observed for the two complementary metrics, with overall MER dropping from 36.50% to 13.80% and CER from 19.67% to 5.79%. The trilingual model proved comparable to three separately trained monolingual models, with WER differences below 2 percent for all languages, and can recognize code-switching speech. This research produces a trilingual Indonesian-Arabic-English ASR model together with its corpus and evaluation benchmark.