Electronic Theses and Dissertation

Tes Kemampuan dapat dinilai dengan pendekatan Classical Test Theory
(CTT) dan Item Response Theory (IRT). Penilaian dengan metode CTT sudah umum
digunakan namun hasilnya kurang memperhatikan hubungan antara peserta dengan
karakteristik butir soal. IRT merupakan teori yang dikembangkan untuk
memperbaiki keterbatasan yang dimiliki oleh CTT. Berdasarkan kelebihan dari
metode IRT tersebut, penelitian ini mengimplementasikan metode IRT pada sistem
penilaian ujian berbasis komputer, membandingkan hasil CTT dan IRT,
menganalisis fungsionalitas dan usability dari sistem ujian berbasis komputer.
Sistem dibangun menggunakan framework Django, React, dan Safe Exam Browser
(SEB). Sistem yang telah dibangun diujicoba terhadap 709 peserta calon mahasiswa
Program Pascasarjana (PPs) Universitas Syiah Kuala tahun 2020 periode 1 dengan
20 butir soal. Hasil ujian kemudian dianalisis parameter butir soal dengan hasil 10
butir soal tergolong sedang, 9 butir soal tergolong sulit dan 1 butir soal dieliminasi
karena memiliki nilai daya pembeda negatif. Peringkat yang didapatkan
menggunakan metode IRT dibandingkan dengan peringkat dengan metode CTT.
Hasil perbandingan tersebut menunjukkan bahwa peringkat peserta mengalami
perubahan. Sistem yang telah dibangun diuji fungsionalitasnya menggunakan
metode Blackbox Testing dengan hasilnya, seluruh fitur dan fungsi aplikasi yang
telah dibangun berhasil berjalan dengan baik. Selanjutnya pengujian usability
dilakukan terhadap 537 responden menggunakan System Usability Scale (SUS)
memperoleh skor akhir 74,7%. Berdasarkan skor tersebut dapat disimpulkan bahwa
sistem yang sudah dikembangkan berada pada skala C, dapat diterima dan memiliki
adjective rating excellent.
Kata kunci : Item Response Theory, Classical Test Theory, Ujian Berbasis
Komputer, Black Box Testing, System Usability Scale

Ability tests can be assessed using the Classical Test Theory approach (CTT) and Item Response Theory (IRT). Assessment by the CTT method is commonly used but the results did not pay attention to the relationship between participants and item characteristics. IRT is a theory developed to improve the limitations of the CTT. Based on the advantages of the IRT method, this study implements the IRT method on the computer-based exam assessment, comparing CTT and IRT results, analyze the functionality and usability of the computer-based exam system. The system is built using the Django framework, React, and the Safe Exam Browser (SEB). The system that has been built is tested on 709 prospective student participants Syiah Kuala University Postgraduate Program (PPs) 2020 period 1 with 20 items. The test results are then analyzed for item parameters with the results of 10 Items are classified as moderate, 9 items are classified as difficult and 1 item is eliminated because it has a negative discriminatory value. Rank earned using the IRT method compared to ratings by the CTT method. The results of the comparison show that the ranking of participants experienced change. The system that has been built is tested for functionality using Black box Testing method with the results, all the features, and functions of the application have been built successfully running well. Next usability testing conducted on 537 respondents using the System Usability Scale (SUS) obtained a final score of 74.7%. Based on this score, it can be concluded that the system that has been developed is on a C, acceptable and scale have an excellent rating. Keywords : Item Response Theory, Classical Test Theory, Computer Based Test, Black Box Testing, System Usability Scale.