Electronic Theses and Dissertation

Tes kemampuan dapat dinilai dengan pendekatan classical test theory (ctt) dan item response theory (irt). penilaian dengan metode ctt sudah umum digunakan namun hasilnya kurang memperhatikan hubungan antara peserta dengan karakteristik butir soal. irt merupakan teori yang dikembangkan untuk memperbaiki keterbatasan yang dimiliki oleh ctt. berdasarkan kelebihan dari metode irt tersebut, penelitian ini mengimplementasikan metode irt pada sistem penilaian ujian berbasis komputer, membandingkan hasil ctt dan irt, menganalisis fungsionalitas dan usability dari sistem ujian berbasis komputer. sistem dibangun menggunakan framework django, react, dan safe exam browser (seb). sistem yang telah dibangun diujicoba terhadap 709 peserta calon mahasiswa program pascasarjana (pps) universitas syiah kuala tahun 2020 periode 1 dengan 20 butir soal. hasil ujian kemudian dianalisis parameter butir soal dengan hasil 10 butir soal tergolong sedang, 9 butir soal tergolong sulit dan 1 butir soal dieliminasi karena memiliki nilai daya pembeda negatif. peringkat yang didapatkan menggunakan metode irt dibandingkan dengan peringkat dengan metode ctt. hasil perbandingan tersebut menunjukkan bahwa peringkat peserta mengalami perubahan. sistem yang telah dibangun diuji fungsionalitasnya menggunakan metode blackbox testing dengan hasilnya, seluruh fitur dan fungsi aplikasi yang telah dibangun berhasil berjalan dengan baik. selanjutnya pengujian usability dilakukan terhadap 537 responden menggunakan system usability scale (sus) memperoleh skor akhir 74,7%. berdasarkan skor tersebut dapat disimpulkan bahwa sistem yang sudah dikembangkan berada pada skala c, dapat diterima dan memiliki adjective rating excellent. kata kunci : item response theory, classical test theory, ujian berbasis komputer, black box testing, system usability scale

Ability tests can be assessed using the Classical Test Theory approach (CTT) and Item Response Theory (IRT). Assessment by the CTT method is commonly used but the results did not pay attention to the relationship between participants and item characteristics. IRT is a theory developed to improve the limitations of the CTT. Based on the advantages of the IRT method, this study implements the IRT method on the computer-based exam assessment, comparing CTT and IRT results, analyze the functionality and usability of the computer-based exam system. The system is built using the Django framework, React, and the Safe Exam Browser (SEB). The system that has been built is tested on 709 prospective student participants Syiah Kuala University Postgraduate Program (PPs) 2020 period 1 with 20 items. The test results are then analyzed for item parameters with the results of 10 Items are classified as moderate, 9 items are classified as difficult and 1 item is eliminated because it has a negative discriminatory value. Rank earned using the IRT method compared to ratings by the CTT method. The results of the comparison show that the ranking of participants experienced change. The system that has been built is tested for functionality using Black box Testing method with the results, all the features, and functions of the application have been built successfully running well. Next usability testing conducted on 537 respondents using the System Usability Scale (SUS) obtained a final score of 74.7%. Based on this score, it can be concluded that the system that has been developed is on a C, acceptable and scale have an excellent rating. Keywords : Item Response Theory, Classical Test Theory, Computer Based Test, Black Box Testing, System Usability Scale.