Electronic Theses and Dissertation

Pemanfaatan pengenalan wajah pada sistem presensi daring masih rentan terhadap serangan face spoofing berupa recaptured face melalui layar digital. Penelitian ini bertujuan mengembangkan model Convolutional Neural Network berbasis MobileOne-S2 dengan mengintegrasikan metode adaptive early fusion untuk menggabungkan informasi domain spasial dan domain frekuensi. Dataset hibrida terdiri atas 10.000 citra yang terbagi secara seimbang menjadi 5.000 citra genuine face dan 5.000 citra recaptured face. Dataset dibagi menjadi data pelatihan, validasi, dan pengujian dengan rasio 60%:20%:20%. Model baseline menggunakan tiga kanal RGB, sedangkan model usulan menggunakan tiga kanal RGB dan satu kanal FFT yang diperoleh melalui pemrosesan fitur frekuensi. Kanal FFT tersebut dibobotkan secara adaptif berdasarkan karakteristik citra RGB sebelum digabungkan dan diproses oleh backbone MobileOne-S2. Evaluasi dilakukan melalui lima eksperimen berpasangan menggunakan seed 12, 13, 14, 15, dan 16 dengan sampel, augmentasi, urutan pengacakan, pembagian dataset, konfigurasi pelatihan, dan strategi pemilihan checkpoint yang sama. Hasil pengujian menunjukkan bahwa modul adaptive early fusion yang merupakan bagian dari feature fusion memperoleh rata-rata Accuracy sebesar 0,9644 ± 0,0190, lebih tinggi 0,0076 poin dibandingkan model baseline yang memperoleh Accuracy sebesar 0,9568 ± 0,0310. Model usulan juga menghasilkan precision sebesar 0,9350, recall sebesar 0,9994, dan F1 Score sebesar 0,9659. Pada Presentation Attack Detection metric, model usulan memperoleh APCER sebesar 0,0706, BPCER sebesar 0,0006, dan ACER sebesar 0,0356, sedangkan model baseline memperoleh APCER sebesar 0,0842, BPCER sebesar 0,0022, dan ACER sebesar 0,0432. Hasil tersebut menunjukkan bahwa adaptive early fusion memberikan peningkatan kinerja rata-rata dan stabilitas antar-seed dibandingkan MobileOne-S2 tanpa kanal frekuensi, meskipun peningkatannya belum konsisten pada seluruh seed. Dengan demikian, integrasi informasi spasial dan frekuensi melalui adaptive early fusion berpotensi meningkatkan kemampuan MobileOne-S2 dalam mendeteksi face spoofing berbasis artefak digital serta dapat dikembangkan lebih lanjut untuk implementasi pada sistem presensi berbasis perangkat bergerak.

Kata kunci: face spoofing, feature fusion, MobileOne, FFT, presensi daring.

The use of facial recognition in online attendance systems remains vulnerable to face spoofing attacks involving recaptured faces displayed on digital screens. This study aims to develop a Convolutional Neural Network model based on MobileOne-S2 by integrating an adaptive early fusion method to combine spatial-domain and frequency-domain information. The hybrid dataset consists of 10,000 images, evenly divided into 5,000 genuine face images and 5,000 recaptured face images. The dataset was divided into training, validation, and testing sets at a ratio of 60%:20%:20%. The baseline model uses three RGB channels, whereas the proposed model uses three RGB channels and one FFT channel obtained through frequency-domain feature processing. The FFT channel is adaptively weighted based on the characteristics of the RGB image before being fused and processed by the MobileOne-S2 backbone. Evaluation was conducted through five paired experiments using seeds 12, 13, 14, 15, and 16, with identical samples, augmentations, shuffling order, dataset partitioning, training configurations, and checkpoint-selection strategies. The test results show that the model with adaptive early fusion achieved a mean accuracy of 0.9644 ± 0.0190, which was 0.0076 points higher than the baseline model, which achieved an accuracy of 0.9568 ± 0.0310. The proposed model also achieved a precision of 0.9350, a recall of 0.9994, and an F1-score of 0.9659. In terms of Presentation Attack Detection metrics, the proposed model achieved an APCER of 0.0706, a BPCER of 0.0006, and an ACER of 0.0356, whereas the baseline model achieved an APCER of 0.0842, a BPCER of 0.0022, and an ACER of 0.0432. These results indicate that adaptive early fusion improves average performance and cross-seed stability compared with MobileOne-S2 without the frequency channel, although the improvement is not consistent across all seeds. Therefore, integrating spatial and frequency information through adaptive early fusion has the potential to improve the ability of MobileOne-S2 to detect face spoofing based on digital artifacts and may be further developed for implementation in mobile-device-based attendance systems. Keywords: face spoofing, feature fusion, MobileOne, FFT, online attendance.