Electronic Theses and Dissertation

Protokol message queuing telemetry transport (mqtt) merupakan teknologi komunikasi yang digunakan pada internet of things (iot), namun titik sentral broker mqtt sangat rentan terhadap serangan denial of service salah satunya tipe tcp syn flood yang dapat mengeksploitasi kelemahan three-way handshake tcp. penelitian ini bertujuan membandingkan ketahanan kinerja empat broker mqtt (mosquitto, vernemq, moquette, dan vert.x mqtt) pada kondisi baseline dan saat diserang. pengujian dilakukan pada lingkungan virtual terisolasi menggunakan virtual machine ubuntu menggunakan utilitas hping3 untuk serangan terhadap broker, variasi konfigurasi tingkat quality of service (qos 0, 1, 2) dan laju pesan (100, 1.000, 10.000 msg/s). data pengujian dikumpulkan berdasarkan metrik throughput, latency, packet loss, dan penggunaan cpu, kemudian dianalisis menggunakan metode ordinal score untuk menentukan rekomendasi broker secara objektif. hasil pengujian menunjukkan bahwa serangan memicu degradasi kinerja yang signifikan pada semua broker: throughput rata-rata menurun 830,75–1.174,79 msg/s, latency melonjak 6.190,04–14.973,60 ms, packet loss meningkat 72,66%–92,42%, dan penggunaan cpu naik 22,48%–56,97%. berdasarkan hasil skoring, mosquitto memperoleh nilai tertinggi (24/32) berkat ketahanan throughput dan efisiensi penggunaan cpu, sehingga paling direkomendasikan untuk sistem iot berkeandalan tinggi. vernemq menempati posisi kedua (skor 21/32) sebagai alternatif sistem yang mengutamakan responsivitas, sedangkan moquette dan vert.x mqtt menunjukkan kelemahan fatal saat diserang sehingga tidak direkomendasikan. kata kunci: iot, mqtt broker, denial of service, tcp syn flood, quality of service, ordinal score, wireshark

The Message Queuing Telemetry Transport (MQTT) protocol is a communication technology widely used in the Internet of Things (IoT). However, the centralized nature of the MQTT broker makes it highly vulnerable to Denial of Service attacks, one of which is the TCP SYN Flood type that can exploit the weaknesses of the TCP three-way handshake. This study aims to compare the performance resilience of four MQTT brokers (Mosquitto, VerneMQ, Moquette, and Vert.x MQTT) under baseline conditions and under attack. Testing was conducted in an isolated virtual environment using Ubuntu virtual machines, employing the hping3 utility to launch attacks against the brokers, with variations in Quality of Service (QoS) levels (0, 1, 2) and message rates (100, 1,000, 10,000 msg/s). Test data were collected based on throughput, latency, packet loss, and CPU usage metrics, then analyzed using the ordinal score method to objectively determine broker recommendations. The test results show that the attack triggered significant performance degradation across all brokers: average throughput decreased by 830.75–1,174.79 msg/s, latency spiked by 6,190.04–14,973.60 ms, packet loss increased by 72.66%–92.42%, and CPU usage rose by 22.48%–56.97%. Based on the scoring results, Mosquitto obtained the highest score (24/32) due to its throughput resilience and CPU usage efficiency, making it the most recommended broker for high-reliability IoT systems. VerneMQ ranked second (score 21/32) as an alternative system prioritizing responsiveness, while Moquette and Vert.x MQTT exhibited fatal weaknesses under attack and are therefore not recommended. Keywords: IoT, MQTT broker, Denial of Service, TCP SYN Flood, Quality of Service, Ordinal Score, Wireshark