Comparative Evaluation of Machine Learning and Signa- ture-Based NIDS for Multi-Class and Binary Threat De- tection

Wen-Udeoji, Somayina C; Muyeba, Maybin K; Mohammadi, Azadeh

Abstract

Network Intrusion Detection Systems (NIDS) are essential in safeguarding networks from evolving cyber threats. Traditional signature-based NIDS, such as Snort, struggle with zero-day attacks and lack adaptability. This study presents a unified, empirical evaluation framework comparing Snort with machine learning (ML) models, specifically Random Forest, XGBoost, and Decision Tree, using the UNSW-NB15 dataset. By preserving real-world class distributions and assessing both multi-class and binary threat detection, the framework enables fair and practical comparisons. Results indicate that ensemble models , particularly Random Forest and XGBoost, significantly outperform Snort in both multi-class and binary threat detection tasks. Random Forest achieved an accuracy of 87% compared to Snort's 56.21%, with false positive rates of 25% versus 62.5%, respectively. The Decision Tree and XGBoost models achieved accuracies of 80.2% and 82.84%, respectively, with low false positives and high recall rates. By standardising the comparison between rule-based and ML-based NIDS, this study offers a clearer understanding of their relative strengths in practical scenarios and supports the case for hybrid detection strategies.

• Presentation Conference Type: Conference Paper (published)
• Conference Name: International Conference on Data Science, AI and Applications
• Start Date: Jul 18, 2025
• End Date: Jul 19, 2025
• Acceptance Date: Jun 22, 2025
• Deposit Date: Aug 5, 2025
• Publisher: Springer Verlag
• Peer Reviewed: Peer Reviewed
• Keywords: Network Intrusion Detection System; Network Security; Machine Learning; Signature-Based Detection; UNSW-NB15
• Publisher URL: https://www.springer.com/gp/computer-science/lncs