Deep Learning for Network Intrusion Detection on NSL-KDD

Aykhan Mammadli, Electrical and Computer Engineering, New York University

am15226@[Link]

Abstract weighting provides only small gains for logistic regression

and does not improve the MLP in this setting.
Intrusion Detection Systems classify network connections as
benign or malicious. We study binary intrusion detection on
the NSL-KDD benchmark and compare a classical baseline Related Work
like logistic regression with a multilayer perceptron as known
as MLP. Because missed attacks are costly and IDS datasets
are often class-imbalanced, we emphasize attack-class recall
and F1-score and test imbalance-aware training (balanced Benchmark Datasets for IDS
class weights / weighted loss). On the NSL-KDD test set,
the best model is an MLP with pos weight achieving Re- KDD Cup 99 and its variant NSL-KDD have been widely
call=0.688 and F1=0.793, narrowly edging the unweighted used in IDS research. However, the original KDD Cup 99
MLP (Recall=0.680, F1=0.791) and outperforming logistic dataset contains redundant records that can bias training
regression (Recall=0.591, F1=0.718). Source code: (https: and inflate evaluation scores. NSL-KDD was proposed to
//[Link]/AykhanJ/DeepLearningProject) mitigate these issues by reducing redundancy and provid-
ing a more meaningful benchmark for comparing methods
[Tavallaee et al.(2009)Tavallaee, Bagheri, Lu, and Ghorbani].
Introduction NSL-KDD still remains a common reference point for
Networked systems face attacks such as probing, denial-of- controlled evaluations, despite it is not a modern enterprise
service, and credential abuse. An Intrusion Detection Sys- traffic dataset.
tem aims to identify malicious activity from observed traf-
fic while keeping false alarms manageable. The most costly
errors are considered false negatives in many security set-
Classical Machine Learning Approaches
tings. This motivates evaluating models with attack-class re-
call and F1-score rather than accuracy alone. Another practi- Traditional IDS approaches often rely on classical machine
cal challenge is data imbalance. Data imbalance means, be- learning models trained on engineered connection features,
nign traffic may dominate, and certain attack patterns can be including decision trees, support vector machines, and lo-
under-represented. As a result, IDS evaluation typically em- gistic regression. These models are attractive because they
phasizes recall/F1 for the attack class and tests robustness are relatively simple, fast to train, and can be easier to inter-
techniques that mitigate imbalance. pret than deep networks. Logistic regression in particular is
Our project formulates IDS as a supervised bi- a strong baseline for linearly separable problems and is com-
nary classification task using the NSL-KDD dataset, monly used to establish a minimum-performance reference.
a widely used benchmark derived from KDD Cup 99
[Tavallaee et al.(2009)Tavallaee, Bagheri, Lu, and Ghorbani].
Additionally, Our repository includes a README with
step-by-step commands to reproduce preprocessing, train- Deep Learning for IDS
ing, and table/figure generation.
We compare a strong classical baseline such as logistic Deep learning approaches aim to capture non-linear fea-
regression against a simple deep learning model, MLP. To ture interactions and reduce reliance on hand-crafted
strengthen the methodology, we add an imbalance-handling decision rules. Shone et al. report competitive results
extension and empirically evaluate its effect on both model with deep-learning-based intrusion detection and high-
families. Our main finding is that the MLP improves test- light benefits of learned representations for detection
set recall and F1 over logistic regression. But imbalance [Shone et al.(2018)Shone, Ngoc, Phai, and Shi].
Other work studies deep and recurrent architec-
Copyright © 2023, Association for the Advancement of Artificial tures for IDS, including RNN-based intrusion de-
Intelligence ([Link]). All rights reserved. tection [Yin et al.(2017)Yin, Zhu, Fei, and He], and
deep models applied to network traffic features Methodology
[Javaid et al.(2016)Javaid, Niyaz, Sun, and Alam].
Baseline: Logistic Regression
Architectural Choice We train logistic regression on the processed feature vectors
We use an MLP because NSL-KDD provides fixed-length, using the SAGA solver with a maximum of 500 iterations.
connection-level tabular features after one-hot encoding and We evaluate two variants: (a) unweighted training and (b)
standardization. An MLP is a natural baseline for learn- balanced class weights via class weight=balanced,
ing non-linear interactions among these heterogeneous at- which reweights samples inversely proportional to class fre-
tributes. In contrast, models designed for sequences are most quency.
appropriate when raw packet streams or time-ordered flow
records are available, which this benchmark does not di- Deep Model: Multilayer Perceptron
rectly provide. A strong non-neural alternative for tabular Our deep model is a feedforward MLP with hidden layer
IDS is tree-based ensembles (e.g., Random Forest or gradi- sizes [256, 128, 64], ReLU activations, and dropout 0.2. The
ent boosting), which often perform competitively on struc- network outputs a single logit and is trained with binary
tured features. We leave this comparison for future work. cross-entropy with logits. We use Adam with learning rate
Autoencoder ensembles have also been pro- 10−3 and batch size 512. Early stopping is applied based on
posed for online anomaly detection. It empha- validation F1-score with patience 5; the best checkpoint by
sizes adaptive representations and streaming settings validation F1 is used for test evaluation.
[Mirsky et al.(2018)Mirsky, Doitshman, Elovici, and Shabtai].
Imbalance Handling Extension
To address class imbalance as a methodological extension,
we evaluate (i) balanced class weights for logistic regression
Imbalance and Evaluation Metrics and (ii) a weighted loss for the MLP using a positive-class
A persistent issue in IDS datasets is class imbalance. weight pos weight computed from the training split as
Imbalance-aware learning (class weighting, resampling, or the ratio of negative to positive examples. This increases the
loss reweighting) is often used to improve detection of rare penalty for misclassifying attacks.
malicious events. Because missed attacks can be costly, IDS
papers frequently report recall and F1 for the attack class in Hyperparameter Selection and Training Details
addition to accuracy, and confusion matrices are important Hyperparameters were selected using the validation set with
to understand false negatives and false positives. attack-class F1 as the primary model-selection metric. For
logistic regression we fixed common stable settings (SAGA
solver, max iter=500). For the MLP we chose a small archi-
tecture suitable for tabular data and tuned only lightweight
Dataset and Preprocessing parameters (dropout and early stopping). We use a fixed ran-
We use NSL-KDD, where each instance corresponds to a dom seed for reproducibility. All models share the same pre-
network connection described by mixed feature types, in- processing, splits, and evaluation procedure to ensure a fair
cluding categorical attributes (for example, protocol type, comparison.
service, flag) and numeric attributes (byte counts, error rates,
traffic statistics and etc.). We convert original labels into Evaluation Metrics
a binary target: normal (0) versus attack (1). The official We report accuracy, precision, recall, and F1-score for the
dataset provides separate train and test files; we further split positive class (attack). Confusion matrices are also included
the training file to create a validation set. to interpret types of errors. Predictions use a 0.5 threshold
on sigmoid probability (equivalently, logit ≥ 0).
Preprocessing Pipeline
We apply the following preprocessing steps:
• Categorical encoding: one-hot encode categorical at-
tributes. Unknown categories are ignored at inference.
Experiments and Results
• Feature scaling: standardize numeric features using Main Results
training statistics only (mean 0, variance 1).
Table 1 reports test-set performance. Logistic regression
• Data splits: create a stratified validation split from the achieves F1=0.718 and Recall=0.591. Applying balanced
training file (15% validation). The official test file is used class weights yields a modest improvement (F1=0.724,
only for final evaluation to prevent tuning on test data. Recall=0.599). The unweighted MLP substantially out-
After preprocessing, each example becomes a fixed- performs both logistic regression variants (F1=0.791, Re-
dimensional numeric vector used by both logistic regression call=0.680).
and the MLP. We also save the fitted preprocessor to ensure Adding pos weight to the MLP yields a very small
reproducibility. improvement in Recall and F1 (Recall=0.688, F1=0.793),
 Model Acc Prec Recall F1 ods, and cross-dataset evaluation to better estimate real-
LogReg 0.736 0.915 0.591 0.718 world generalization.
LogReg + class weight 0.740 0.915 0.599 0.724
MLP 0.795 0.944 0.680 0.791 References
MLP + pos weight 0.795 0.936 0.688 0.793
[Javaid et al.(2016)Javaid, Niyaz, Sun, and Alam] Javaid,
A.; Niyaz, Q.; Sun, W.; and Alam, M. 2016. A Deep
Table 1: Performance on NSL-KDD (attack is positive
Learning Approach for Network Intrusion Detection
class).
System. Proceedings of the 9th EAI International
Conference on Bio-inspired Information and Communi-
1.0 cations Technologies (formerly BIONETICS).
[Mirsky et al.(2018)Mirsky, Doitshman, Elovici, and Shabtai]
0.8 Mirsky, Y.; Doitshman, D.; Elovici, Y.; and Shabtai,
A. 2018. Kitsune: An Ensemble of Autoencoders for
Online Network Intrusion Detection. Network and
0.6 Distributed System Security Symposium (NDSS).
train loss
val F1 [Shone et al.(2018)Shone, Ngoc, Phai, and Shi] Shone, N.;
0.4 Ngoc, T.; Phai, V. D.; and Shi, Q. 2018. A Deep Learn-
ing Approach to Network Intrusion Detection. IEEE
Transactions on Emerging Topics in Computational In-
0.2 telligence.
[Tavallaee et al.(2009)Tavallaee, Bagheri, Lu, and Ghorbani]
0.0 Tavallaee, M.; Bagheri, E.; Lu, W.; and Ghorbani, A. A.
0 5 10 15 20 25 30 2009. A Detailed Analysis of the KDD CUP 99 Data
epoch Set. In Proceedings of the 2009 IEEE Symposium on
Computational Intelligence for Security and Defense
Figure 1: MLP training loss and validation F1 across epochs Applications (CISDA).
(early stopping selects the best validation F1). [Yin et al.(2017)Yin, Zhu, Fei, and He] Yin, C.; Zhu, Y.;
Fei, J.; and He, X. 2017. A Deep Learning Approach for
Intrusion Detection Using Recurrent Neural Networks.
while slightly reducing precision (0.936 vs. 0.944). This IEEE Access.
suggests the weighted loss mildly shifts predictions toward
the positive class, improving missed-attack rate at the cost
of a small increase in false positives.

Discussion: Validation vs. Test Gap

Validation performance is substantially higher than test per-
formance. The behavior is consistent with known differ-
ences between NSL-KDD training and test distributions as
the test set includes harder and less frequent patterns. There-
fore, we emphasize the official test-set results for final com-
parison and keep model selection restricted to the validation
split.

Conclusion
We originally intended to determine if a simple deep learn-
ing model could outperform a classical baseline on im-
balanced network traffic data. We implemented an end-to-
end IDS classification pipeline on NSL-KDD and com-
pared logistic regression against a simple deep MLP. The
strongest test performance was achieved by the MLP with
pos weight (Recall=0.688, F1=0.793), improving over
the baseline (LogReg: Recall=0.591, F1=0.718). We also
evaluated imbalance-aware training as a methodological ex-
tension: it provided small gains for logistic regression and
only marginal gains for the MLP under the NSL-KDD test
distribution. Future work could explore threshold tuning to
trade off recall vs. false alarms, alternative imbalance meth-