Network Intrusion Detection System Using Machine

Learning
Shubham Sharma1, Kartikeya Bajpai2, Vikas Sagar3
1
Student, CSE(CYS), Noida Institute of Engineering and Technology, Greater Noida, India
2
Student, CSE(CYS), Noida Institute of Engineering and Technology, Greater Noida, India
3
Deputy HOD, CSE(CYS), Noida Institute of Engineering and Technology, Greater Noida, India

Abstract—The growing dependency on computer networks The primary objective of this research is to design and
has resulted in higher cyberattack risks which include implement a Machine Learning-based NIDS that:
Denial-of-Service (DoS) attacks and network probing and a) Accurately detects network intrusions
b) Reduces false positive and false negative rates
unauthorized access attempts. Traditional intrusion detection
c) Operates efficiently in real-time environments
systems face limitations in detecting emerging threats
because they rely on established detection patterns. d) Provides scalability for large network
infrastructures
This paper proposes a Machine Learning-based Network
Intrusion Detection System (NIDS) for real-time detection II. RELATED WORK
of malicious network activities. The system integrates data
preprocessing, feature engineering, and a Random Forest A. Evolution of Intrusion Detection Systems
classifier to achieve efficient and accurate detection. The first intrusion detection systems operated through
predefined rules which used manually developed signatures
The NSL-KDD dataset evaluation of the model resulted for their detection process. The detection systems worked
in 97% accuracy and 95% precision and 96% recall and well against existing threats but they needed continuous
95.5% F1-score result. The system demonstrates its system updates and they could not identify new security
capability to detect both existing and new attacks through its threats. Researchers began investigating machine learning
low false positive rate. The system functions as a methods for intrusion detection because artificial
lightweight solution which can scale to different intelligence development had reached a stage of advanced
requirements and operates in real-time for actual network progress.
environments.
B. Machine Learning Techniques for IDS

Keywords Network Intrusion Detection, Machine Learning, Existing intrusion detection systems display strong
Cybersecurity, Anomaly Detection, NSL-KDD, Classification detection accuracy in controlled testing environments. The
systems face multiple obstacles which limit their ability to
function in real-world environments:
I. INTRODUCTION • Dependence on centralized cloud infrastructure for
The ongoing development of digital networks has model inference, leading to increased latency and
resulted in more cyber attacks that target network systems. privacy concerns.
The essential nature of network connections for • Limited capability for real-time detection in low-
organizational communication and data sharing and cloud resource or high-traffic network environments.
service access makes them susceptible to various cyber • High computational requirements, making
threats which include DoS attacks and phishing attempts deployment difficult on edge devices or local
and ransomware attacks and unauthorized system entry. systems.
Modern threat environments require enhanced security • Lack of lightweight monitoring interfaces or
solutions that go beyond traditional security methods dashboards for practical, real-time visualization
which include firewalls and signature-based intrusion and analysis.
detection systems. The systems operate through To address these limitations, this study proposes a
established guidelines and recognized attack patterns lightweight, locally deployable Machine Learning-based
which restrict their capability to identify both existing and Network Intrusion Detection System capable of real-time
future security threats. traffic analysis without reliance on cloud connectivity.
Intrusion Detection Systems (IDS) are categorized into: The proposed system integrates an optimized classification
• Signature-based IDS: Detect known attacks using model with a simple and efficient monitoring interface (e.g.,
predefined patterns a dashboard using frameworks like Flask or Streamlit) to
• Anomaly-based IDS: Detect deviations from provide real-time visualization of detected intrusions, traffic
normal behavior statistics, and alerts.
Machine Learning (ML) enhances anomaly detection by The new system design operates entirely through local
enabling systems to learn patterns from historical network systems because it avoids all cloud-based dependencies
data and identify abnormal behavior without explicit rule which results in faster processing times and better data
definition. protection while allowing operation in environments with
limited resources like small business networks and
educational facilities and remote computer systems.
 categorical features (such as protocol type, service, and flag).
Table 1. Comparative analysis of existing intrusion detection methods. To simplify the classification task and improve model
performance, the dataset is structured into two main classes:
Approach Key Features Limitations References
normal and intrusion. All attack categories are grouped under
the intrusion class. This binary classification approach helps
Approach Key Features Limitations References
Requires cloud in efficient detection of malicious activities while reducing
High accuracy;
Cloud-based infrastructure;
IDS with ML
scalable; supports
latency issues;
[1] model complexity.
large datasets
privacy concerns B. Feature Transformation and Encoding Procedure
Deep Learning Captures complex High computational
IDS patterns; high [2] The network traffic dataset used in this study originally
cost; requires GPUs
(CNN/LSTM) detection rate contains a mixture of categorical and numerical features,
Effective Slow training; not which are not directly suitable for machine learning models.
SVM-based IDS classification; good scalable [3] Therefore, a preprocessing utility was developed to transform
for small datasets raw network data into a normalized feature vector format
Cannot detect compatible with the classification algorithm.
Signature-based Fast detection of
unknown attacks; [4]
IDS known attacks
requires updates The original dataset includes categorical attributes such as
Local ML model;
protocol type, service, and flag, along with numerical
Requires labeled attributes representing traffic statistics. These categorical
real-time detection;
Proposed dataset; limited by
System
lightweight
training data
features were encoded into numerical representations using
dashboard; low label encoding or one-hot encoding techniques.
diversity
latency
Furthermore, numerical features were normalized with
respect to their minimum and maximum values to ensure
uniform scaling across all input dimensions. The
III. METHODOLOGY AND PROPOSED METHOD normalization process is defined as follows:
𝑥−𝜇
In this section, we describe the dataset, preprocessing 𝑧= (2)
techniques, and machine learning model used for developing 𝜎
where:
the proposed Network Intrusion Detection System (NIDS).
The methodology focuses on analyzing network traffic data • 𝑥represents the original feature value
and classifying it into normal and malicious categories. The • 𝑥′represents the normalized feature
performance of the proposed model is evaluated using
• 𝑥 ,𝑥 are minimum and maximum values
standard evaluation metrics and compared with baseline • 𝑚𝑖𝑛 𝑚𝑎𝑥

approaches. 𝜇is the mean

• 𝜎is the standard deviation
A. System Overview
This transformation ensures that all features contribute
The Machine Learning-based Network Intrusion Detection
equally during model training and prevents bias caused
System (NIDS) in the proposed framework detects
by large-scale variations in feature values.
malicious network traffic in real-time. The system
architecture (illustrated in Fig. 1) consists of four major
components which include data acquisition, preprocessing Additionally, the preprocessing pipeline validates data
and feature engineering, model training, and real-time integrity, removes redundant or inconsistent entries, and
inference with a monitoring dashboard. ensures that each record is correctly mapped to its
The system executes its complete process on local machines corresponding label (normal or attack). The processed
because it does not rely on cloud services which provides dataset is then converted into a structured feature matrix
advantages of reduced latency and better data privacy and suitable for machine learning training.
decreased need for high-bandwidth network connections.
The system monitors network traffic continuously while it The complete transformation procedure is summarized
processes incoming data to classify it as either normal in Algorithm 1.
activity or an intrusion based on results from its trained
machine learning model.
The system architecture supports deployment in educational
institutions and small businesses and local networks that Inputs:
require continuous security monitoring.
B. Dataset Description and Class Design RAW dataset: D
The researchers conducted all their experiments in this No of Features : n
research study using the NSL-KDD dataset which serves as
a standard benchmark dataset for studying network intrusion Outputs:
detection. The dataset contains labeled network traffic
records which are divided into normal traffic and different Feature vector: 𝐹 = [f₁, f₂, ..., fₙ]
attack categories including Denial-of-Service (DoS) attacks, 1: load dataset 𝐷
Probe attacks and Remote-to-Local (R2L) as well as User- 2: remove duplicate and corrupted records
to-Root (U2R) attacks. The dataset record includes multiple
features which describe the network connection
characteristics. These include both numerical features (such
as duration, number of- bytes.
3: handle missing values minimizing classification error while maintaining stability
4: for each feature fᵢ in dataset do and robustness of the model.
5: if fᵢ is categorical then
6: apply encoding (label / one-hot)
7: else
8: normalize using:
fᵢ' = (fᵢ − min) / (max − min)
9: end if
10: end for
11: construct feature vector 𝐹
12: assign class label (normal / attack)
13: return processed dataset

Algorithm1. Data preprocessing and feature transformation pipeline for

converting raw network traffic data into normalized feature vectors suitable
for machine learning-based intrusion detection.

D. Feature Selection and Transformation

Feature selection is performed to reduce dimensionality
and improve computational efficiency.
Techniques used include:
• Correlation analysis
• Feature importance ranking
• Recursive feature elimination
This step eliminates redundant and irrelevant features,
enhancing model accuracy and reducing training time.

E. Model Architecture
The research employs a Random Forest classifier
because it provides fast processing capabilities while
successfully detecting intrusions through its ability to
analyze high-dimensional network traffic data in real-time.
Fig 1Training strategy pipeline for the proposed Network Intrusion Detection
The system begins its operation by processing network data System.
through three steps which include normalization and
encoding and feature selection to create structured input
features from raw network data. The processed features are G. Inference Pipeline
forwarded to the ensemble learning core which contains The deployed trained model operates in an inference
multiple decision trees that have been trained through pipeline for real-time network traffic data analysis. The
bagging techniques to identify patterns in both normal and system continuously monitors network traffic by processing
malicious traffic. The classification stage combines all tree incoming traffic instances as they arrive instead of using
outputs through majority voting to produce the final batch processing methods. The inference process starts with
prediction along with confidence scores. The architecture network record preprocessing which includes normalizing
outperforms traditional single-model approaches by data and converting it into a feature vector for classification
providing better accuracy and stronger resistance to noise model analysis.
while delivering consistent performance, which makes it The system predicts normal traffic patterns and potential
suitable for use in real-world network environments. security breaches through its analysis of incoming data. The
system establishes trustable identification results through its
use of a specific prediction threshold method. The system
F. Training Strategy generates alerts based on the results while it continuously
The model is trained through multiple epochs to achieve updates network traffic data throughout the operational
proper convergence while learning network traffic patterns. period. The inference pipeline delivers efficient performance
Cross-validation techniques help the model to generalize at minimal latency, which makes it suitable for real-world
better while preventing overfitting on training data. The network deployment without needing cloud-based services.
performance of the system gets optimized through the The process of intrusion detection is summarized in
precise tuning of specific hyperparameters which include the Algorithm 2.
number of trees and maximum depth. The overall
training process focuses on the dataset NSL KDD
Dataset.
Inputs: varying traffic patterns, making it suitable for real-
Image/frame input: 𝐼𝑡, where 𝑡 indexes time time intrusion detection scenarios.
Model parameters: Θ
Threshold: 𝜏
Table 2. Performance metrics for the proposed intrusion detection model

Outputs:
Metric Value
Predicted lables : Y
Confidence values: 𝑝 Precision 0.95
Recall 0.96
1: Load trained model parameters θbest Accuracy 0.97
2: Acquire input network instance 𝐼ₜ F1-Score 0.955
3: Preprocess 𝐼ₜ (encoding and normalization) The table displays Precision, Recall, Accuracy, and F1-
4: Convert 𝐼ₜ into feature vector 𝐹ₜ Score values which the proposed Machine Learning-based
5: Forward propagate 𝐹ₜ through model f(·; θbest) intrusion detection model achieved on previously untested
6: Obtain prediction probabilities: network traffic data for its evaluation process. The model
𝑝ₜ = {p_normal, p_attack} demonstrates continuous performance enhancement from its
7: If p_attack ≥ τ then first to its last training epoch through improved
8: label = "Attack" classification accuracy and stable prediction results. The
9: Else system exhibits strong model convergence and statistical
10: label = "Normal" stability because its performance across training epochs
11: End if shows minimal changes during the final testing period. The
12: Generate alert if intrusion detected system demonstrates its ability to detect malicious network
13: Update logs and monitoring dashboard activities through balanced precision and recall values
14: Return label and confidence values which help reduce false positive errors.
Algorithm 2. Inference procedure using a trained machine learning model for
real-time network intrusion detection, including preprocessing,
classification, threshold-based decision making, and alert generation.
B. Training Behaviour Analysis
The assessment of model development involved studying
IV. RESULTS AND DISCUSSIONS its training process across different training periods to
The research on network intrusion detection found major determine how well the model achieved learning goals and
behavioral patterns through its detection of illegal and maintained stability while acquiring knowledge. The
unusual network activities which occurred during various training loss demonstrated a continuous downward trend
cyberattack types. The study showed through its assessment which progressed in a steady manner because the learning
of Machine Learning-based intrusion detection performance process would not experience major interruptions or
that researchers faced major challenges when trying to unpredictable behavior during this time period.
detect intrusions in network environments which Progressive loss reduction demonstrates how the model
experienced high traffic and operated with limited resources. develops its capacity to identify essential traffic network
The main results demonstrate that the system successfully patterns through ongoing learning while maintaining low
distinguishes between standard and attack traffic while classification mistakes. The model achieves effective traffic
maintaining performance across different network scenarios. pattern recognition through its integrated feature extraction
The results demonstrate that the system functions effectively and model training system which allows it to identify both
in real-time applications which require immediate threat standard and unusual traffic patterns.
detection and response in extensive network systems with The proposed model demonstrates its ability to maintain
constantly changing environments. proper optimization throughout training because it avoids
the unstable behavior which occurs during training when
A. Quantitative Evaluation gradient noise or learning rate problems cause sudden
Table II presents the test performance of the performance drops. The system needs stable operation for
proposed Machine Learning-based Network Intrusion its components to function properly in actual deployment
environments.
Detection System evaluated on unseen network traffic
data. The model achieved a Precision of 0.95, Recall The training loss curve shows a complete downward
of 0.96, and an overall Accuracy of 0.97. These results trend which starts from high initial values to reach much
indicate a low rate of false positives and high lower values throughout the duration of the training process
sensitivity in detecting malicious network activities. thus demonstrating the model's strong ability to achieve
The balanced precision and recall values demonstrate training goals through effective learning. The training
the model’s capability to effectively distinguish process demonstrates model stability through its continuous
curve behavior which neither shows any training pattern
between normal and attack traffic without bias.
breaks nor shows the model development stage between
Furthermore, the consistently high performance across
underfitting and overfitting thus enabling the model to
different attack categories highlights the robustness of perform well on new network data.
the system in handling diverse network conditions and
 Fig. 4. Representative prediction outputs on unseen test samples.

Fig 2. Training loss curve of the proposed intrusion detection model. The system detects intrusions correctly under noisy network
conditions which have overlapping traffic patterns. The
The training loss demonstrates strong model convergence model predictions maintained their consistency and
through its continuous drop from initial high values to final reliability because the test samples showed no significant
low values throughout the training period. The training false positive results.
process achieved better results through loss reduction
because it enabled the network to learn network traffic
patterns more effectively while its optimization process D. Dashboard Deployment
operated efficiently. The training process maintained a To check if deployment locally was working out, it
constant decline pattern which demonstrated stable implemented real-time dashboard system.
performance with no overfitting problems, thus enabling
reliable intrusion detection during real-time operations.

Fig. 5. Screenshot of the deployed Streamlit dashboard running local

inference.

Photos may be uploaded at how-ever-high they can be

produced and the function allows for site-determined taking
of user-uploaded photos.

Fig 3. Training loss curve of the proposed intrusion detection model E. Inference Performance
The model shows fast accuracy growth during its first The researchers evaluated the intrusion detection system
training periods which reaches a high level when learning through performance testing to determine its operational
stops. The model demonstrates rapid progress because it can capabilities and suitability for active deployment. The
learn crucial network traffic patterns within a short time. research team executed all experiments in a Python
The model demonstrates good parameter fitting and feature environment while they used standard computing resources
learning abilities, which enables it to detect intrusions to run optimized libraries for maximum efficiency.
through its strong performance in training loss reduction and The system processes incoming network traffic instances
accuracy enhancement. sequentially, where each input is preprocessed and passed
C. Visual Detection Output through the trained model for classification. The system can
The system can correctly identify intrusions through its detect intrusions in real time through its inference process
visual analysis of new network traffic which shows its which has low latency and quick response times that require
capability to operate in different environmental conditions minimal computation power for detection.
while maintaining high accuracy for actual deployment in
operations.
 restrictions found in current intrusion detection systems.

Fig. 6. Inference performance stability graph recorded over 200 continuous

network traffic samples.

The system maintained its regular inference speed Fig. 7. Accuracy comparison of IDS models
throughout its testing of ongoing network traffic samples
which showed effective performance for real-time
applications. The system processing rate operated at The proposed model shows exceptional performance by
constant speed for most samples despite the preprocessing achieving accurate results with fast inference times and low
overhead and execution differences which caused minor resource requirements. The system provides continuous
system fluctuations. The experimental results demonstrate monitoring capabilities which enable it to deliver real-time
that the Machine Learning-based intrusion detection intrusion detection at stable fast prediction speeds. The
framework successfully operates with real-time network system demonstrates practical value which businesses can
traffic under the current system configuration. use to protect their networks from modern security threats.
V. CONCLUSION
F. Baseline Comparison
We compared the proposed model with existing intrusion The researchers developed a machine learning system which
detection approaches by evaluating standard performance detects network intrusions to identify cyber threats in real-
metrics such as accuracy, precision, recall, and F1-score. time through efficient operations. The system uses data
preprocessing and feature transformation together with a
Table 3. Benchmark Accuracy Comparison Between Methods. lightweight classification model to detect all network traffic
types without needing cloud-based systems for external data
Method Accuracy Reference processing. The validation and test results show that the
proposed system enables network environments to achieve
SVM-based IDS 0.91 [1] accurate detection while maintaining operational capability
KNN-based IDS 0.89 [4] in real-world situations.
Logistic Regression IDS 0.88 [5]
Deep Learning IDS 0.94 [12]
The study suffers from two main limitations because it relies
My proposed model on one particular dataset which only allows researchers to do
(Random Forest ) 0.97 -- binary traffic traffic classification. The research will reach
its highest level of improvement when researchers use multi-
class attack detection together with larger datasets and
The Machine Learning-based intrusion detection model advanced deep learning methods. The future research will
showed 97% accuracy which surpassed the performance of focus on three areas which include deploying systems at
traditional methods that included SVM and KNN and scale for real-time security monitoring, developing adaptive
Logistic Regression. The new model demonstrated better learning systems that protect against emerging threats, and
network intrusion detection capabilities because it creating security systems that operate with automated
outperformed existing methods. protective measures. The current digital defense systems will
The experimental findings demonstrate that the model can receive major improvements through these scientific
identify harmful activities through all testing conditions discoveries which will help secure monitoring and
while maintaining steady detection performance. The system protection processes in present-day digital environments.
demonstrates its ability to identify normal and attack traffic The developers created a Machine Learning-based Network
through balanced precision and recall values which remain Intrusion Detection System (NIDS) which operates in real
unaffected by network behavior variations and extraneous time to detect cyber threats. The system uses data
signals The feature learning process captures important preprocessing together with feature transformation and
patterns through its training process because it does not classification model processes to separate normal network
memorize samples from the training data. traffic from malicious traffic.
You can extend your paragraph with these additional lines:
The model demonstrates better performance for handling The system also enhances detection accuracy by reducing
complicated traffic patterns than traditional methods. Real- false positives and improving response time, making it
time inference capability enables systems to operate without suitable for practical deployment in modern networks.
cloud dependence which results in faster processing times Furthermore, integration with real-time alert mechanisms
and less data transmission hazards and stronger system enables quick identification and mitigation of potential
protection. The proposed approach offers practical threats.
deployment capabilities because it resolves multiple existing
VI. References
[1] M. Tavallaee, E. Bagheri, W. Lu, and A. A. Ghorbani, “A
detailed analysis of the KDD CUP 99 dataset,” Proc. IEEE
Symposium on Computational Intelligence for Security and
Defense Applications, 2009.

[2] I. Sharafaldin, A. H. Lashkari, and A. A. Ghorbani,

“Toward generating a new intrusion detection dataset and
intrusion traffic characterization,” Proc. 4th Int. Conf.
Information Systems Security and Privacy, 2018.

[3] W. Lee and S. J. Stolfo, “A framework for constructing

features and models for intrusion detection systems,” ACM
Transactions on Information and System Security, vol. 3, no.
4, pp. 227–261, 2000.

[4] J. Kim, J. Kim, H. L. T. Thu, and H. Kim, “Long short

term memory recurrent neural network classifier for intrusion
detection,” Proc. Int. Conf. Platform Technology and
Service, 2016.

[5] G. Folino, C. Pizzuti, and G. Spezzano, “An ensemble-

based evolutionary framework for coping with distributed
intrusion detection,” Genetic Programming and Evolvable
Machines, vol. 9, pp. 201–233, 2008.

[6] A. L. Buczak and E. Guven, “A survey of data mining and

machine learning methods for cyber security intrusion
detection,” IEEE Communications Surveys & Tutorials, vol.
18, no. 2, pp. 1153–1176, 2016.

[7] Y. Xin et al., “Machine learning and deep learning

methods for cybersecurity,” IEEE Access, vol. 6, pp. 35365–
35381, 2018.

[8] T. T. Nguyen and G. Armitage, “A survey of techniques

for internet traffic classification using machine learning,”
IEEE Communications Surveys & Tutorials, vol. 10, no. 4,
pp. 56–76, 2008.

[9] N. Moustafa and J. Slay, “UNSW-NB15: A

comprehensive data set for network intrusion detection
systems,” Military Communications and Information
Systems Conference, 2015.

[10] R. Sommer and V. Paxson, “Outside the closed world:

On using machine learning for network intrusion detection,”
IEEE Symposium on Security and Privacy, 2010.