Security Machine Learning — Exam Study Guide IT Engineering | All Questions Covered

SECURITY MACHINE LEARNING

Adversarial ML, Anomaly

Detection
& Security Vulnerabilities
IT Engineering | Exam-Focused Study Guide

Covers All 6 Questions — 5 Mark & 10 Mark Format

Feature Details

Security ML Requirements, Real-World Uses, Anomaly Detection vs Supervised

Topics Covered
Learning

More Topics Challenges, Adversarial ML Importance, Security Vulnerabilities in ML

Exam Questions 6 Questions with Full Model Answers

Marks Covered 5-Mark & 10-Mark Answer Formats

Special Sections Exam Tips, Common Mistakes, Quick Revision, Last-Day Sheet, Keywords

■ HOW TO Read each section carefully. The Quick Revision Summaries cover everything you
USE need to recall in the exam hall. Highlighted boxes = guaranteed marks.

VESIT IT Engineering Page 1 © Study Notes — All Concepts Covered

Security Machine Learning — Exam Study Guide IT Engineering | All Questions Covered

■ TABLE OF CONTENTS
1 What's Important for Security ML Systems?

2 Real-World Uses of ML in Security

3 Anomaly Detection vs Supervised Learning

4 Challenges of ML in Anomaly Detection, Response & Mitigation

5 The Importance of Adversarial ML

6 Security Vulnerabilities in ML Algorithms

— EXAM QUESTIONS WITH MODEL ANSWERS (Q1–Q6)

— Last-Day Revision Sheet

— Important Keywords List

— Short Notes (5 Marks) & Long Answer Format (10 Marks)

VESIT IT Engineering Page 2 © Study Notes — All Concepts Covered

Security Machine Learning — Exam Study Guide IT Engineering | All Questions Covered

■ SECTION 1: WHAT'S IMPORTANT FOR SECURITY

ML SYSTEMS?
For ML to serve as a reliable driver of security solutions, the ML
■ Security ML system itself must first be secure, robust, and explainable.
System Requirement Designers must proactively assume systems will be targeted by
adversarial agents.

1.1 Core Requirements at a Glance

Requirement Why it Matters

System must withstand targeted adversarial attacks; not just handle random
Security & Robustness
errors.

Users must understand WHY a model made a decision; "black box" models
Transparency & Explainability
slow investigations.

Low False Positive Rate Frequent false alarms waste analyst time and erode trust in the system.

Adaptability Must adjust to data seasonality and organic changes in traffic patterns.

Adversarial Awareness Must anticipate evasion attacks and model poisoning attempts.

Resource Efficiency Must be deployable in real-time streaming or on embedded devices.

Maintainability Modular design allows easy algorithm swapping as threat landscapes evolve.

1.2 Detailed Breakdown

1.2.1 Security & Robustness

• ML must NOT be treated as a traditional software component — it will be deliberately attacked.
• Unlike standard apps, a single false negative in security ML can cause a crippling breach.
• Designers must move beyond "black box" mentality and proactively build safeguards.

1.2.2 Transparency & Explainability

The difficulty in explaining why an ML model flagged a specific event.
■ Semantic Gap When analysts cannot understand alert reasons, they resist the system
and cannot audit decisions in time-pressured environments.

• Most current models are "black boxes" — decisions cannot be explained.

• Explainable alerts allow human analysts to quickly audit decisions.
• Without explainability → slower incident response → lower adoption.

1.2.3 Low False Positive Rate

• Frequent spurious alerts = alert fatigue → analysts start ignoring all alerts.
• False alarms waste precious analyst time.
• "Crying wolf" effect: operators eventually ignore even real threats.

1.2.4 Adaptability to Seasonality

• Network traffic naturally fluctuates: more activity on weekdays vs weekends.
• A model that fails to adapt → flags normal variation as anomalies.

VESIT IT Engineering Page 3 © Study Notes — All Concepts Covered

Security Machine Learning — Exam Study Guide IT Engineering | All Questions Covered

• Systems must dynamically recalibrate baselines to organic changes.

1.2.5 Adversarial Vulnerability Awareness

Vulnerability Description Example

Model trained on incomplete data → blind spots Hacker finds input that bypasses
Imperfect Learning
(adversarial space) malware detector

Attacker crafts input to cause misclassification Malware modified to look like benign
Evasion Attacks
post-training file

Adversary injects chaff into online learner to shift Spam filter poisoned to accept spam
Model Poisoning
decision boundary as legitimate

1.2.6 Resource Efficiency & Maintainability

• Resource-heavy models may be unsuitable for real-time streaming or IoT/embedded devices.
• Modular design allows swapping algorithms as new threats emerge.
• Goal: minimize false assumptions and remain resilient under stress.

For 10 marks: Cover ALL 7 requirements with explanations. Always define "semantic
■ EXAM
gap" and "model poisoning" — they are high-value keywords. Include the comparison
TIP
table for full structure marks.

■■ COMMON Don't say "just avoid bugs" — security ML failures are often fundamental
MISTAKE algorithmic limitations, not mere implementation bugs.

■ QUICK REVISION SUMMARY

✔ Security & Robustness: Single false negative can cause a full system breach

✔ Transparency: Semantic gap slows incident response; explainable alerts needed

✔ Low False Positives: Alert fatigue causes operators to ignore all alerts

✔ Adaptability: Must handle seasonality (weekday vs weekend traffic patterns)

✔ Adversarial Awareness: Imperfect learning creates blind spots; evasion + poisoning attacks

✔ Efficiency: Must work on streaming and embedded devices

✔ Maintainability: Modular design for easy algorithm updates

VESIT IT Engineering Page 4 © Study Notes — All Concepts Covered

Security Machine Learning — Exam Study Guide IT Engineering | All Questions Covered

■ SECTION 2: REAL-WORLD USES OF ML IN

SECURITY
ML applications in security fall into three broad categories: Pattern Recognition, Anomaly Detection, and
Access Control. These approaches together handle both known threats and novel, zero-day attacks.

2.1 Category Overview

Category Core Idea Examples

Spam detection, malware

Pattern Recognition Identify specific malicious characteristics in data
analysis, fuzzing

Network intrusion, fraud detection,

Anomaly Detection Establish "normality" baseline and flag deviations
host monitoring

Hospital data access, privileged

Access Control Infer flexible access patterns for roles
user monitoring

2.2 Pattern Recognition Applications

A. Spam Detection
The most established real-world use of ML in security. Content-based
■ Spam Detection models + metadata block more than 99.9% of unsolicited messages
using Naive Bayes and Locality-Sensitive Hashing (LSH).

• Techniques: Naive Bayes classification, Locality-Sensitive Hashing (LSH)

• LSH: Groups similar inputs into same hash bucket to quickly identify near-duplicate spam
• Goal: Generalize past spam patterns to predict future spam

B. Malware & Botnet Detection

• Malware uses polymorphism — changing its appearance to avoid signature-based detection.
• ML identifies latent malicious characteristics that persist despite appearance changes.
• Botnets detected by analyzing unusual communication patterns across hosts.

C. Optimized Fuzzing
Testing software for vulnerabilities by providing random/malformed
inputs. ML guides fuzzing campaigns toward vulnerable code paths
■ Fuzzing
by learning from previously discovered flaws — more efficient than
blind random input.

2.3 Anomaly Detection Applications

A. Fraud Detection (Financial)

• Detects fraudulent transactions by comparing against normal spending patterns.
• Flags deviations: large purchases after small ones, purchases from unusual locations.
• Critical for identifying rare fraudulent events in massive legitimate data pools.

B. Network Intrusion Detection (NIDS)

• Compares real-time network traffic against established baselines.

VESIT IT Engineering Page 5 © Study Notes — All Concepts Covered

Security Machine Learning — Exam Study Guide IT Engineering | All Questions Covered

• Detects: Advanced Persistent Threats (APTs), botnets, spyware.

• Focuses on unusual communication patterns, unexpected port usage, abnormal data volumes.

C. Host-Level Security
• Monitors metrics: running processes, user account activity, file access patterns.
• Key detection: Flags anomalous running binaries that have been deleted from disk to hide tracks.
• Establishes per-user behavioral baselines to detect account compromise.

D. Web Application Security

• Deep Packet Inspection (DPI): Identifies SQL injection, XSS attack vectors.
• Analyzes POST body content; compares character frequencies against baseline.
• Web log analysis: Extracts IP-level stats, unusual referrer patterns, out-of-order endpoint access.
• These patterns indicate bot automation or reconnaissance activity.

2.4 Access Control

• Unsupervised learning infers information access patterns for specific organizational roles.
• Goes beyond rigid rule-based policies — allows legitimate but unconventional queries.

Role/Event Typical Pattern ML Action

Broad diagnostic queries across many patients and

Doctors Normal pattern — allowed
conditions

Technicians Narrow, specific queries for assigned tasks Normal pattern — allowed

Anomaly Technician accessing all patient records suddenly Flagged — out-of-role behavior

Emergency Doctor accessing records outside their department Context-aware allowance

Autonomous Vehicles & Adversarial Risk

• ML used for street sign recognition in autonomous vehicles.
• Risk: Adversarial perturbations (small sticker on stop sign) can cause targeted misclassification.
• These systems must be secured against physically realizable adversarial attacks.

For 10 marks on "Real-World Uses": Cover all 3 categories (Pattern Recognition,

■ EXAM
Anomaly Detection, Access Control). Mention specific examples: spam (99.9% block
TIP
rate), NIDS (APTs/botnets), hospital access control.

■■ COMMON Don't confuse Pattern Recognition with Anomaly Detection. Pattern recognition =
MISTAKE known threats; Anomaly detection = unknown/zero-day threats.

■ QUICK REVISION SUMMARY

✔ Pattern Recognition: Spam (Naive Bayes + LSH), Malware (polymorphism), Fuzzing optimization

✔ Anomaly Detection: Fraud detection, Network IDS (APTs/botnets), Host monitoring, Web app (DPI)

✔ Access Control: Unsupervised learning for role-based flexible patterns (hospital example)

✔ Spam detection blocks 99.9% using content-based models + metadata

✔ NIDS detects APTs, botnets, spyware via traffic baseline comparison

✔ Host monitoring flags binaries deleted from disk to hide tracks

VESIT IT Engineering Page 6 © Study Notes — All Concepts Covered

Security Machine Learning — Exam Study Guide IT Engineering | All Questions Covered

■ SECTION 3: ANOMALY DETECTION VS

SUPERVISED LEARNING
The choice between anomaly detection and supervised learning depends primarily on data availability and
predictability of threats being targeted.

3.1 Core Definitions

Extracts patterns strictly from labeled training data to teach an

■ Supervised algorithm to recognize specific threat forms. Also known as "pattern
Learning recognition" approach. Requires both positive and negative labeled
examples.

Establishes a notion of normality from the majority of a dataset (e.g.,

■ Anomaly Detection 95%) and flags deviations as outliers. Can identify an infinite number
of anomalous patterns, including those never seen in training data.

3.2 When to Use Supervised Learning

• Condition: Representative pool of BOTH positive and negative examples is available.

Scenario Condition Example

Future attacks expected to look similar to past Credit card fraud: large purchase after
Predictable Threats
examples small one

Threats have known, repeatable Spam detection, malware signatures,

Established Patterns
characteristics botnet patterns

Explicit or latent characteristics can be Malware detection using file structure

Pattern Recognition
extracted features

3.3 When to Use Anomaly Detection

• Condition: Difficult or impossible to find representative "positive" (malicious) examples.

Scenario Why Anomaly Detection Example

Cannot predict new vulnerabilities in advance;

Zero-Day Attacks New exploit with no prior signature
no positive examples exist

Malicious events are rare vs massive

Class Imbalance legitimate traffic; supervised learning 1 fraud per 10,000 transactions
struggles

No labeled dataset available; unsupervised New enterprise environment with no

Unlabeled Data
learning draws abstractions attack history

3.4 Categories of Anomaly Detection

Category Description Example

Training data is CLEAN (no outliers). Model Baseline trained on verified clean
Novelty Detection
learns pure "normality." network traffic

VESIT IT Engineering Page 7 © Study Notes — All Concepts Covered

Security Machine Learning — Exam Study Guide IT Engineering | All Questions Covered

Training data contains BOTH regular data and

Production logs mixed with some
Outlier Detection outliers. Algorithm must be insensitive to initial
anomalous events
outliers.

Uses regression (supervised) to predict future

Network traffic volume prediction;
Forecasting values from time-series; flags substantial
spike = anomaly
deviations.

■■ SPECIAL Forecasting is unique: it uses regression (a supervised technique) but functions within
CASE anomaly detection to flag deviations from predicted time-series values.

3.5 Direct Comparison Table

Aspect Supervised Learning Anomaly Detection

Labeled Data Required Yes — both +ve and -ve examples No — only normal data needed

No — cannot recognize unseen

Handles Zero-Day Yes — any deviation is flagged
patterns

Excels — works with rare malicious

Class Imbalance Struggles — needs balanced classes
events

Pattern Predictability Best for predictable, repeating threats Best for unknown, novel threats

Training Approach Supervised (Naive Bayes, SVM, LR) Unsupervised or semi-supervised

Spam, known malware, credit card Network intrusions, zero-day exploits,

Security Examples
fraud APTs

Cannot detect threats it has never

Key Limitation Higher false positive rate
seen

ALWAYS explain BOTH approaches and end with the comparison table for full marks.
■ EXAM Mention "Novelty Detection vs Outlier Detection" sub-categories — examiners love this
TIP detail. Key phrase: anomaly detection can identify an INFINITE number of anomalous
patterns.

■■ COMMON Don't say anomaly detection is always better. Supervised learning achieves
MISTAKE HIGHER accuracy when labeled data is available. Each has its place.

■ QUICK REVISION SUMMARY

✔ Supervised = labeled data + known pattern recognition; best for predictable threats

✔ Anomaly Detection = normality baseline + flag deviations; best for zero-day attacks and class
imbalance

✔ 2 categories: Novelty Detection (clean training data) vs Outlier Detection (mixed training data)

✔ Forecasting uses regression but functions as anomaly detection via deviation flagging

✔ Supervised: Spam, credit card fraud, malware. Anomaly: NIDS, APTs, zero-day.

VESIT IT Engineering Page 8 © Study Notes — All Concepts Covered

Security Machine Learning — Exam Study Guide IT Engineering | All Questions Covered

■ SECTION 4: CHALLENGES OF ML IN ANOMALY

DETECTION, RESPONSE & MITIGATION
■ EXAM This section covers 3 sub-topics: (1) Detection Challenges, (2) Response Challenges,
PRIORITY (3) Mitigation Challenges. For 10 marks cover ALL three.

4.1 Detection Challenges

1 High Cost of Errors False Negative: Missing one anomaly = crippling

Unlike a shopping site where wrong recommendations system breach. False Positive: Spurious alerts
have low cost, security errors are catastrophic. degrade system integrity → analyst alert fatigue.

2 Semantic Gap (No Explainability) One-class SVM or neural network flags traffic —
Analysts cannot understand WHY an event was flagged; analyst cannot explain or debug the alert in
"black box" models cannot be audited under time real-time.
pressure.

3 Evaluation Difficulty No test set can cover the "vast possibilities of

Anomaly detection involves data never seen before — different anomalies" in the wild.
impossible to comprehensively test against all future
anomalies.

4 Data Seasonality & Organic Drift Monday morning spike flagged as anomaly
Regular patterns (weekends = lower traffic) cause false because model was trained only on stable
positives if model does not adapt. New user flows also weekday data.
trigger alarms.

5 Adversarial Risks & Poisoning "Boiling frog": poison injected so gradually that
Attackers actively manipulate ML systems using model system accepts it as organic drift — malicious
poisoning and "boiling frog" attacks. traffic becomes "normal."

6 Training Data Pollution Existing intrusions in training logs cause model to

Cleaning training data of ALL anomalies is "laborious and treat attack traffic as normal.
sometimes downright impossible." Outliers in training
data corrupt the learned baseline.

7 Configuration & Maintenance ML-based IDS with 50+ tunable parameters —

Too many parameters → users revert to default values → team sets all to defaults → 60% accuracy instead
poor performance. Constant retraining needed as threat of 95%.
landscape evolves.

4.2 Response Challenges

Human-in-the-Loop Because false negatives are so costly, fully Analyst must confirm "potential
Necessity automated end-to-end detection+response is APT" alert before blocking IPs.
rare. A human must verify before action.

Alert Fatigue & Organizations face massive volumes from SIEM aggregates 50,000
Fragmented Data fragmented security systems. SIEM platforms alerts/day from NIDS, antivirus,
aggregate alerts but add infrastructure WAF — analyst only reviews top
complexity. 100.

VESIT IT Engineering Page 9 © Study Notes — All Concepts Covered

Security Machine Learning — Exam Study Guide IT Engineering | All Questions Covered

Manual Forensics Incident response remains "stubbornly manual." ML identifies unusual lateral
ML can mine patterns but cannot substitute for movement — human must deduce
human reasoning about attacker intentions. if it is APT reconnaissance or
insider threat.

4.3 Mitigation Challenges

Feedback Loop Problem Immediately banning attacker IP reveals IP banned → attacker rotates to
detection to attacker, who iterates until they new IP and refines attack method.
evade it.

Strategic vs Immediate Defenders must choose: block threat now OR Let APT continue under
Action observe attacker to gather intelligence on observation to map out entire
capabilities/origin. botnet infrastructure before taking
it down.

Stealth Banning (Shadow Attacker's actions appear valid to them but have Spammer still "sends emails" but
Banning) no real effect. Buys time without alerting attacker. emails never actually delivered to
Technically complex to implement. anyone.

For 10 marks: Cover ALL 3 sub-topics (Detection + Response + Mitigation). Define

■ EXAM
"boiling frog attack," "shadow banning," and "semantic gap" explicitly. Mention SIEM for
TIP
response challenges.

■■ COMMON Many students forget Response and Mitigation challenges and only write
MISTAKE Detection. This loses 4+ marks in a 10-mark question.

■ QUICK REVISION SUMMARY

✔ 7 Detection Challenges: High error cost, Semantic gap, Evaluation difficulty, Seasonality, Adversarial
poisoning, Data pollution, Configuration

✔ 3 Response Challenges: Human-in-loop needed, Alert fatigue (SIEM), Manual forensics

✔ 3 Mitigation Challenges: Feedback loop (IP ban reveals detection), Strategic vs immediate, Shadow
banning

✔ Boiling frog: gradual poisoning that mimics organic drift

✔ Shadow banning: attacker thinks actions work but have no real effect

✔ SIEM: Security Information and Event Management — aggregates fragmented alerts

VESIT IT Engineering Page 10 © Study Notes — All Concepts Covered

Security Machine Learning — Exam Study Guide IT Engineering | All Questions Covered

■ SECTION 5: THE IMPORTANCE OF ADVERSARIAL

ML
The study of machine learning vulnerabilities in adversarial
■ Adversarial
environments. It focuses on understanding, attacking, and defending
Machine Learning
ML systems against intentional manipulation by malicious actors.

5.1 Why Adversarial ML Matters — 7 Key Reasons

#1 Protection of Critical Infrastructure Example: Antivirus engine fooled by

ML powers antivirus engines, spam filters, NIDS, and sentiment adversarial malware = full enterprise
analyzers. When national sovereignty, human lives, or critical compromise.
systems are at stake, designers have a professional duty to
preemptively safeguard against attacks.

#2 Closing the Confidence Gap Example: 75% of ML security researchers

A significant gap exists between expectations for AI-driven say existing models are inadequate.
security and current reality. 3 out of 4 researchers feel today's
solutions are too easy for adversaries to bypass. Adversarial ML
is the key to closing this gap.

#3 Overcoming Inherent Model Vulnerabilities Example: Even a 99.9% accurate model

Vulnerabilities are FUNDAMENTAL, not just bugs: Imperfect has a finite set of adversarial inputs that
learning creates adversarial blind spots; Bayes error rate means fool it.
a perfect learner still has exploitable adversarial samples.

#4 Challenging Broken Assumptions Example: Spammer changes email

Most ML assumes data stationarity — distributions do not patterns to evade filter trained on last
change over time. Adaptive adversaries intentionally shatter this month's spam.
assumption, making it vital to study ML under real-world
adversarial stress.

#5 Addressing Explainability Example: Analyst cannot tell if NIDS false

Black-box models offer no assurance of robustness. Without positive is a bug or a poisoning attack.
explainability, it is impossible to detect when a malicious actor
has influenced the model, leading to resistance in using ML for
critical roles.

#6 Managing Attack Transferability Example: Adversarial image fools

Adversarial samples designed to fool one model often fool ResNet-50 AND VGG-16 despite different
OTHER models, even those using different algorithms. This architectures.
enables "black-box attacks": attacker trains a local substitute
model to find adversarial samples that transfer to the real target
system.

#7 Proactive System Design Example: Design principle: assume

A prerequisite for ML-driven security: the ML itself must be adversary knows your model and will
secure. Adversarial ML teaches designers to proactively expect actively attack it.
systems to misbehave under malicious conditions, moving from
false assumptions to resilient architecture.

5.2 Key Concepts in Adversarial ML

VESIT IT Engineering Page 11 © Study Notes — All Concepts Covered

Security Machine Learning — Exam Study Guide IT Engineering | All Questions Covered

The "blind spots" in a model's learned representation — regions of

input space where the model has no training examples and makes
■ Adversarial Space
wrong predictions. Created by imperfect learning on incomplete
training distributions.

The theoretical lower bound of error for a given classifier and feature
■ Bayes Error Rate set. Even a "perfect" learner cannot achieve zero error — a finite set of
adversarial samples will ALWAYS exist to fool any classifier.

Adversarial examples crafted for one model successfully fool other

■ Attack independently trained models. Enables black-box attacks: attacker
Transferability creates a local substitute model to find adversarial samples, then
attacks the real (secret) target system.

List ALL 7 reasons with brief explanations for 10 marks. Define Bayes Error Rate and
■ EXAM
Attack Transferability explicitly — these are guaranteed exam keywords. Mention "3 out
TIP
of 4 researchers" stat.

Don't just say "adversarial ML is important for security." You must explain WHY
■■ COMMON
with specific reasons — imperfect learning, Bayes error, transferability, broken
MISTAKE
assumptions, etc.

■ QUICK REVISION SUMMARY

✔ 7 Reasons: Critical infrastructure, Confidence gap (75% researchers), Inherent vulnerabilities (Bayes
error), Broken assumptions, Explainability, Transferability, Proactive design

✔ Adversarial space = blind spots in model from incomplete training data

✔ Bayes error rate = perfect learner still has finite adversarial samples

✔ Transferability = one adversarial sample can fool multiple different models

✔ Black-box attack = attacker builds substitute model to find transferable adversarial samples

✔ Data stationarity assumption is SHATTERED by adaptive adversaries in security

VESIT IT Engineering Page 12 © Study Notes — All Concepts Covered

Security Machine Learning — Exam Study Guide IT Engineering | All Questions Covered

■ SECTION 6: SECURITY VULNERABILITIES IN ML

ALGORITHMS
Security vulnerabilities in ML are not merely implementation flaws — they arise from fundamental algorithmic
limitations and the statistical nature of learning itself. Understanding these is essential for building resilient
systems.

6.1 Primary Vulnerability Classes

1. Imperfect Learning & Adversarial Space

An algorithm can never be provided a dataset drawn from the ENTIRE
theoretical distribution (e.g., every possible malware variant). This
■ Imperfect Learning
creates a discrepancy between training data and reality, resulting in
adversarial space (blind spots) that attackers exploit.

• Model trained on 10,000 malware samples cannot know about the 10,001st variant.
• Blind spots allow crafting of adversarial samples that cause intentional misclassification.
• The more incomplete the training distribution, the larger the adversarial space.

2. Bayes Error Rate

The lower bound of possible error for a specific combination of
classifier + features. Even a theoretically perfect learner with complete
■ Bayes Error Rate
data representation has non-zero Bayes error. Consequence: a finite
set of adversarial samples will ALWAYS exist.

• This is a THEORETICAL fundamental limit — not fixable by better implementation.

• No classifier can achieve 100% security against all adversarial inputs.

3. Broken Assumptions (Non-Stationarity)

• Data Stationarity Assumption: ML assumes training and test distributions are the same.
• In security: adversaries intentionally change their behavior to shatter this assumption.
• Feature Independence Assumption: Often violated in real-world correlated features.
• Result: Models fitted to historical attack patterns fail against evolved attack styles.

4. Attack Transferability
• Adversarial samples crafted for Model A often fool Model B (different algorithm, different training).
• Black-box attack: Attacker queries target system → builds local substitute model → finds adversarial
samples → transfers to target.
• Attacker does NOT need internal knowledge of the target model's parameters.

6.2 Core Attack Techniques

Attack Type Mechanism Example / Variant Risk Level

VESIT IT Engineering Page 13 © Study Notes — All Concepts Covered

Security Machine Learning — Exam Study Guide IT Engineering | All Questions Covered

Model Poisoning Targets online learners. Attacker "Boiling Frog" variant: poisoning High —
(Causative Attack) injects "chaff" (synthetic malicious so gradual it mimics organic data corrupts
traffic) to gradually shift decision drift — system accepts new training
boundaries. "normal." data;
permanent
effect until
retraining.

Evasion Attack Post-training only. Attacker finds Malware file modified with Very High
(Exploratory Attack) inputs in adversarial space that cause benign-looking strings to evade — can
misclassification. No influence over antivirus while maintaining target ANY
training phase. malicious functionality. classifier
regardless
of training
phase.

6.3 Taxonomy of Security Violations

Reduces the true positive rate — malicious actions pass through

■ Integrity Attack detectors unnoticed. Attacker's goal: make malicious traffic look
legitimate.

Degrades the overall usability of the system by simultaneously

reducing true positives AND increasing false positives — making the
■ Availability Attack
system's output completely unreliable. Attacker's goal: make the
security system useless.

Attack Type Effect on Metrics Attacker Method Impact

Reduces True Positive Malicious traffic misclassified System passes threats

Integrity Attack
Rate (TPR) as benign undetected

System becomes
Reduces TPR + Increases Flood system with borderline
Availability Attack completely unreliable;
FPR inputs
operators abandon it

6.4 Vulnerability Summary Diagram

VESIT IT Engineering Page 14 © Study Notes — All Concepts Covered

Security Machine Learning — Exam Study Guide IT Engineering | All Questions Covered

ML ALGORITHM VULNERABILITIES

IMPERFECT LEARNING BAYES ERROR RATE BROKEN ASSUMPTIONS

(Adversarial blind spots) (Theoretical min. error) (Non-stationarity)

ATTACK METHODS

Model Poisoning (Causative) Evasion Attacks (Exploratory)

SECURITY VIOLATIONS

Integrity Attacks (pass threats) Availability Attacks (degrade system)

Cover ALL 4 vulnerability classes + 2 attack types + 2 violation types for 10 marks. Define
■ EXAM
"adversarial space," "Bayes error rate," "model poisoning," and "evasion attack" explicitly.
TIP
Draw the taxonomy diagram.

■■ COMMON Don't say evasion attacks only happen during training. Evasion attacks are
MISTAKE POST-TRAINING — they exploit the deployed model, not the training process.

■ QUICK REVISION SUMMARY

✔ 4 Vulnerability Classes: Imperfect learning (blind spots), Bayes error, Broken assumptions,
Transferability

✔ 2 Attack Types: Model Poisoning = causative (during training); Evasion = exploratory (post-training)

✔ Boiling frog = gradual model poisoning that mimics organic drift

✔ 2 Violation Types: Integrity (reduces TPR, passes threats) vs Availability (reduces TPR + raises FPR,
degrades system)

✔ Black-box attack = build substitute model → find transferable adversarial samples → attack real target

✔ Fundamental: Bayes error means NO classifier can be fully foolproof

VESIT IT Engineering Page 15 © Study Notes — All Concepts Covered

Security Machine Learning — Exam Study Guide IT Engineering | All Questions Covered

✍■ EXAM QUESTIONS — MODEL ANSWERS

■ HOW TO All 6 exam questions answered below in exam-ready format. Structure = Introduction
USE + Detailed Body + Examples/Table + Conclusion.

Q1. What's Important for Security Machine Learning Systems? [10 Marks]

Introduction: For ML to serve as a reliable driver of security solutions, the ML system itself must first be secure,
robust, and explainable. Unlike traditional software, security ML systems will be deliberately attacked by
adversaries.

Requirement Explanation

ML must withstand deliberate adversarial attacks. A single false negative can

Security & Robustness
cause a crippling breach.

Avoid "semantic gap" — analysts must understand WHY an alert was triggered
Transparency & Explainability
to investigate effectively.

Spurious alerts cause alert fatigue → operators ignore all alerts → system
Low False Positive Rate
integrity degrades.

Must adjust to data seasonality (weekday vs weekend traffic) and organic

Adaptability
changes.

Imperfect learning creates blind spots; design must anticipate evasion attacks
Adversarial Awareness
and model poisoning.

Resource Efficiency Must support real-time streaming and deployment on embedded/IoT devices.

Maintainability Modular design allows algorithm swapping as threat landscape evolves.

Conclusion: A successful security ML system minimizes false assumptions, remains resilient under stress, and
bridges the gap between current flawed reality and AI-driven security expectations.

Q2. Real-World Uses of Machine Learning in Security. [10 Marks]

Introduction: ML applications in security are broadly categorized into Pattern Recognition (identifying known
threat characteristics) and Anomaly Detection (flagging deviations from established normal baselines).
Together, they address both known and zero-day threats.

Application How ML is Used

Naive Bayes + LSH — blocks 99.9% of unsolicited messages using content

Spam Detection
features.

Malware Detection Identifies latent malicious characteristics despite polymorphic appearance changes.

Fuzzing Optimization Learns from past vulnerabilities to guide fuzzing toward vulnerable code paths.

Fraud Detection Flags deviations from normal spending patterns in financial transactions.

Network IDS Baseline comparison detects APTs, botnets, spyware via traffic anomalies.

Host-Level Monitoring Flags anomalous processes and binaries deleted from disk to hide tracks.

Web App Security DPI detects SQL injection, XSS; web log analysis identifies bot activity.

VESIT IT Engineering Page 16 © Study Notes — All Concepts Covered

Security Machine Learning — Exam Study Guide IT Engineering | All Questions Covered

Unsupervised learning infers role-based access patterns (hospital doctor vs

Access Control
technician).

Autonomous Vehicles Street sign recognition — must be secured against adversarial perturbations.

Conclusion: ML applications in security span from well-established pattern recognition (spam, malware) to
more complex anomaly detection and behavioral analysis, enabling adaptive defense against a constantly
evolving threat landscape.

Q3. Anomaly Detection vs Supervised Learning. [10 Marks]

Introduction: The choice between anomaly detection and supervised learning depends on data availability and
threat predictability.

Supervised Learning
• Definition: Extracts patterns from labeled data to recognize specific threat forms.
• Best for: Predictable threats, established patterns, balanced labeled datasets.
• Examples: Spam detection (Naive Bayes), credit card fraud, botnet detection.

Anomaly Detection
• Definition: Establishes normality baseline; flags deviations as suspicious.
• Best for: Zero-day attacks, class imbalance, unlabeled data scenarios.
• Categories: (1) Novelty Detection — clean training data; (2) Outlier Detection — mixed data; (3) Forecasting
— regression-based time-series.

Aspect Supervised Anomaly Detection

Labeled Data Yes — both +ve and -ve No — normal data only

Zero-Day Threats Cannot handle Handles well

Class Imbalance Struggles Excels

Security Examples Spam, known malware NIDS, APTs, zero-day

Conclusion: Both approaches are complementary — supervised for known threats, anomaly detection for novel
threats. Most real-world systems combine both.

Q4. Challenges of Using ML in Anomaly Detection, Response, and Mitigation. [10 Marks]

Introduction: Using ML for anomaly detection in security presents unique challenges across three dimensions:
detection, response, and mitigation.

Detection Challenges (7):

Challenge Impact

High Error Cost False negative = breach; False positive = alert fatigue

Semantic Gap Black-box models cannot be audited; analysts lose confidence

Evaluation Difficulty Cannot test against all possible future anomalies

Data Seasonality Natural patterns (weekends) cause false positives if unaddressed

Adversarial Poisoning Boiling frog attacks gradually shift decision boundaries

VESIT IT Engineering Page 17 © Study Notes — All Concepts Covered

Security Machine Learning — Exam Study Guide IT Engineering | All Questions Covered

Training Data Pollution Existing anomalies in training data corrupt learned baseline

Configuration Burden Too many parameters → defaults → poor performance

Response Challenges (3):

• Human-in-loop: Fully automated end-to-end response is rare; human verification needed.
• Alert Fatigue: SIEM platforms aggregate fragmented alerts but add infrastructure complexity.
• Manual Forensics: ML cannot substitute for human reasoning about attacker intent.

Mitigation Challenges (3):

• Feedback Loop: Immediate banning reveals detection mechanism; attacker iterates.
• Strategic Choice: Block now vs observe to gather intelligence.
• Shadow Banning: Technically complex but prevents attacker awareness of detection.
Conclusion: These challenges explain why fully automated security ML remains aspirational. Human oversight,
modular design, and adversarially aware architectures are essential.

Q5. The Importance of Adversarial ML. [10 Marks]

Introduction: Adversarial Machine Learning studies ML vulnerabilities in hostile environments. It is critical

because ML systems in security will be deliberately and systematically attacked.

Reason Explanation

1.
Critical I
ML powers antivirus, spam filters, NIDS — stakes include national security and human lives.
nfrastru
cture

2. Confi
3 in 4 researchers say current ML security solutions are too easy to bypass; adversarial ML closes this
dence
gap.
Gap

3.
Inherent
Imperfect learning + non-zero Bayes error rate = adversarial samples ALWAYS exist.
Vulnera
bilities

4.
Broken
Adversaries intentionally violate data stationarity assumption to evade detection.
Assump
tions

5. Expla
Without transparency, cannot detect when a model has been poisoned or influenced.
inability

6. Trans
ferabilit Adversarial samples transfer across models — enables black-box attacks on secret systems.
y

7. Proa
ctive Teach designers to EXPECT misbehavior under adversarial conditions; build resilience from the start.
Design

Conclusion: Adversarial ML is not optional — it is a prerequisite for any serious deployment of ML in

security-critical applications. Security without adversarial robustness is security theater.

VESIT IT Engineering Page 18 © Study Notes — All Concepts Covered

Security Machine Learning — Exam Study Guide IT Engineering | All Questions Covered

Q6. Security Vulnerabilities in Machine Learning Algorithms. [10 Marks]

Introduction: Security vulnerabilities in ML arise from fundamental algorithmic limitations rather than just
implementation flaws. Understanding these is essential for building resilient ML-based security systems.

Vulnerability Explanation Example

Training data never covers entire theoretical Malware variant not in training set
Imperfect Learning
distribution → adversarial blind spots. bypasses antivirus.

Theoretical lower bound of error — even perfect 1-in-10,000 adversarial input always
Bayes Error Rate
learner has finite adversarial samples. exists regardless of accuracy.

Data stationarity & feature independence Spammer changes email format to

Broken Assumptions
assumptions violated by adaptive adversaries. evade trained filter.

Adversarial samples fool multiple models → Adversarial image fools different

Transferability
black-box attacks possible. neural network architectures.

Attack Techniques:
Attack Mechanism Example

Boiling frog: gradual

Causative — injects chaff to shift decision boundaries during
Model Poisoning poisoning mimics organic
online learning.
drift.

Exploratory — finds adversarial samples post-training; works Crafted malware file

Evasion Attacks
on any deployed classifier. classified as benign.

Security Violation Taxonomy:

• Integrity Attack: Reduces true positive rate — malicious actions pass unnoticed.
• Availability Attack: Reduces TPR + raises FPR — entire system becomes unreliable.
Conclusion: These vulnerabilities are structural, not incidental. Robust ML security systems must be designed
with adversarial robustness as a core requirement, not an afterthought.

VESIT IT Engineering Page 19 © Study Notes — All Concepts Covered

Security Machine Learning — Exam Study Guide IT Engineering | All Questions Covered

■ LAST-DAY REVISION SHEET — SECURITY ML

■ Security ML Requirements

✔ 7 Requirements: Security, Explainability (semantic gap), Low FPR (alert fatigue), Adaptability
(seasonality), Adversarial awareness, Efficiency, Maintainability

✔ Semantic Gap = analysts cannot understand black-box decisions → slow incident response

✔ Model Poisoning = chaff injection to shift online learner decision boundaries

■ Real-World ML Uses

✔ Pattern Recognition: Spam (Naive Bayes + LSH, 99.9% block), Malware (polymorphism), Fuzzing

✔ Anomaly Detection: Fraud, Network IDS (APTs/botnets), Host monitoring (deleted binaries), Web
(DPI, SQL injection)

✔ Access Control: Unsupervised learning for role-based patterns (hospital doctor vs technician)

■ Anomaly Detection vs Supervised Learning

✔ Supervised: labeled data, known threats, Naive Bayes/SVM/LR — best for spam, credit fraud

✔ Anomaly Detection: normality baseline, zero-day/class imbalance — best for NIDS, APTs

✔ 3 Types: Novelty (clean data), Outlier (mixed data), Forecasting (regression + deviation flagging)

■ Challenges (Detection + Response + Mitigation)

✔ 7 Detection: Error cost, Semantic gap, Evaluation, Seasonality, Poisoning (boiling frog), Data pollution,
Config

✔ 3 Response: Human-in-loop, Alert fatigue (SIEM), Manual forensics

✔ 3 Mitigation: Feedback loop, Strategic vs immediate, Shadow banning

■ Adversarial ML Importance

✔ 7 Reasons: Critical infra, Confidence gap (75% researchers), Inherent vulnerabilities (Bayes error),
Broken assumptions, Explainability, Transferability, Proactive design

✔ Bayes Error = perfect learner still has finite adversarial samples

✔ Transferability enables black-box attacks via substitute model

■ Security Vulnerabilities in ML

✔ 4 Vulnerabilities: Imperfect learning, Bayes error, Broken assumptions, Transferability

✔ 2 Attacks: Model Poisoning (causative, during training) vs Evasion (exploratory, post-training)

✔ 2 Violations: Integrity (passes threats, reduces TPR) vs Availability (degrades system, raises FPR)

VESIT IT Engineering Page 20 © Study Notes — All Concepts Covered

Security Machine Learning — Exam Study Guide IT Engineering | All Questions Covered

■ IMPORTANT KEYWORDS LIST

Adversarial ML Study of ML vulnerabilities in adversarial/hostile environments

Adversarial Space Blind spots in a model — regions not covered by training distribution

Bayes Error Rate Theoretical lower bound of error; even perfect learner has finite adversarial samples

Semantic Gap Difficulty explaining why an ML model flagged an event; causes analyst resistance

Model Poisoning Causative attack: injects chaff into online learner to shift decision boundaries

Boiling Frog Attack Gradual model poisoning that mimics organic drift to avoid tripwires

Exploratory post-training attack: crafts inputs in adversarial space for

Evasion Attack
misclassification

Adversarial samples designed for one model fool other independently trained
Transferability
models

Black-box Attack Builds substitute model to find adversarial samples; transfers to secret target model

Integrity Attack Reduces true positive rate — malicious events pass undetected

Availability Attack Reduces TPR + increases FPR — makes entire security system unreliable

Alert Fatigue When too many false positives cause operators to ignore all alerts

Security Information and Event Management — aggregates fragmented security

SIEM
alerts

Shadow Banning Stealth banning: attacker's actions appear valid to them but have no real effect

Novelty Detection Anomaly detection trained on clean dataset with no outliers

Outlier Detection Anomaly detection trained on mixed dataset containing both normal and outlier data

Online Learner ML model that adapts in real-time to new incoming data/feedback

Organic Drift Legitimate natural changes in data patterns over time

Polymorphism Malware technique: changes its appearance/code to avoid signature detection

Deep Packet Inspection Analyzes full packet content (not just headers) to detect attack payloads

Locality-Sensitive Hashing — groups similar inputs into same hash for fast spam
LSH
detection

APT Advanced Persistent Threat — sophisticated, long-term targeted attack

Imperfect Learning Model trained on incomplete distribution — creates adversarial blind spots

Data Stationarity Assumption that data distribution does not change over time — violated in security

Feature Independence Assumption that input features are uncorrelated — often violated in practice

False Negative (FN) Malicious event classified as benign — the worst outcome in security ML

False Positive (FP) Benign event classified as malicious — causes alert fatigue

Chaff Synthetic malicious traffic injected to manipulate an online learner

Fuzzing Testing software by providing unexpected/malformed inputs to find vulnerabilities

Class Imbalance When one class (malicious) is much rarer than the other (benign) in training data

VESIT IT Engineering Page 21 © Study Notes — All Concepts Covered

Security Machine Learning — Exam Study Guide IT Engineering | All Questions Covered

■ SHORT NOTES (5 MARKS) & LONG FORMAT (10

MARKS)
5 Marks: 1 intro + 4-5 points + conclusion (~150-200 words). 10 Marks: intro +
✍■ FORMAT
sub-sections + table/diagram + examples + conclusion (~400-500 words).

■ SHORT NOTE: Adversarial ML — Importance

• Adversarial ML studies ML vulnerabilities in hostile/adversarial environments.
• Needed because: (1) ML powers critical infrastructure; (2) 75% researchers say current solutions are
bypassable.
• Fundamental limits: Imperfect learning creates adversarial blind spots; Bayes error means perfect learners
still have adversarial samples.
• Adversaries break stationarity assumption — models trained on past data fail against evolved threats.
• Attack transferability: adversarial samples fool multiple different models, enabling black-box attacks.

■ SHORT NOTE: Model Poisoning vs Evasion Attacks

• Both are attack techniques against ML security systems.
• Model Poisoning (Causative): targets online learners; injects chaff to shift decision boundaries during training.
• "Boiling frog" variant: gradual poisoning that mimics organic drift — system accepts new malicious normal.
• Evasion Attacks (Exploratory): post-training; attacker finds inputs in adversarial space; works on ANY
deployed classifier.
• Key difference: Poisoning corrupts the model itself; Evasion exploits the already-trained model.

■ SHORT NOTE: Challenges of Anomaly Detection

• High error cost: False negative = breach; False positive = alert fatigue.
• Semantic gap: black-box models cannot be audited; analysts cannot understand alert triggers.
• Evaluation difficulty: impossible to test against all possible future anomalies.
• Data seasonality: natural patterns (weekday vs weekend) cause spurious alarms if unaddressed.
• Adversarial risks: model poisoning + boiling frog attacks deliberately subvert the learning process.

Long Answer (10 Marks) Template

Section What to Write Marks

Introduction (2-3 lines) Define main term + state what you will explain 1-2 marks

Point 1 with Sub-heading First major aspect + 3-4 bullet points + example 2 marks

Point 2 with Sub-heading Second major aspect + 3-4 bullet points + example 2 marks

Point 3 (if applicable) Third aspect, challenge, or classification 2 marks

Comparison Table Side-by-side comparison (at least 4-5 rows) 1-2 marks

Diagram/Flowchart Process flow, taxonomy, or architecture sketch 1 mark

Conclusion Summarize + real-world application/implication 0.5-1 mark

VESIT IT Engineering Page 22 © Study Notes — All Concepts Covered

Security Machine Learning — Exam Study Guide IT Engineering | All Questions Covered

■ ALL THE BEST FOR YOUR

EXAMS! — SECURITY ML FULLY
COVERED

VESIT IT Engineering Page 23 © Study Notes — All Concepts Covered