CB3591 Engineering Secure Software Systems

Unit I Need of Software Security and Low-Level Attacks

Software Assurance and Software Security - Threats to software security - Sources of
software insecurity - Benefits of Detecting Software Security - Properties of Secure
Software – Memory-Based Attacks: Low-Level Attacks Against Heap and Stack -
Defense Against Memory-Based Attacks

Need for Secure Software

 Today, people and businesses rely a lot on internet-based software to handle
important and private information.
 This makes them more open to hackers, criminals, and online attacks.
 Many of these problems happen because the software is not built carefully, and
mistakes in the code make it easy for attackers to break in.
 As software keeps getting bigger and more complex, each update can bring new
risks.
 Since online attacks are becoming smarter and more common, it’s important to build
software in a safer and more careful way to keep data and systems protected.

Definition: “Engineering Secure Software Systems is the systematic application of

engineering processes, tools, and techniques to create software that is resistant to security
threats and complies with security standards from design to deployment.”
Engineering Processes: Refers to a structured approach to problem-solving and system
development using established engineering principles and methodologies.
Software Engineering Principles: Fundamental guidelines that help developers design,
build, and maintain high-quality software.

SOFTWARE AND SOFTWARE SECURITY

1. Software: Collection of programs, procedures, and associated documentation that
perform specific tasks on a computer system. It tells the hardware what to do, how to do it,
and when to do it.
Types of Software:
I. System Software – Controls and manages computer hardware.
Examples: Operating systems (Windows, Linux), device drivers.
II. Application Software – Helps users perform specific tasks.
Examples: MS Word, Photoshop, browsers.
III. Middleware – Acts as a bridge between applications or between application and
OS. Ex: Windows frontend application sends and receives data from a Linux
backend server.
IV. Programming Software – Tools for developers to write programs.
Examples: Compilers, IDEs (Eclipse, Visual Studio).

Characteristics of Good Software:

1|Page
 - Correctness – Performs required functions accurately.
- Efficiency – Uses system resources optimally.
- Reliability – Works consistently under defined conditions.
- Usability – Easy to learn and operate.
- Maintainability – Can be updated and fixed easily.
- Portability – Can run on different hardware/OS environments.

2. Software Security: Software security is the practice of designing, developing, and

testing software to withstand malicious attacks and prevent unauthorized access, misuse,
or damage. It is not just about fixing vulnerabilities after development — it’s about
integrating security throughout the Software Development Life Cycle (SDLC).
[Structured process that is used to design, develop, and test high-quality software.]
3. System: Security is not just about the software itself, but also the entire environment it
operates within. It encompasses the software, the hardware it runs on, the network it
connects to, and even the people who use it.

Why Secure Software Systems Important?

1. Protects Sensitive Data
 Software often handles personal, financial, or confidential information. Security
ensures this data isn’t stolen, misused, or leaked.
 Example: Online banking apps protect account details from hackers.
2. Prevents Cyber attacks
 Weak or insecure software can be exploited by attackers to install malware, steal
resources, or cause system failures.
 Secure design reduces vulnerabilities (like buffer overflows or SQL injections).
3. Ensures System Reliability
 Secure systems are dependable even under malicious attempts or unexpected
conditions.
 Example: An insecure hospital system crash could delay critical medical care.
4. Maintains User Trust
 Users only rely on software if they trust it to safeguard their privacy and work as
intended.
 Breaches damage reputation, lead to financial loss, and erode confidence.
5. Compliance and Legal Requirements
 Many industries (finance, healthcare, government) have strict regulations (e.g.,
GDPR, HIPAA).
 Secure software is essential for meeting these standards and avoiding penalties.
6. Cost Efficiency
 Fixing vulnerabilities during development is far cheaper than handling breaches
after deployment.
 Security-by-design reduces long-term maintenance and patching costs.
7. National and Organizational Security
 Critical infrastructure (power grids, defense, transport) runs on software.
Insecure systems can become entry points for cyberwarfare or terrorism.

SOFTWARE ASSURANCE AND SOFTWARE SECURITY

2|Page
Software assurance is about making sure that software keeps working properly, even when
something tries to break it, whether by accident or on purpose. Unfortunately, software
often has weaknesses—caused by mistakes in coding, old programs, or unsafe components
from outside sources—that hackers can find and exploit.

“The level of confidence that software is free from vulnerabilities, either intentionally
designed into the software or accidentally inserted at any time during its life cycle, and that
the software functions in the intended manner.”

Objective: The ability to trust that software will remain dependable under all circumstances,
with a justified level of confidence.

Why Software Assurance is Critical

 Software is the weakest link – If software has flaws, the whole system is at risk.
Example: A bug in Windows allowed WannaCry ransomware to spread worldwide.
 Too complex to fully test – Large, complicated software hides issues. Example: Boeing
737 MAX software errors were not caught in testing, leading to disasters.
 Supply-chain risks – Outsourced or third-party code may contain hidden threats.
Example: The SolarWinds hack spread through trusted software updates.
 Smarter cyberattacks – Hackers use stealthy, advanced techniques. Example: Stuxnet
malware secretly damaged Iran’s nuclear systems.
 Old software reuse – Legacy code in new systems creates vulnerabilities. Example: Old
Windows XP systems were exploited in hospital cyberattacks.
 Low security investment – Companies often underfund security. Example: Equifax did
not patch a known flaw, leading to 147 million data breaches.

Software assurance includes the disciplines of

 Software reliability,
 Software safety, and
 Software security.

3|Page
Software security is a key part of software assurance. It focuses on building strong, reliable
programs that can resist attacks, limit damage if something goes wrong, and recover
quickly. Secure software should work as expected, be as free from exploitable flaws as
possible, and follow clear standards.

The main objective of software security is to build more-robust, higher quality,

defect-free software that continues to function correctly under malicious attack.

In practice, this means designing software to spot harmful inputs, recognize attack patterns,
and respond in a way that protects the system. If an attack does succeed, the software
should still provide at least a basic level of service and then fully recover. Achieving this
requires balancing strong security with time, cost, and project constraints.

Software assurance improves software security by identifying and mitigating vulnerabilities

at every stage of the development process, from initial design to deployment and
maintenance. This is fundamentally different from traditional security practices that often
focus on protecting the network and infrastructure the software runs on. SwA instead
focuses on the software itself, aiming to build security in from the start.

Shifts Left on Security: Software assurance "shifts left" on the security timeline, meaning
it moves security considerations to the earliest possible stages of development. It's much
cheaper and easier to fix a design flaw than to patch vulnerability in a deployed system.
Proactive Vulnerability Management: Rather than just reacting to discovered
vulnerabilities, SwA uses a range of techniques to proactively find them. This includes:
Threat Modeling: Analyzing the system's design to identify potential threats and
vulnerabilities before any code is written. This helps developers and architects build
security controls from the beginning.
Static and Dynamic Analysis: Using automated tools to scan source code (static analysis)
and running applications (dynamic analysis) to find common weaknesses, like buffer
overflows, injection flaws, and insecure configurations.
Code Reviews and Peer Inspections: Developers manually review each other's code to
spot logical errors and security flaws that automated tools might miss.
Secure Coding Practices: SwA enforces the use of secure coding standards and
guidelines. This training and oversight help developers avoid common pitfalls that lead to
vulnerabilities, such as improper input validation, poor error handling, and weak
cryptographic implementations.
Dependency Management: Modern applications rely heavily on third-party libraries and
components. SwA includes practices to manage these dependencies by ensuring they are
up-to-date and free from known vulnerabilities.
Continuous Monitoring and Improvement: The process doesn't end at deployment. SwA
involves continuous monitoring of the application for new vulnerabilities and emerging
threats. It also fosters a culture of continuous improvement, where lessons learned from
incidents or new findings are fed back into the development process.

Approaches to Software Security:

- Static Application Security Testing (SAST) – Analyze code without running it.
- Dynamic Application Security Testing (DAST) – Test software during execution.
- Code Review – Peer reviews to find vulnerabilities.
- Penetration Testing – Simulated attacks to test defenses.

4|Page
 - Secure Coding Practices – Use safe libraries, validate inputs.
Key Principles of Software Security:
a. Secure by Design – Security considered from the start.
b. Least Privilege – Users/programs get only the permissions they need.
(Ex: Marketing Department User can’t access HR files, Backup Program cannot
modify the original database apart from copying)
c. Defense in Depth – Multiple security layers.
d. Fail Securely – Systems should fail in a safe way.
e. Regular Updates – Patch vulnerabilities quickly.

Main Principles of software assurance:

1. Confidentiality – Protect sensitive data.

2. Integrity – Prevent unauthorized modifications.

3. Availability – Ensure system remains usable.
4. Authentication – Verify user/system identity.
5. Authorization – Grant access based on privileges.
6. Accountability – Enable auditing and logging.
7. Reliability – Ensure consistent, fault-tolerant operations.

Security depends on processes and practices.

Vulnerabilities often arise from poor decisions during specification, design, and coding
(intentional or accidental), as well as from choices of programming languages, tools, and
operational environments.
 Key difference between secure and insecure software is not just the technology used,
but the development processes and practices followed.

THREATS TO SOFTWARE SECURITY

 Vulnerability: It is a flaw, weakness, or gap in a system, application, or process that

can be exploited. It's the "unlocked door" or the "faulty lock" of a system. Vulnerability
is passive; it exists whether or not it's ever exploited.
o Example: A web application's code fails to properly validate user input,
allowing malicious code to be submitted.
 Threat: A threat is a potential danger or an adverse event that could exploit a
vulnerability. A threat is a possibility, not an action itself.
o Example: A hacker who discovers the input validation flaw and knows how to
use it for an attack.
 Attack: An attack is the actual act of a threat exploiting vulnerability to cause harm.
o Example: The hacker successfully executes malicious SQL code through the
web application's search bar, accessing sensitive data in the database.
 Risk: Risk is the potential for loss or damage that results from a threat exploiting a
vulnerability.
o Example: The risk is the potential for financial loss and reputational damage if a
hacker successfully steals customer data from the database.

5|Page
 “The threat (source of danger) is often a person intending to do harm, using one or
more malicious software agents such as stealing data, disrupting operations, or
gaining unauthorized access.”

Software is subject to two general categories of threats:

 Development threats: When someone inside the company (like a programmer)

changes or hides something during software building.
Example: A developer adds a secret “backdoor” in the code so they can later access
the system.
 Operational threats: When attackers try to break the software after it is released
and running.
Example: Hackers exploit a weak login system to steal user data.
 Insider risks – A dishonest engineer can mess with the design, code, or test cases to
create hidden weaknesses.
Example: A tester ignores reporting a serious bug on purpose so it can be exploited later.

 Operational attacks – Hackers use known software flaws that are not fixed (unpatched)
to attack systems.
Example: A buffer overflow in a browser lets hackers run their own malicious code on a
user’s computer.

 Targeted weaknesses – Attackers usually go after the “entry points” of software like web
applications, databases, or operating systems.
Example: SQL injection in a website login form to steal all usernames and passwords.

Malware: Malware or malicious software is a general term used to describe software that
is harmful. It includes various subtypes: Viruses, Worms, Trojans, Ransomware, and
Spyware.

Denial-of-Service (DoS) Attacks: These attacks aim to make a service unavailable by

overwhelming it with a flood of traffic or requests, causing it to slow down or crash. A
Distributed Denial-of-Service (DDoS) attack uses a network of compromised devices (a
botnet) to launch the attack from multiple sources, making it harder to block.

SOURCES OF SOFTWARE INSECURITY

Software insecurity stems from various sources, primarily falling into three categories:
design flaws, coding errors, and misconfigurations. These vulnerabilities can be
introduced at any stage of the software development lifecycle.

Design Flaws
Insecure design is a fundamental source of software insecurity. If security isn't considered
from the very beginning, the software's architecture may be built on faulty assumptions,
making it inherently vulnerable, even if the code itself is bug-free.
Lack of Threat Modeling: Without systematically identifying potential threats and their
impact during the design phase, developers may fail to implement necessary security
controls, leaving the system exposed to predictable attacks.
Poor Security Architecture: This includes a failure to apply principles like least privilege
or defense in depth. An application might grant excessive permissions to a user or
6|Page
component, or it might rely on a single security control that, if bypassed, leads to a total
compromise.
Coding Errors
These are the most common source of vulnerabilities, often resulting from a lack of secure
coding practices or developer inexperience.
 Input Validation Failures: This is a classic vulnerability. If an application doesn't
properly validate, filter, or sanitize user input, an attacker can inject malicious code.
Common examples include SQL injection (manipulating a database query) and
Cross-Site Scripting (XSS) (injecting malicious scripts into a web page).
 Memory Management Errors: Languages like C and C++ require manual memory
management, which can lead to vulnerabilities like buffer overflows. This occurs
when a program tries to write more data into a memory buffer than it can hold,
overwriting adjacent memory and potentially allowing an attacker to execute
arbitrary code.
 Insecure Cryptography: Improper use of encryption, such as using weak
algorithms, hard-coding cryptographic keys, or failing to protect sensitive data at
rest and in transit, can lead to data exposure.

Misconfigurations and External Dependencies

Even a well-designed and coded application can be insecure if it's deployed with incorrect
settings or relies on vulnerable components.
1. Security Misconfiguration: This includes using default, easily guessable
credentials; having overly permissive file and directory permissions; or enabling
unnecessary services and features. These simple mistakes create an easy entry point
for attackers.
2. Vulnerable and Outdated Components: Most modern software relies on a vast
network of third-party libraries and open-source components. If these components
have known vulnerabilities that are not patched, the entire application becomes
insecure. This is a major factor in supply chain attacks, where an attacker
compromises a single component to affect thousands of downstream applications.
3. Inadequate Logging and Monitoring: Without proper logging, an organization
might not be able to detect a security breach in time to prevent significant damage.
Insufficient monitoring means that even if a system is breached, there may be no
record of the attacker's actions.

Common causes of vulnerabilities include:

 Complexity and changes in software models,

 Wrong assumptions about environment or inputs,

 Poor input validation, error handling, and interface design,
 Flaws in middleware, OS, or hardware layers,
 Unintended interactions with third-party components.

 Mistakes are unavoidable, so vulnerabilities can arise at any stage (requirements →

design → coding → deployment).

 As software grows in size and complexity; exploitable weaknesses will always exist,
even with secure processes.

7|Page
 BENEFITS OF DETECTING SOFTWARE SECURITY DEFECTS EARLY

 ROI & Cost of Security: Limited ROI data exists for fixing security flaws, but studies
show reducing defects (including security flaws) early in the SDLC leads to huge cost
savings. Fixing defects late (after release) can cost 60–100 times more than fixing them
during design.
 Common Mistake: Organizations often cut quality and security practices to save time
or money, but this leads to more defects, longer schedules, and higher costs. Skipping
quality assurance during requirements, design, and testing is one of the worst decisions.
 Defect Impact on Schedule: Projects with fewer defects finish faster. Poor quality
causes delays, schedule overruns, and even project cancellations (about 50% of canceled
projects cite poor quality). IBM and Capers Jones studies confirm this.
 95% Defect Removal Rule: Projects that remove ≥95% of defects before release
achieve the shortest schedules, lowest cost, and highest user satisfaction. If more than
5% of defects remain, the software becomes vulnerable, costly, and delayed.
 Schedule Pressure = More Defects: Excessive deadline pressure makes developers cut
corners, leading to 4× more defects. This causes rework that takes more time than the
original development.
 Error-Prone Modules: A small number of modules (20% of code) cause most errors
(80%). These modules are complex, rushed, and poorly tested, and they cost 2–4× more
per function point than normal modules.
 Defect Prevention Saves Time & Cost:
 Reworking defective requirements, design, and code makes up 40–50% of total
project cost.
 Every 1 hour spent on prevention saves 3–10 hours of repair later.
 Requirements errors are especially costly: fixing them after release can cost 50–
200× more than fixing them early.
 Proactively addressing quality and security early in the SDLC ensures faster
schedules, lower costs, fewer defects, and more secure software. Neglecting this leads to
delays, high expenses, and insecure products.

Business point of view:

 Executives need ROI justification: New development approaches, including software

security, must show clear business or strategic value (e.g., higher profit, market share,
prestige, safety) rather than just being technically innovative.
 Evidence of ROI in security:
 Investments in patching, CSIRTs, and early security practices show measurable
benefits.
 Case studies (Hoover project) prove that secure design reduces defects and can cut
security risks by up to 80%.
 Well-designed applications have only ¼ the security flaws compared to poorly
designed ones.
 Market value impact: Vendors lose about 0.6% stock value (~$0.86B) for each public
vulnerability disclosure, showing strong financial incentives to build secure software.
 Future outlook: Companies like Microsoft may publish ROI data from their Security
Development Lifecycle, and industry research continues to link secure software with
tangible business benefits.
8|Page
 Managing Secure Software Development

We now see some of the key project management and software engineering
practices to aid in accomplishing secure software.

1. Which Security Strategy Questions Should be asked?

Achieving adequate software security is not just about regulations or best practices—it
depends on the specific product, system, or service and what needs protection.
Organizations must:
 Identify what value and assets must be protected, and why.
 Consider the consequences if assets are not protected and how much disruption can
be tolerated.
 Understand and manage residual risk (risk left after mitigation).
 Develop an implementable and enforceable security strategy based on these
insights.
Since no system can protect everything, stakeholder input is crucial to determine risk
tolerance and resilience. Security as part of risk management means assessing:
 What could go wrong,
 How likely it is,
 What impact it will have, and
 What actions reduce likelihood and impact to acceptable levels.

2. A Risk Management Framework for Software Security.

A Risk Management Framework (RMF) is a structured process used to identify, assess,

manage, and monitor risks throughout the Software Development Life Cycle (SDLC).
It ensures that software security is not a one-time activity, but a continuous and iterative
process.

Core Stages of the RMF (Closed-loop, Iterative Process)

I. Risk Identification
a. Find possible threats, vulnerabilities, and weaknesses.
b. Example: “Our login module is vulnerable to brute force attacks.”
II. Risk Assessment / Analysis
a. Determine likelihood (chance of happening) and impact (damage if it
happens).
b. Example: High likelihood + high impact = critical risk.
III. Risk Mitigation / Control
a. Take actions to reduce risks (design fixes, coding standards, patching,
extra security controls).
b. Example: Add rate-limiting and account lockout for login attempts.
IV. Risk Monitoring
a. Continuously track risks and check if mitigation is effective.
b. Example: Run penetration tests, monitor logs, review new vulnerabilities.
V. Risk Communication / Reporting
a. Share results with stakeholders (developers, managers, executives).
9|Page
 b. Example: “We reduced X% of vulnerabilities; residual risk remains in Y
module.”

3. Software Security Practices in the Development Life Cycle

Software security is not just about testing at the end—it must be integrated into every phase of the
development life cycle. These practices are often called security touchpoints.

1. Requirements Phase
 Define security requirements alongside functional requirements.
 Use abuse/misuse cases (what could go wrong) to identify threats early.
 Example: Require user authentication, data encryption, and access control policies.
2. Design / Architecture Phase
 Perform architectural risk analysis → check for insecure design patterns, weak
protocols, poor trust boundaries.
 Apply threat modeling → identify assets, attackers, threats, and mitigations.
 Example: Use secure design principles (least privilege, defense-in-depth).
3. Implementation / Coding Phase
 Enforce secure coding standards (e.g., OWASP, CERT).
 Use static code analysis tools to detect vulnerabilities like buffer overflows,
injection flaws.
 Example: Validate all inputs, use parameterized queries instead of string
concatenation in SQL.
4. Testing Phase
 Create security test plans (risk-based testing, not just functional).
 Perform code reviews, penetration testing, and fuzz testing.
 Example: Try injecting malicious inputs to ensure the system rejects them.
5. Deployment & Operations Phase
 Apply configuration management to avoid insecure defaults.
 Ensure patch management for vulnerabilities after release.
 Perform ongoing monitoring, audits, and incident response.
 Example: Apply OS and library security updates, log suspicious activities.
6. Maintenance & Feedback Phase
 Gather feedback from users, bug reports, and attack data.
10 | P a g e
  Continuously update threat models and risk assessments.
 Example: If a new attack pattern emerges, re-test the system for resilience.

PROPERTIES OF SECURE SOFTWARE

Software Security:
 Concept / Discipline: The field of practices, methods, and activities aimed at
protecting software from threats, vulnerabilities, and attacks.
 Focus: How to design, develop, and maintain software to make it resilient against
security risks.
 Example: Applying secure coding practices, threat modeling, input validation,
encryption, and security testing.

Software security is understood within the broader context of software assurance, which
involves three key qualities:

1. Trustworthiness – no exploitable vulnerabilities (whether accidental or malicious).

2. Predictable execution – software works as intended, resists attacks, and avoids
unintended behaviors.
3. Conformance – processes and products follow requirements, standards, and
procedures.

Secure Software:
 End Product / Result: A software system that actually demonstrates security
properties.
 Focus: The quality of the software itself – being trustworthy, predictable under
attack, and free from exploitable weaknesses.
 Example: A banking app that prevents unauthorized access, protects sensitive data,
and continues working safely even under attempted attacks.

The properties by which security characteristics can be described. These properties consist
of
(1) a set of core properties whose presence (or absence) are the ground truth that
makes software secure (or not) and
(2) a set of influential properties that do not directly make software secure but do
make it possible to characterize how secure software is.

Core Properties of Secure Software

1. Confidentiality – Protects information, assets, and behavior from unauthorized
access.
2. Integrity – Prevents unauthorized or unintended modifications (code, data,
configuration, behavior).
3. Availability – Ensures software is usable by authorized users when needed, and
inaccessible to unauthorized users.
4. Accountability – Tracks and records all security-relevant actions with attribution of
responsibility.
5. Non-repudiation – Prevents denial of actions performed, ensuring accountability
cannot be bypassed.

11 | P a g e
Core Property Meaning Example Attack / Violation
SQL Injection – attacker extracts
Protects data, assets, and behavior from
Confidentiality private information from
unauthorized access.
database.
Prevents unauthorized or unintended Cross-Site Scripting (XSS) –
Integrity modification of code, data, modifies web app behavior;
configuration, or behavior. Tampering with logs.
Ensures software/services are usable by
Denial of Service (DoS) or XSS
Availability authorized users when needed; blocks
causing service disruption.
unauthorized use.
Records and tracks all security-relevant Log deletion or manipulation to
Accountability
actions with attribution. hide malicious actions.
Buffer overflow with log
Non- Prevents denial of actions performed;
alteration – attacker denies
repudiation enforces accountability.
responsibility.

Definition:
A log is a file or data stream that stores time-stamped information about events, actions, or
states in a system, application, or network.

Examples of Logs:
 System logs – record OS events (startup, shutdown, errors).
 Application logs – record software activity (user logins, data access, errors).
 Security logs – record authentication attempts, access control decisions, firewall
actions.
 Audit logs – track sensitive or security-relevant operations (e.g., financial
transactions, configuration changes).

Quiz: Core Properties of Secure Software

1. Which property ensures that sensitive information is not exposed to unauthorized users?
Answer: Confidentiality
2. An attacker tampers with database records. Which property is violated?
Answer: Integrity
3. A Denial-of-Service (DoS) attack mainly affects:
Answer: Availability
4. Tracking and recording every admin action is part of which property?
Answer: Accountability
5. Which property prevents a user from denying that they performed an action?
Answer: Non-repudiation
True/False Questions
6. Integrity ensures data is accurate and cannot be altered without authorization.
Answer: True
7. Availability means the system should be accessible to both authorized and unauthorized
users.
Answer: False
8. Non-repudiation supports accountability by ensuring users cannot deny their actions.
Answer: True

12 | P a g e
Short Answer Questions
9. Give one example of a confidentiality attack.
Answer: SQL Injection used to steal sensitive data from a database.
10. If an attacker deletes security logs, which two properties are violated?
Answer: Accountability and Integrity.
11. Why is Availability important?
Answer: It ensures that authorized users can access the system or service whenever
needed.

MEMORY-BASED ATTACKS

A memory-based attack is a type of cyber attack that exploits vulnerabilities in a computer's

memory to gain access to sensitive information or to take control of the system. These
attacks are becoming more common and can be difficult to detect with traditional antivirus
software.

These attacks bypass traditional antivirus solutions that scan for file signatures. They work
by injecting or exploiting code within legitimate processes, often using vulnerabilities in
software or legitimate administrative tools like PowerShell (a powerful tool for automation
and system management very useful for admins, but also abused by attackers.). Attackers
can then gain control over the system to steal data, such as passwords or encryption keys, or
disrupt operations.

How Memory-Based Attacks Work

1. Initial Access
 The attacker first needs a way into the system.
 Common methods:
o Phishing → sending fake emails with malicious links or attachments.
o Exploiting vulnerabilities → taking advantage of weaknesses in software
(like unpatched applications).
 Think of this like a thief finding an unlocked window or tricking you into opening
the door.

2. Malicious Code in Memory

 Once inside, the attacker doesn’t immediately put files on your hard disk.
 Instead, they inject code directly into RAM (Random Access Memory).
 This makes the attack “fileless” – nothing suspicious is stored on the hard drive.
 Example: A piece of malware script or small program fragment sits in memory and
waits to act.

3. Exploiting Legitimate Tools

 To stay hidden, attackers use built-in system tools that are normally trusted:
o PowerShell (command-line automation tool in Windows).
o WMI (Windows Management Instrumentation) for system management
tasks.
 Since these tools are trusted by the operating system, their usage looks normal,
making the attacker blend in like a spy wearing a uniform.

13 | P a g e
4. Code Injection and Execution
The attacker must execute their code inside memory. They use techniques like:
 Buffer Overflow
o A memory buffer has a limited size (like a glass of water).
o If the attacker pours too much data, it “spills” into nearby memory areas.
o This lets them overwrite instructions and make the system run their own
code.
 Heap Spraying
o The “heap” is a memory region where programs store data.
o Attackers fill it with malicious patterns (shellcode).
o If a vulnerability misdirects the program’s execution, it’s very likely to land
on the attacker’s code, which then runs.

5. Data Exfiltration or Disruption

Once the malicious code is active in memory, attackers can:
 Steal sensitive information → passwords, session tokens, encryption keys.
 Alter system behavior → disable security functions, corrupt files, or install
backdoors.
 Gain further control → move deeper into the network (lateral movement).
It’s like a spy stealing documents or sabotaging systems after sneaking inside
undetected.

6. Evasion of Detection
 Traditional antivirus tools mainly scan files on disk for known malware signatures.
 Since memory-based attacks avoid writing files to disk, they leave fewer traces.
 This makes them harder to detect and stop.
 It’s like a thief who leaves no footprints because they only walk on air.

How to Find Memory-Based Attacks

1. Monitor System Behavior (not just files)

 Look for unusual patterns in memory usage, CPU spikes, or network activity.
 Example: PowerShell or WMI executing suspicious scripts when they normally
wouldn’t.
2. Memory Forensics
 Use tools to analyze running memory (RAM dumps).
 Tools like Volatility, Rekall, or Redline can help detect injected code or anomalies
in processes.
3. Hunting for Abnormal Process Behavior
 Legitimate apps (e.g., Word, Excel, or [Link]) suddenly spawning PowerShell
= suspicious.
 Look for code running in processes that normally wouldn’t have executable code
injected.
4. Log and Event Analysis
 Monitor Windows Event Logs, especially:
o PowerShell logs (event ID 4104 for script execution).
o Process creation logs (event ID 4688).
o WMI activity logs.
5. Network Traffic Monitoring

14 | P a g e
  Detect data exfiltration attempts (sensitive data leaving the network).
 Unusual outbound connections to unknown IPs or domains can be a clue.
6. Use EDR/XDR Tools
 Modern Endpoint Detection & Response (EDR) tools watch memory, scripts, and
system calls.
 They use behavior-based detection, not just signature scanning.

Solutions (Prevention & Defense)

A. Hardening the System

1. Patch Vulnerabilities → Keep OS and applications updated.
2. Disable or Restrict PowerShell if not required.
3. Use Application Whitelisting → only allow trusted programs to run.
4. Enforce Least Privilege → limit admin rights, attackers can’t escalate easily.
B. Security Tools & Configurations
1. EDR/XDR solutions (e.g., CrowdStrike, SentinelOne, Microsoft Defender for
Endpoint).
2. Memory Protection → OS features like Data Execution Prevention (DEP),
Address Space Layout Randomization (ASLR) make memory exploitation
harder.
3. Antimalware with Behavior Monitoring → detects abnormal runtime behavior.
4. SIEM systems → collect and correlate logs for suspicious activity.
C. Network-Level Defenses
1. Network Segmentation → limit attacker’s lateral movement.
2. Monitor Outbound Traffic → block suspicious IPs/domains.
3. Use Intrusion Detection Systems (IDS/IPS) tuned for fileless attack patterns.
D. Incident Response & Recovery
1. If suspicious activity is found, isolate the machine immediately.
2. Perform memory dump and forensic analysis before rebooting (since data in
RAM is volatile).
3. Rotate passwords and keys in case credentials are stolen.
4. Restore from clean backups if system integrity is lost.
✅ Summary:
 Detection → behavioral monitoring, memory forensics, EDR tools, log analysis.
 Prevention → patching, disabling unnecessary tools, least privilege, OS protections
(DEP, ASLR).
 Response → isolate, investigate memory, reset credentials, restore.

Low-Level Attacks Against Heap and Stack

They are called low-level attacks because they exploit vulnerabilities at the foundation of
program execution—memory and CPU control—rather than the higher-level logic of the
software.
 Low-level means they directly manipulate raw memory addresses, CPU registers, and
execution flow.
 Stack and heap are fundamental parts of the process memory layout, managed at a level
close to the hardware.
Background: Memory Layout in a Process
When a program runs, its memory is divided into different regions:
 Stack → Stores local variables, function parameters, and return addresses (LIFO -
15 | P a g e
 Last In First Out structure).
 Heap → Stores dynamically allocated memory (malloc, new, etc.), grows as needed
during execution.
 Code/Text Segment → Stores the actual program instructions.
 Data Segment → Stores global/static variables.
Both stack and heap are crucial for program execution, but their dynamic and flexible
nature makes them prime targets for low-level attacks.

2. Low-Level Attacks on the Stack

Stack-based attacks exploit vulnerabilities like buffer overflows.
a) Stack Buffer Overflow
 Happens when a program writes more data to a stack buffer than it can hold.
 Extra data overwrites adjacent memory, often the saved return address.
 Attacker injects malicious code (shellcode) and overwrites the return address so
that the program executes the injected code.
b) Return-to-Libc Attack
 Instead of injecting new code, attacker reuses existing library functions (like
system("/bin/sh")) by overwriting the return address with libc function addresses.
 Bypasses protections like non-executable stack.
c) ROP (Return-Oriented Programming)
 Advanced version of return-to-libc.
 Attacker chains small code snippets (gadgets) already present in memory.
 Avoids detection by using legitimate instructions.
3. Low-Level Attacks on the Heap
Heap-based attacks exploit how memory allocators manage dynamic memory.
a) Heap Overflow
 Similar to stack overflow, but it overwrites memory in the heap region.
 Can corrupt function pointers, virtual table pointers (C++ vtables), or heap
management structures.
 Leads to arbitrary code execution.
d) Heap Spraying
 Attacker fills heap with malicious payloads at predictable addresses.
 Then triggers a vulnerability (like overflow or UAF) to redirect execution into
sprayed code.
Feature Stack Attack Heap Attack
Target memory Stack (local variables, return
Heap (dynamic allocations)
region addr)
Common
Buffer overflow UAF, double free, heap overflow
vulnerability
Overwrite return address, hijack Corrupt pointers, hijack heap
Attack goal
control flow metadata, execute code
Complexity Usually simpler More complex, allocator-dependent
5. Defenses Against Heap & Stack Attacks
 Stack canaries → Detect overwritten return addresses.
 DEP/NX (Data Execution Prevention) → Mark stack/heap as non-executable.
 ASLR (Address Space Layout Randomization) → Randomizes memory
addresses to prevent predictable exploitation.
 Safe unlinking & hardened malloc → Protects heap metadata.
 Static & dynamic analysis tools → Detect unsafe code (e.g., AddressSanitizer).

16 | P a g e