RN SHETTY TRUST®

RNS INSTITUTE OF TECHNOLOGY

Autonomous Institution Affiliated to VTU, Recognized by GOK, Approved by
AICTE
(NAAC ‘A+ Grade’ Accredited, NBA Accredited (UG - CSE, ECE, ISE, EIE and EEE)
Channasandra, Dr. Vishnuvardhan Road, Bengaluru - 560 098
Ph:(080)28611880,28611881 URL: [Link]

Department of Computer Science & Engineering (Cyber Security)

Subject Name Vulnerability Assessment And Penetration Testing

Subject Code BCY701
Academic Year/Sem 2025-26(ODD)/ VII

Module – 5

Client-Side Browser Exploits: Why Client-Side Vulnerabilities are Interesting, Internet

Explorer Security Concepts, History of Client- Side Exploits and Latest Trends, Finding New
Browser-Based Vulnerabilities, Heap Spray to Exploit, Protecting Yourself from Client-Side
Exploit.
Malware Analysis: Collecting Malware and Initial Analysis: Malware, Latest Trends in
Honeynet Technology, Catching Malware: Setting the Trap, Initial Analysis of Malware.

Textbook: Ch. 23, Ch. 28.

Chapter 23: Client-Side Browser Exploits

In this chapter, we cover the following topics:

• Why client-side vulnerabilities are interesting

• Internet Explorer security concepts
• History of client-side exploits and latest trends
• Finding new browser-based vulnerabilities (with mangleme, jsfunfuzz, css-grammar-fuzzer,
AxEnum, and AxMan)
• Heap spray to exploit
_________________________________________________________________________
Page | 1
Dept. of CSE-CY, RNSIT
• Protecting yourself from client-side exploits

1. Why Client-Side Vulnerabilities Are Interesting

Client-side vulnerabilities exist in software like web browsers, email applications, and media
players. They are increasingly attractive to attackers for several reasons:

Bypass Firewall Protections: Firewalls typically block new inbound connections. If an attacker
lures a user to a malicious website (e.g., [Link]), an outbound connection is established,
creating a communication channel for the attack. If a browser or component vulnerability is
found, the firewall becomes irrelevant as the attack is delivered over the established channel.

Often Run with Administrative Privileges: Exploited client-side vulnerabilities execute code
at the same privilege level as the application. Many users run Internet Explorer or Outlook with
administrative privileges. A successful exploit in this context grants administrative rights,
allowing the attacker to install rootkits, key loggers, start services, access LSA secrets, and cover
their tracks. This leads to potential "browse-and-you're-owned" exploits.

Note: Windows Vista and later introduce features like Internet Explorer Protected Mode and
User Access Control (UAC) to help client-side applications run with lower privileges.

Easy to Target Specific People/Organizations: Client-side vulnerabilities are effective in

spear phishing. Attackers send convincing emails with a link, and when the victim clicks, the
attacker serves malicious web content over the established connection. This allows code
execution on a specific person's computer whose email is known.

2. Internet Explorer Security Concepts

Two key concepts are vital for understanding these attacks:

ActiveX Controls
Microsoft introduced ActiveX to allow developers to extend the browsing experience.
Controls are small programs run from within a container like Internet Explorer (IE). They can
perform actions the user can, such as accessing the registry or modifying the file [Link]

_________________________________________________________________________
Page | 2
Dept. of CSE-CY, RNSIT
typically presents a security warning and a digital signature before installing/running a control,
requiring the user to make a trust decision.
The Danger: An attacker can exploit a security vulnerability in a control that is marked as safe
to be scripted and is signed by a trustworthy corporation. The attacker can host the control on
their server, build HTML to instantiate it, and lure the user to the page.

Internet Explorer Security Zones

Zones allow flexibility to trust some websites more than others.

The Four Built-in Zones (Least to Most Permissive):

1. Restricted Sites
2. Internet
3. Intranet
4. Trusted Sites

Local Machine Zone (LMZ): An implicit security zone.

Prior to Windows XP Service Pack 2 (SP2): Web pages loaded in the LMZ could run
unsigned/unsafe ActiveX controls, run Java applets without prompt, and run other dangerous
functions. It was "basically trivial" to install malware if a page could be loaded in the LMZ.

Zone Elevation Attacks: The goal of these attacks was to "jump cross-zone" (e.g., from
Internet Zone to LMZ) to run scripts with fewer security restrictions.

Restricted Sites Zone: The most restrictive zone, not allowing ActiveX controls or JavaScript.
Outlook uses this zone to safely view HTML-based email in the preview pane.

Internet Zone: Allows scripting and safe-for-scripting ActiveX controls, but IE won't pass
NTLM authentication credentials.

3. History of Client-Side Exploits and Latest Trends

_________________________________________________________________________
Page | 3
Dept. of CSE-CY, RNSIT
Client-side vulnerabilities became more prominent on the Windows platform around 2004,
driven by two changes:

Windows XP SP2 arrived with an on-by-default firewall and hardened system services, largely
protecting against directed attacks. Cybercriminals became more aggressive, creating a "pay-per-
install" business model for malware. With firewalls as a barrier, criminals shifted their focus to
client-side attacks. Symantec's analysis in 2007 showed a significant increase in the proportion
of Microsoft security bulletins addressing client-side vulnerabilities over time, especially starting
in Q1/2004.

Notable Attacks and Vulnerabilities: To understand the present-day threat environment from
client-side attacks, it will help to understand recent history and the set of attacks that got us
here. Due to its prevalence, we’ll again focus on vulnerabilities affecting Microsoft Windows.

MS04-013 (Ibiza and [Link] Attacks):

This was a zone elevation attack that loaded the attacker's HTML into the Local Machine Zone
(LMZ).It was the first widespread "browse-and-you're-owned" attack.

Exploitation: The attack abused a flaw in the mhtml: protocol handler. The flaw allowed a
cross-zone elevation from the Internet zone into the LMZ.
The Trick: The attacker used a URL like mhtml:[Link]
The <ROOT-URL> ([Link] was a local, non-existent file. IE then loaded only the
<BODY-URL> ([Link] but loaded it in the LMZ, which is where the root
URL would have loaded.

Payload: From the LMZ, the attacker used the [Link] ActiveX control (installed by
default as part of MDAC) to download and run arbitrary malware. This attack did not involve a
buffer overrun or shellcode; it was a vulnerability in the protocol handler's logic.

MS04-040 (IFRAME Tag Parsing Buffer Overrun):

_________________________________________________________________________
Page | 4
Dept. of CSE-CY, RNSIT
This was an HTML parsing vulnerability in Internet [Link] by a hacker named ned
using a Python port of the HTML fuzzer mangleme.

The vulnerability was an exploitable Buffer Overflow (BoF) in the FRAME, EMBED, and
IFRAME tags using the SRC and NAME properties. Control over the EAX register led to control
over ECX, which could then be used to control EIP (Instruction Pointer), leading to Remote
Command Execution. The exploit JavaScript, called "Internet Exploiter," posted a week later,
became the basis for exploiting IE vulnerabilities.

[Link] (First of the COM Objects):

This attack exploited a generic COM object that was not an ActiveX control but was registered
via a Java runtime installation. IE loads objects requested via the <OBJECT> tag and valid
Class ID (CLSID), even if they are not ActiveX controls. When IE tried to use ActiveX
interfaces, the generic COM object would crash.

Exploitation: The [Link] object required a custom initialization method after

CreateInstance(). IE, knowing only generic ActiveX interfaces, did not call this method. This left
the object's variables and function table uninitialized, using memory controlled by the attacker.
Attackers used JavaScript (like the InternetExploiter framework) to allocate a bunch of heap
memory, filling it with NOP sleds and shellcode, which was then released for reuse. When IE
attempted to release the object, the assembly instruction looked up and called the release()
function via a pointer (call [object-pointer]+0x8).

The EAX register was loaded with uninitialized memory (attacker-controlled). The subsequent
code loaded a value from memory pointed to by EAX into ECX, and then called a function at
an offset from ECX (call dword ptr [ecx+0x8]). Since ECX was controlled by the attacker, this
was an exploitable condition.
4. Finding New Browser-Based Vulnerabilities
mangleme
Mangleme was the first publicly released fuzzing tool to specifically target browserbased
client-side vulnerabilities. It’s a little outdated now, but it is super simple to set
up, use, and understand, so we’ll start here. You can follow along with this discussion

_________________________________________________________________________
Page | 5
Dept. of CSE-CY, RNSIT
 by downloading the mangleme source code from [Link] mangleme. The
extracted tarball (.tar file) has three relevant files. Tags.h has a list of HTML tags
and relevant parameters for each. Here’s a snippet of the file:

In main(), you’ll see that each page starts with the following hard-coded HTML:

1. Self-Reloading Mechanism
A meta refresh tag is present in the HTML header.
The instruction is to fully load the page and immediately (0 seconds later) redirect to the URL
[Link].

This causes the page to reload repeatedly, with each load generating a different set of HTML.

2. HTML Generation - Initialization

The main() function begins by generating a random seed.
It generates a random number between 1 and 100.
This random number dictates how many times the random_tag() function is called.

3. HTML Generation - Tag Creation (random_tag())

Each call to random_tag() selects one line from tags.h (an HTML element).
The generated tag includes:
A valid HTML element.
Some valid parameters (set to bogus values).
Some bogus parameters (set to bogus values).
4. HTML Generation - Value Creation (make_up_value())
The make_up_value() function determines the content/values for the tag parameters.
It sometimes returns valid HTML constructs.
It sometimes returns a random string of characters (garbage).
5. Output Variability
The final output can range from:
Completely well-formed HTML (when valid constructs are used).
Complete garbage (when random strings/bogus parameters are used).

_________________________________________________________________________
Page | 6
Dept. of CSE-CY, RNSIT
 Points about Mangleme
 We can use the meta refresh tag to easily loop over a large number of test cases.
 If you can define the vocabulary understood by the component, you can build better test
cases by injecting invalid bits into valid language constructs.
 When the application being tested crashes, you need some way to reproduce
the input that caused the crash. mangleme does this with its [Link]
component.

Mozilla Security Team Fuzzers

Jesse Ruderman and the Mozilla security team have publicly released their JavaScript
and Cascading Style Sheet (CSS) fuzzers.
jsfunfuzz Analysis
 Target & Scope
Target: Focuses exclusively on JavaScript parsing and execution.
Difference from mangleme: It does not target core HTML elements or use the meta
refresh tag for reloading.
 Fuzzing Mechanism
File Structure: Uses a single core HTML file ([Link]) that references the fuzzing
logic within [Link].
Input Generation: The fuzzer creates semi-random, sometimes invalid JavaScript
functions.
Testing Steps: It attempts to:
1. Compile these functions.
2. Decompile these functions.
3. Execute these functions, simulating a web browser's behavior.
 Effectiveness & Techniques
1. High Quality Input: The generated JavaScript is "more correct" and exercises a
larger portion of the engine compared to purely random fuzzing.
2. "Dirty Tricks" Employed:
3. Splitting: Splitting a function in half and compiling each half separately to
uncover bugs in the JavaScript compiler's error handling.
4. Nesting: Generating functions with horrendous levels of nesting.

 Impact & Findings

_________________________________________________________________________
Page | 7
Dept. of CSE-CY, RNSIT
 Initial Release (2007): Mozilla announced jsfunfuzz found 280 bugs in Firefox's
JavaScript engine.
Critical Bugs: Two dozen (24) of these bugs were memory safety bugs that could lead to
code execution exploits.
 Crash Reproducibility & Solution
Browser Limitation: When run directly in the browser, a crash only yields a crash dump
or a debugger break-in, making it difficult to reveal the vulnerable JavaScript function.

Reproducibility Issue: Unlike mangleme, jsfunfuzz lacks an equivalent of [Link]

for easy condition reproduction.
Recommended Solution: The tool's author suggests running it from a stand-alone
JavaScript shell (e.g., the Mozilla SpiderMonkey shell).
Benefit: Allows easier isolation of the JavaScript trigger that caused the crash.
Exclusion Mechanism: Once a crash is isolated, the code can be excluded from future
generation iterations to prevent continually hitting the same issue.
Location: Look for whatToTestSpidermonkey and whatToTestJavaScriptCore functions
in [Link] for examples of crash exclusion.

AxEnum

AxEnum is not a fuzzer itself, but rather a component (tool) used for the initial setup phase of an
ActiveX/COM object fuzzer. It was developed by Shane Hird and later utilized within the more
comprehensive fuzzing engine called AxMan.

1. AxEnum's Role (The Enumerator)

 Purpose: AxEnum (or the [Link] component in AxMan) is used to perform

enumeration.
 Action: It lists all registered COM (Component Object Model) objects and retrieves
their associated Type Library (typelib) information on the target system.
 Output: This enumeration process generates the necessary definition files (e.g.,
JavaScript source files for each object) that the main fuzzer uses as input.

_________________________________________________________________________
Page | 8
Dept. of CSE-CY, RNSIT
2. AxMan Fuzzing Engine (The Framework)

 Goal: To discover vulnerabilities in COM objects that are exposed through Internet
Explorer (IE).
 Web-Based Testing: AxMan is a web-based ActiveX fuzzing engine.
o This allows for a realistic test because it is subject to the browser's security
changes, unlike some other COM-based assessment tools.
o It was specifically designed for use with Internet Explorer 6 (based on its
original release notes).

3. Fuzzing Process

 Input Data: Uses the COM/typelib information generated by the AxEnum component.
 Action: It systematically tests each object's properties and methods.
 Output: The fuzzer reports the currently tested CLSID (Class ID) and the property or
method in the browser's status bar, allowing real-time tracking of which call triggers a
crash.

4. Reproducibility and Exclusion

 Crash Frequency: AxMan can trigger hundreds of crashes on a typical system.

 Blacklisting: It uses a [Link] script.
o This script lists all objects, properties, and methods to skip during fuzzing.
o This mechanism allows the user to exclude known bugs/crashes in future runs,
similar in purpose to the exclusion lists in jsfunfuzz.
o

AxFuzz is a foundational, early-stage ActiveX control fuzzer that was later instrumental in
the development of more sophisticated tools like AxMan and Dranzer.

Core Purpose and Origin

 Definition: AxFuzz is a testing tool designed to find vulnerabilities in ActiveX

controls on Microsoft Windows systems, typically through Internet Explorer.
_________________________________________________________________________
Page | 9
Dept. of CSE-CY, RNSIT
  Creator: It was released by Shane Hird.
 Functionality: It performs basic fuzz tests on ActiveX controls.

Key Capabilities

 Safety Check: It has the ability to check if an ActiveX control implements the
IObjectSafety interface.
o Significance: This interface is critical for determining if a control can be safely
scripted from a web page without posing a security risk. A failure to implement
it, or an incorrect implementation, often leads to vulnerabilities.
 Fuzzing Scope: It focuses on exercising the control's exposed properties and methods,
sending malformed or unexpected data to look for crashes or security exceptions.

Relationship to Other Fuzzers

 Predecessor/Influence: AxFuzz's component, axenum (or a similar COM enumerator),

was directly used as the basis for the [Link] component within AxMan.
o AxMan: Developed by H. D. Moore (of Metasploit fame), AxMan took the
enumeration idea from AxFuzz and built a more comprehensive, web-based
fuzzer around it, using JavaScript to automate the calling of methods and
properties.
 Fuzzing Comparison: Early comparisons (e.g., by CERT/CC) showed that while
AxFuzz was effective, later tools like Dranzer offered more extensive coverage of
vulnerability classes and better crash reporting.

AxMan is a significant, automated ActiveX control fuzzing engine developed by H. D.

Moore (creator of the Metasploit Project). It was designed to systematically find security
vulnerabilities in the vast number of ActiveX/COM objects exposed to the web browser,
particularly Internet Explorer (IE).

Here are the pointwise notes on the key aspects of AxMan:

_________________________________________________________________________
Page | 10
Dept. of CSE-CY, RNSIT
1. Purpose and Scope

 Goal: To discover vulnerabilities (especially memory corruption flaws) in

ActiveX/COM objects that can be instantiated and scripted via a web page in Internet
Explorer.
 Methodology: It is a web-based, automated fuzzer, meaning it runs within the
environment (the browser) it is testing, ensuring realistic testing conditions.
 Target Browser: It was primarily designed for and highly effective against Internet
Explorer 6 and subsequent versions.

2. Architecture and Components

 Core Technology: AxMan is an evolution of earlier concepts (like Shane Hird's

AxFuzz) and is built on two main components:
o [Link] (The Enumerator): This component (similar to AxEnum) is run
first. It scans the local registry to discover and list all registered COM objects
on the system.
o Type Library Analysis: It extracts the Type Library (typelib) information
for each object, detailing all exposed properties and methods.

3. Fuzzing Logic

 Code Generation: Using the typelib data, AxMan dynamically generates JavaScript
code designed to instantiate the ActiveX controls and then systematically call every
property and method with various fuzzed (malformed or unexpected) input data.
 Execution: The generated JavaScript is executed by the browser's engine. AxMan then
monitors the process for signs of failure, such as crashes or exceptions.
 Real-time Feedback: It often provides real-time status updates in the browser's status
bar, indicating the currently tested CLSID (Class ID) and the specific property or
method call that is executing.

4. Reproducibility and Filtering

 High Crash Rate: AxMan is known for triggering a very high volume of crashes on a
typical Windows installation due to the large number of third-party controls installed.
 [Link]: The tool includes a [Link] file.
_________________________________________________________________________
Page | 11
Dept. of CSE-CY, RNSIT
 o Purpose: This file allows the user to list objects, properties, or methods that are
known to cause crashes or are deemed out of scope.
o Benefit: This enables researchers to exclude known bugs or unreliable tests
and focus on finding new, unique vulnerabilities. (Similar to the exclusion
mechanisms in jsfunfuzz).
 Output: Upon a crash, the fuzzer reports the exact CLSID, property, and the input that
caused the fault, making it easier to reproduce the vulnerability for exploit
development.

5. Heap Spray to Exploit

Heap Spraying, a technique that revolutionized exploiting vulnerabilities in browsers,

particularly those involving jumps into random heap memory.

1. The Shift in Exploitability

_________________________________________________________________________
Page | 12
Dept. of CSE-CY, RNSIT
  Old Belief: Initially, security experts believed that only stack-based buffer overruns
were reliably exploitable, while heap-based buffer overruns and jumps into
uninitialized/bogus heap memory were not.
 Change Agent: The technique of Heap Spraying, introduced by the hacker Skylined
with his tool InternetExploiter, proved that crashes jumping into "random" heap
memory could, in fact, be exploited for code execution.

2. Internet Exploiter and the Problem

 The Problem: How to control execution when an IE crash causes the program to jump
to an unknown or continually changing random heap memory location?
 The Goal: Ensure that the heap location jumped to is populated with the attacker's
shellcode or a NOP sled leading to the shellcode.

3. The Heap Spray Technique

 Definition: Heap Spraying is the act of filling a process's entire accessible heap
memory with a large amount of attacker-controlled data, typically a NOP sled
followed by the shellcode.
 Attacker Control: An attacker-controlled web page with JavaScript enabled has a
high degree of control over heap memory, as scripts can easily allocate and fill arbitrary
amounts of memory.

4. Implementation Details in Internet Explorer (IE)

 Contiguous Allocation: The key "trick" is ensuring the memory used stays as a
contiguous block and does not get fragmented across heap chunk boundaries.
 IE Heap Manager: Skylined exploited knowledge of the Windows/IE heap manager:
o Large memory chunks are allocated in $0\text{x}40000$-byte blocks.
o 20 bytes are reserved for the heap header.
o A $0\text{x}40000 - 20$ byte allocation fits neatly and completely into one
heap block.
 The Spray Process:

_________________________________________________________________________
Page | 13
Dept. of CSE-CY, RNSIT
 1. The attacker programmatically creates a block consisting of a long NOP sled
(usually many repetitions of $0\text{x}90$) concatenated with the shellcode,
sized to be $0\text{x}40000 - 20$ bytes.
2. It creates a simple JavaScript Array().
3. It fills "lots and lots" of array elements with this built-up heap block.
4. Result: Filling 500+ MB of heap memory grants a high probability that the
"random" jump will land within the controlled NOP sled, leading to the
shellcode.

5. Learning and Testing

 Recommended Practice: The best way to learn is by studying real-world exploits that
used InternetExploiter and Heap Spray.
 Testing Setup: Use an unpatched virtual machine (e.g., Windows XP SP1) with a
debugger.
 Experimentation Steps:
1. Remove the Heap Spray: Observe IE crashing with execution pointing to truly
random heap memory.
2. Restore the Heap Spray: Inspect memory after the spray is finished but before
the vulnerability is triggered.
3. Step Through Assembly: Trace execution when the vulnerability is triggered
to watch the NOP sled being hit, which then smoothly executes the final
shellcode.

6. Protecting Yourself from Client-Side Exploits

A. Keep Up-to-Date on Security Patches:

 Critical Importance: Most real-world compromises are due to unpatched workstations,

not zero-day attacks.

 Action: Leverage Automatic Updates to apply security updates (especially for Internet
Explorer) immediately.

_________________________________________________________________________
Page | 14
Dept. of CSE-CY, RNSIT
 Enterprise Security: Conduct regular scans to identify and update workstations missing
patches.

 Significance: This is cited as the single most important thing to protect against malicious
cyberattacks.

B. Stay Informed

 Resource Reliability: Microsoft is proactive in warning users about active attacks

exploiting unpatched IE vulnerabilities.

 Key Sources (Low-Noise, High-Relevance):

 Microsoft Security Response Center (MSRC) Blog: Provides regular updates about
attacks.
 Microsoft Security Advisories: Offers detailed workaround steps to protect from
vulnerabilities before a security update is available.

 Access: Both resources are available via RSS feeds.

C. Run Internet-Facing Applications withReduced Privileges

This practice serves as a defense-in-depth measure to mitigate the impact of an attack (even a
zero-day) if it successfully exploits a vulnerability.

A. Protected Mode (Windows Vista/7 and Newer)

 Default Behavior: Internet Explorer runs by default in Protected Mode on newer

Windows versions.
 Mechanism: IE operates at low rights, even if the logged-in user is an Administrator.
 Protection: IE is unable to write to the file system or registry and cannot launch
processes.
 Weakness: An attack could still operate in memory and send data off the victim
workstation (e.g., steal data).

_________________________________________________________________________
Page | 15
Dept. of CSE-CY, RNSIT
  Strength: Highly effective at preventing user-mode or kernel-mode rootkits from
being loaded via a client-side browser vulnerability.

B. SAFER Software Restriction Policy (SRP) (Windows XP and Later)

 Purpose: Allows reduced privilege levels on older/down-level platforms where

Protected Mode is unavailable.
 Levels: Allows applications (like IE) to run as Normal/Basic User,
Constrained/Restricted User, or Untrusted User.
 Basic User: The most practical level; it simply removes the Administrator SID
(Security Identifier) from the process token.
o Malware Mitigation: Without administrative privileges, malware cannot
install a keylogger, install/start a server, or install a new driver (to establish
a rootkit).
o Limitations: Malware still runs on the same desktop and could potentially
inject into or remotely control higher-privilege processes.
o Conclusion: Running as a limited user greatly reduces the attack surface.

C. Using Psexec (Alternative for Administrators)

 Tool: psexec (from SysInternals, published by Mark Russinovich).

 Method: Administrators can use the -l argument to launch IE with limited privileges,
stripping out the administrative privileges from the token.
 Convenience: Allows creating separate desktop shortcuts for a fully privileged IE
session and a limited user IE session.
 Command Example: psexec –l –d "c:\Program Files\Internet
Explorer\[Link]"

_________________________________________________________________________
Page | 16
Dept. of CSE-CY, RNSIT
 Chapter- 28

Collecting Malware and Initial Analysis

• Malware
• Latest trends in honeynet technology
• Catching malware: setting the trap
• Initial analysis of malware

[Link]

Malware can be defined as any unintended and unsolicited installation of software on

a system without the user knowing or wanting it.

Types of Malware

There are many types of malware, but for our purposes, the following list of malware will
suffice.

Malware Type, Definition and Key Characteristics

A. Virus

A parasitic program that attaches itself to another program to infect it and perform unwanted
functions. Requires user assistance (e.g., launching the application or script) to execute and
move. Can use polymorphic (changing) technology to prolong detection.

B. Trojan Horse

Malicious software that performs a nefarious deed for an attacker without the user's
[Link] embedded within another piece of software (e.g., pirated software).

C. Worms

Self-propagating viruses. Require no action on the user's part to execute and move from
system to system. Prevalent and used for purposes like distributing Trojan horses and other
malware.

_________________________________________________________________________
Page | 17
Dept. of CSE-CY, RNSIT
 D. Spyware/Adware

Software installed without a user's knowledge to report the user's behavior to an attacker,
often disguised as an advertiser or marketer. Generally not malicious (mostly privacy issues).
High-threat forms use key-logging technology to capture keystrokes, passwords, and financial
information.

2. Malware Defensive Techniques

Attackers employ great defensive measures to protect malware from being detected, focusing
on persistence after reboots and longevity.

 Rootkits: A category of software that hides itself and other software (processes, files,
registry entries, network connections) from system administrators to perform nefarious
tasks. A good rootkit provides reboot survivability.
 Packers: Used to "pack" or compress the Windows PE file format. Common
examples include UPX, ASPack, and tElock.
 Protective Wrappers with Encryption: Tools used to wrap the malware binary
with encryption. Examples are Burneye and Shiva.
 VM Detection: As defenders increasingly use virtualization (like VMware) to study
malware, many pieces of malware now employ techniques to detect the presence of a
virtual machine (VM)

3. Latest Trends in Honeynet Technology

Honeynet technology is a key component in the "arms race" between attackers and defenders,
allowing security professionals to detect and study the latest threats.

Honeypots and Honeynets

 Honeypot: A decoy system placed in a network for the sole purpose of attracting
hackers. They are not valuable and contain no sensitive information but are configured
to appear valuable.
 Honeynet: A collection of two or more honeypots posing as a decoy network.
_________________________________________________________________________
Page | 18
Dept. of CSE-CY, RNSIT
Why Honeypots Are Used

1. Deception as a Motive: Honeypots can be used to deceive attackers by tricking them

into missing the real "crown jewels" and triggering an alarm.
2. Intelligence as a Motive:
o Indications and Warnings: Since a honeypot has no legitimate purpose, any
traffic destined for or coming from it is immediately assumed to be malicious.
This can be the only way to detect malicious activity if signature-based and
anomaly-based intrusion detection systems (IDS) fail.
o Research: Honeypots are used for research to observe attackers' activities in a
controlled environment ("fishbowl") to learn from them and improve overall
network protection. The Honeynet Project is the leader in this effort.

Limitations of Honeypots

 Limited Viewpoint: A honeypot only sees traffic directed at it and may sit undetected
for long periods. Attackers who detect they are in a honeypot will leave.
 Risk: Introducing a new system poses a risk. The greatest risk is a compromised
honeypot being used as a "leaping-off point" to attack the rest of the organization's
network or to attack other organizations (downstream liability).

Types of Honeypots

Type, Description, Risk Level, Common Tools

Low-Interaction

Emulate services and systems to trick the attacker but do not offer full access to the
underlying system.

Low Risk, often used in production environments.

honeyd (de facto standard for low-interaction, good for basic malware) , Nepenthes (more
interactive, built to extract malware binaries) , and Dionaea (successor to Nepenthes, written in
Python).

High-Interaction

_________________________________________________________________________
Page | 19
Dept. of CSE-CY, RNSIT
Actual virgin builds of operating systems (often unpatched) that may be fully compromised
by the attacker.

High Risk, requires high supervision and is often used for research.

Generations of Honeynets

Honeynets are collections of honeypots that offer a small network of vulnerable systems for
study.

 Gen I (2000): Used routers for connection and offered little data collection or
control. Led by Lance Spitzner, who formed [Link].
 Gen II (2003): Used bridging technology to allow the honeynet to reside inside an
enterprise network to attract insider threats. The bridge served as a reverse firewall
called a "honeywall," which provided basic data collection and control.
 Gen III (2005): The honeywall evolved into a product called roo. This generation
significantly enhanced data collection and control and provided data analysis
through an interactive web interface called Walleye.

Gen III Architecture and Functionality

The Gen III honeywall (roo) acts as the invisible front door of the honeynet, placed next to
production systems on the same segment.

1. Data Control: Restricts outbound network traffic from the compromised honeypots
to mitigate risk. This is achieved using iptable rate-limiting rules and snort-inline
(Intrusion Prevention System).
2. Data Collection: The honeywall collects data from multiple sources and forges them
into a common format called hflow. Sources include:
o Argus flow monitor
o Snort IDS
o P0f-passive OS detection
o Sebek defensive rootkit data from honeypots
o Pcap traffic capture
3. Data Analysis: The Walleye web interface allows for querying of attack and forensic
data, including capturing keystrokes and zero-day exploits
_________________________________________________________________________
Page | 20
Dept. of CSE-CY, RNSIT
Data Control
The honeywall provides data control by restricting outbound network traffic from the
honeypots. Again, this is vital to mitigate risk posed by compromised honeypots attacking
other systems. The purpose of data control is to balance the need for the compromise system to
communicate with outside systems (to download additional tool participate in a command-and-
control IRC session) against the potential of the system to attack others. To accomplish data
control, iptable (firewall) rate-limiting rules are used in conjunction with snort-inline (intrusion
prevention system) to actively modify or block outgoing traffic.

Data Collection: The honeywall has several methods to collect data from the honeypots. The
following
information sources are forged together into a common format called hflow:
• Argus flow monitor
• Snort IDS
• P0f—passive OS detection
• Sebek defensive rootkit data from honeypots
• Pcap traffic capture.

_________________________________________________________________________
Page | 21
Dept. of CSE-CY, RNSIT
Data Analysis
The Walleye web interface offers an unprecedented level of querying of attack and
forensic data. From the initial attack, to capturing keystrokes, to capturing zero-day
exploits of unknown vulnerabilities, the Walleye interface places all of this information
at your fingertips.

Figure 28-1 The Walleye web interface of roo

4. Thwarting VMware Detection Technologies

Attackers constantly develop ways to detect virtual environments like VMware. Tools used for
VM detection include

Tool, Method

Red Pill:Uses the Store Interrupt Descriptor Table (SIDT) command to retrieve the Interrupt
Descriptor Table (IDT) address and analyzes it.

Scoopy: Builds on Red Pill's SIDT/IDT trick by checking the Global Descriptor Table (GDT)
and the Local Descriptor Table (LDT) address.

_________________________________________________________________________
Page | 22
Dept. of CSE-CY, RNSIT
Doo: Checks for clues in registry keys, drivers, and other differences between VMware and
real hardware.

Jerry/VmDetect: Checks for differences in the expected versus actual result of normal x86
instructions (overridden by VMware) or checks for VirtualPC-specific or privileged
instructions used by VMware. This is the most effective method.

5. Catching Malware: Setting the Trap

A safe test environment can be set up using a virtual machine (VMware Guest) running a tool
like Nepenthes on a host machine.

 VMware Host Setup: The honeypot's IP address can be set as the DMZ host on the
firewall (e.g., Linksys Firewall) to direct traffic to it. * Caution: Running this setup
introduces risk, as the honeypot is inside the network, trusting the Nepenthes program
not to have vulnerabilities that an attacker could exploit to gain access to the underlying
system and attack the rest of the network.

 VMware Guest Setup: A secure Linux distribution like BackTrack can be used as
the guest operating system, as it is secure and well-maintained, with no services
(except bootp) started by default.
 Using Nepenthes: Nepenthes, which requires the adns package, can be installed on the
BackTrack guest to set the trap.

6. Initial Analysis of Malware

Once you catch a fly (malware), you may want to conduct some initial analysis to
determine the basic characteristics of the malware. The tools used for malware analysis

_________________________________________________________________________
Page | 23
Dept. of CSE-CY, RNSIT
 can basically be broken into two categories: static and live. The static analysis tools
attempt
to analyze a binary without actually executing the binary.

Live analysis tools study the behavior of a binary once it has been executed.

A. Static Analysis

String Analysis extracts sequences of readable ASCII or Unicode characters embedded in the
binary data. These strings often reveal crucial information:

 Network Indicators: URLs, IP addresses, domains used for C2 communication or

payload downloading.
 System Artifacts: Registry keys, filenames, file paths, and service names the malware
intends to create, modify, or delete.
 Error/Log Messages: Human-readable text that might reveal the malware author's
language or coding style.
 API Functions (Unpacked): Names of key Windows API functions that the malware
will call at runtime.

PEiD

The first thing you need to do with a foreign binary is determine what type of file it is. The
PEiD tool is very useful in telling you if the file is a Windows binary and if the file is
compressed, encrypted, or otherwise modified. The tool can identify 600 binary signatures.
Many plug-ins have been developed to enhance its capability. PEiD used to look at our binary.

PE File Structure Analysis (for Windows Executables)

The Portable Executable (PE) format is standard for Windows executables. Analyzing its
components reveals core functionality:

 Import Table: The most critical component. It lists all the DLLs (Dynamic Link
Libraries) and API Functions a program relies on. By reading this list, an analyst can
infer the malware's purpose:

_________________________________________________________________________
Page | 24
Dept. of CSE-CY, RNSIT
 o Networking: InternetOpenUrlA, HttpSendRequest (suggests
C2/downloading).
o Process Manipulation: CreateProcessA, ShellExecute (suggests launching
other processes or shells).
o File/Registry: CreateFileW, RegSetValueEx (suggests modifying the file
system or persistence mechanisms).

Malware authors use packers (e.g., UPX, ASPack) to compress or encrypt the main malicious
code to avoid detection and hinder static analysis.

 Detection Method: Look for high levels of entropy (a measure of randomness) in the
file sections. Highly randomized data suggests encryption or compression. An entropy
level near 8.0 is a strong indicator of a packed binary.
 Action: If packed, static analysis is limited. The analyst must proceed to dynamic
analysis to automatically unpack the file in memory, or manually reverse-engineer the
unpacking stub to get the original code.

Malcode Analysis Pack (iDefense)

iDefense Labs offer a great set of tools called the Malcode Analysis Pack (MAP). The
following tools are contained in MAP:

Norman SandBox Technology

Norman SandBox Technology" refers to a pioneering and patented anti-malware solution

originally developed by the Norwegian company Norman ASA (later Norman Safeground).

_________________________________________________________________________
Page | 25
Dept. of CSE-CY, RNSIT
How it Works:

 It places an unknown or suspicious executable file into a simulated computer system

(complete with emulated hardware, OS, and network connections).
 It then runs the file and monitors its behavior in real-time.
 If the file performs actions typical of malware (like trying to delete system files, harvest
email addresses, or spread across a network), it is identified and blocked before it can
harm the user's actual system.

Key Benefit: Its main purpose was to detect new and unknown viruses, worms, and Trojans
(Zero-Day threats) that had not yet been added to traditional signature-based antivirus
definition files.

_________________________________________________________________________
Page | 26
Dept. of CSE-CY, RNSIT