Unit 3
Source-Level vs. Assembly-Level Debuggers
Definition
Key Differences
Applications in Malware Analysis
Benefits in Malware Analysis
Source-Level Debugger:
Easier to understand if debugging known applications or POCs.
Useful in white-box analysis or internal app security reviews.
Assembly-Level Debugger:
Critical for black-box analysis of malware.
Enables reverse engineering even without any source code.
Helps identify obfuscation, encryption routines, system call manipulations.
Example Tools
Both types of debuggers play important roles in security analysis:
Source-level debugging is ideal when the code is available.
Assembly-level debugging is essential for malware analysis, where the code is
typically obfuscated, packed, or compiled without symbols.
In advanced malware analysis, proficiency in assembly-level debugging is a must, as most
malware comes only as binary executables.
Kernel vs. User-Mode Debugging
Definitions
Comparison
User-Mode Debugging:
Ideal for analyzing droppers, trojans, ransomware, and backdoors.
Allows tracing of API calls, shellcode execution, and decryption routines.
Cannot see stealthy behaviors that rely on kernel manipulation.
Kernel-Mode Debugging:
Crucial for uncovering rootkits, bootkits, and malware that manipulates OS internals.
Enables analysis of system calls, driver routines, IRPs (I/O Request Packets).
Can detect API hooking, kernel patching, and memory hiding techniques used by
advanced persistent threats (APTs).
Tools
Modifying Execution with a Debugger
Modifying execution refers to the process of altering how a program behaves at runtime
using a debugger.
Skipping instructions
Changing register values
Modifying memory contents
Forcing conditional branches
Patching code in memory
Used extensively in malware analysis to:
Bypass protections (e.g., anti-debugging, encryption routines)
Observe alternative code paths
Neutralize harmful behavior
Techniques
Examples
Tools
Modifying execution with a debugger is a critical skill in advanced malware analysis. It
allows analysts to:
Bypass protections
Reveal hidden behavior
Understand logic flow without executing dangerous code
Mastering these techniques provides greater control and faster analysis of even highly
obfuscated or packed malware.
Modifying Program Execution in Practice
Modifying program execution means actively intervening in how a program runs by
altering its behavior during runtime—without modifying the original binary permanently.
This is done using a debugger or runtime environment to:
Bypass code
Change values
Redirect control flow
Force or fake conditions
Tools
Common Techniques with Examples
A. Instruction Patching
What: Overwrite instructions in memory with others (e.g., NOPs or unconditional
jumps).
Example:
o Original: CMP EAX, 0x1 / JNZ 0x401000
o Patched: JMP 0x401000 (force jump always)
B. Forcing Return Values
What: Alter the return value of a function.
Example:
o Set EAX = 0 after a call to IsDebuggerPresent() to fake "no debugger
detected".
C. Skipping Code Sections
What: Modify the instruction pointer (EIP/RIP) to jump past certain code.
Example:
o Skip license verification or error-handling code in cracked software.
D. Memory Editing
What: Change values stored in heap/stack/global memory during runtime.
Example:
o Modify a hardcoded check like if (password == 0xDEADBEEF) by editing
memory.
E. Breakpoint-Triggered Logic
What: Set breakpoints that pause execution and modify state before resuming.
Example:
o Break on decryption routine, dump decrypted strings, skip the malware's
payload launch.
Applications
Challenges & Considerations
Volatile Changes: Changes are in-memory only unless you patch the binary.
Anti-Debugging Tricks: Advanced malware may detect tampering.
System Instability: Modifying kernel-mode execution can crash the OS.
Ethics: Always work in isolated lab environments; never modify execution of real-
world systems without permission.
Ollydbg
OllyDbg is a 32-bit, user-mode, assembly-level debugger for Microsoft Windows, widely
used in malware analysis, exploit development, and binary reverse engineering.
Developed by OllyDbg author Oleh Yuschuk
Focuses on binary-level analysis—no source code needed
Excellent for analyzing packed, obfuscated, or unknown binaries
Key Features
Use Cases
Core Interface Components
Common Plugins
Loading Malware for Analysis
Before loading or executing malware, always follow strict precautions to avoid infecting
your host or network:
Setting Up the Environment
Acquiring the Malware Sample
Common Malware Loading Tricks
Goals
Observe initial behavior (file drop, registry changes)
Trigger payload execution
Extract or unpack encrypted components
Capture C2 domains or IP addresses
Trace control flow using debugger
Identify persistence mechanisms
The OllyDbg Interface
Interface Components
Key Features
Memory Map
The Memory Map is a visual and tabular representation of all memory regions allocated by
the target process during debugging.
It shows:
Code sections
Data segments
Stack and heap allocations
Loaded DLLs and modules
Importantance
Indicators of Malicious Memory Usage
Breakpoints
A breakpoint is a debugging tool that pauses program execution at a specific instruction
or memory location, allowing analysts to inspect the program’s state at that moment.
In malware analysis, breakpoints are used to:
Intercept malicious behavior before it executes
Observe encryption/decryption routines
Bypass obfuscation or anti-debugging logic
Dump unpacked payloads from memory
Types
Setting and Managing Breakpoints in OllyDbg
Usage
Advanced Breakpoint Techniques
Common Breakpoint Evasion Techniques by Malware
Overwriting INT3 bytes
Scanning for debug registers (DRx)
Using time checks to detect delays
Exception-based control flow to avoid detection
Tracing
Tracing is a debugging technique where the program is automatically stepped through
instruction by instruction, logging each operation. In OllyDbg, tracing allows you to
observe the control flow, track execution paths, and identify hidden or obfuscated
code without manually stepping through every line.
Types
Use
Helpful Trace Settings
Challenges
Tracing in OllyDbg gives analysts powerful, fine-grained control over malware execution. It
is ideal for:
Stepping through encryption/decryption loops
Unpacking or dumping hidden code
Understanding obfuscated control flows