SOFTWARE TESTING

Study Notes for Graduate Students

Types · Levels · Methodologies · Key Metrics · Quick Reference

4 Testing Levels Functional Types Non-Functional Methodologies Key Metrics

1. Core Definition & Fundamental Principles

Software testing is a systematic process of evaluating a software system to verify it meets specified
requirements, behaves correctly under expected and unexpected conditions, and is free of critical defects
before delivery. It encompasses activities performed at all stages of the Software Development Life Cycle
(SDLC).

1.1 The 7 Fundamental Principles (Mandatory for Exams)

• Testing shows defect presence: Testing can prove bugs exist but cannot guarantee a completely bug-free
product.
• Exhaustive testing is impossible: Not all input combinations can be tested; risk-based prioritisation is
essential.
• Shift-left (early testing): Defects found in requirements/design cost 10–100x less to fix than those found in
production.
• Defect clustering: ~80% of defects reside in ~20% of modules — apply Pareto analysis to focus effort.
• Pesticide paradox: Repeated identical tests stop finding new bugs; refresh and extend the test suite
regularly.
• Context-dependent: A banking app, embedded firmware, and a mobile game each require very different
strategies.

Page 1
Software Testing — Study Notes for Graduate Students April 2026

• Absence-of-errors fallacy: A bug-free product that does not meet actual user needs is still a failed product.

Page 2
Software Testing — Study Notes for Graduate Students April 2026

2. Levels of Software Testing (The Test Pyramid)

The test pyramid prescribes the relative volume of tests at each level. Unit tests form the wide base (fast,
cheap, isolated), while end-to-end tests form the narrow tip (slow, expensive, brittle). The pyramid guides
investment decisions in any test strategy.

Level Scope Performed By Stage Volume

Unit Individual function/method Developers During coding Highest

Integration Module interaction & data flow Dev / QA Post-unit High

System Full integrated system QA Team Pre-UAT Medium

Acceptance Business requirements &

Users / QA Pre-release Lowest
(UAT) usability

2.1 Unit Testing

Tests the smallest testable unit (function, method, class) in complete isolation.

• Written and maintained by developers, often alongside code (TDD)

• Extremely fast — a full suite may run in seconds
• Uses mocking/stubbing to isolate from external dependencies
• Coverage metrics: statement, branch, path, and condition coverage
• Common frameworks: JUnit (Java), PyTest (Python), Jest (JS), NUnit (.NET)

2.2 Integration Testing

Verifies that separate modules communicate and share data correctly when combined.

• Detects interface mismatches, incorrect API contracts, and data-format errors

• Approaches: Big Bang, Top-Down (stubs), Bottom-Up (drivers), Sandwich
• Particularly important in microservices and SOA architectures
• Tools: Postman, REST-Assured, Spring Test, TestNG

2.3 System Testing

Evaluates the fully integrated system against its specification in a staging environment.

• Tests end-to-end workflows across all system components

• Covers both functional and non-functional requirements
• Performed in an environment that mirrors production as closely as possible

Page 3
Software Testing — Study Notes for Graduate Students April 2026

• Includes regression testing to protect existing functionality

2.4 Acceptance Testing

The final quality gate — validates the system against business requirements and user needs.

• User Acceptance Testing (UAT): real users validate actual business workflows
• Alpha testing: internal stakeholders in a controlled environment
• Beta testing: external users in real-world, uncontrolled conditions
• Contract/Regulation testing: validates legal and compliance requirements

Page 4
Software Testing — Study Notes for Graduate Students April 2026

3. Functional Testing Types

Functional testing validates WHAT the system does — verifying that each feature behaves according to its
specification. Below are the most commonly examined functional testing types.

Smoke Testing

A quick gate-check of the most critical paths in a new build. If smoke tests fail, the build is rejected and deeper
testing is halted — saving the team's time.

• Covers only login, core transactions, and navigation (not exhaustive)

• Automated and triggered automatically on every CI/CD build

• Fast execution — typically 10–20 minutes maximum

• Named after electronics: 'does it smoke when you switch it on?'

When to use: First test on every new build / deployment

Tools: Selenium · Cypress · Jenkins

Sanity Testing

A narrow, focused regression check verifying that a specific bug fix or new feature works correctly without breaking
directly related functionality.

• Narrower in scope than smoke testing — targets a specific area

• Usually unscripted, exploratory, and performed manually by QA

• Performed after hotfixes, patches, or minor configuration changes

• Key distinction: smoke = whole build stability; sanity = specific change

When to use: After targeted bug fixes or minor patches

Tools: Manual execution · TestRail

Page 5
Software Testing — Study Notes for Graduate Students April 2026

Regression Testing

Re-executes previously passing test cases after any code change to ensure new development has not introduced
unintended side effects or broken existing functionality.

• The single biggest candidate for automation due to sheer repetition

• Suite grows with every release — automation is eventually unavoidable

• Covers positive scenarios, negative scenarios, and boundary cases

• Core practice in agile sprints and CI/CD continuous delivery pipelines

When to use: After every code change, bug fix, or feature addition

Tools: Selenium · Playwright · Cypress · Jest

Exploratory Testing

Simultaneous test design and execution without pre-written scripts. Testers use domain knowledge, experience,
and creativity to probe the system for unexpected defects.

• Charter-based exploration replaces formal test cases

• Consistently finds defects that scripted tests fail to anticipate

• Documented via session-based notes, charters, and mind maps

• Requires experienced testers with strong domain and product knowledge

When to use: New features, ambiguous requirements, post-major-change verification

Tools: Xmind · Jira · Session notes

User Acceptance Testing (UAT)

The final gate before release — real users or business stakeholders validate that the software meets agreed
business requirements and is genuinely ready for production use.

• Alpha: internal stakeholders in a controlled, supervised environment

• Beta: external early-adopter users in real, uncontrolled conditions

• Validates complete business workflows, not just isolated features

• Failure at UAT can halt a release and trigger a full regression cycle

When to use: Final stage immediately before production deployment

Tools: Cucumber · FitNesse · TestRail

Page 6
Software Testing — Study Notes for Graduate Students April 2026

4. Non-Functional Testing Types

Non-functional testing evaluates HOW the system performs — addressing qualities like speed, security,
usability, and reliability that are often equally as critical as functional correctness.

Performance Testing

Evaluates system behaviour under varying load conditions, measuring response time, throughput, resource
utilisation, and stability to ensure performance benchmarks are met.

• Load testing: expected normal and peak concurrent user loads

• Stress testing: pushes system beyond capacity to find breaking points

• Spike testing: sudden extreme load surges to test reaction and recovery

• Endurance / soak testing: sustained load over hours or days to find leaks

• Volume testing: large data sets to identify database and I/O bottlenecks

When to use: Before launching any high-traffic or mission-critical system

Tools: JMeter · Gatling · k6 · Locust · LoadRunner

Security Testing

Identifies vulnerabilities, attack vectors, and weaknesses that could be exploited by malicious actors. Ensures the
system protects data, enforces access control, and complies with security standards.

• Penetration testing: simulates real-world attacker behaviour and techniques

• OWASP Top 10: the standard reference checklist for web application security

• Key targets: SQL injection, XSS, CSRF, insecure deserialization, broken auth

• Compliance testing: PCI-DSS (payments), HIPAA (healthcare), GDPR (privacy)

When to use: Any system handling sensitive, financial, or personal user data

Tools: OWASP ZAP · Burp Suite · Nessus · Checkmarx

Page 7
Software Testing — Study Notes for Graduate Students April 2026

Usability Testing

Evaluates how intuitive, efficient, and satisfying the software is for real end users. Identifies UX friction, confusing
navigation, poor labelling, and design issues.

• Real users attempt to complete defined tasks while being observed

• Moderated (in-person facilitator) or unmoderated (remote, unsupervised)

• Key metrics: task completion rate, time-on-task, error rate, satisfaction (SUS score)

• Results drive iterative UX improvements and design changes

When to use: Consumer-facing products; before major releases or redesigns

Tools: UserTesting · Hotjar · Maze · Lookback

Compatibility Testing

Verifies that the software functions correctly across different browsers, operating systems, devices, screen
resolutions, network conditions, and software versions.

• Cross-browser: Chrome, Firefox, Safari, Edge — including legacy IE if required

• Cross-platform: Windows, macOS, Linux, iOS, Android

• Responsive/device: desktop, tablet, and smartphone form factors

• Network conditions: 3G, 4G, 5G, Wi-Fi, and low-bandwidth environments

When to use: Products targeting diverse user environments and device ecosystems

Tools: BrowserStack · Sauce Labs · LambdaTest

Reliability Testing

Evaluates the ability of a system to perform its intended function consistently and without failure over a defined
period under specified conditions.

• Measures MTBF (Mean Time Between Failures) as a key output metric

• Also tracks MTTR (Mean Time To Repair) and availability percentages

• Detects memory leaks, resource exhaustion, and degradation over time

• Validates logging, monitoring, alerting, and recovery mechanisms

When to use: Mission-critical systems: healthcare, finance, telecoms, infrastructure

Tools: JMeter · Chaos Monkey · Gremlin · custom long-run scripts

Page 8
Software Testing — Study Notes for Graduate Students April 2026

5. Structural Testing Approaches

These three approaches differ in how much internal knowledge the tester has of the system under test.
Understanding their distinctions is a common exam question.

Approach Internal Knowledge Primary Use Cases Who Performs It

Black Box None — tests input/output only System testing, UAT, functional QA testers, business
(Opaque) from user perspective testing analysts

White Box Full — source code, architecture, Unit testing, code coverage, Developers, security
(Transparent) logic paths security audit engineers

Grey Box Partial — API contracts, Integration testing, API testing, QA engineers, pen
(Translucent) architecture docs, DB schemas penetration testing testers

5.1 Key Techniques Within Black Box Testing

• Equivalence Partitioning: Divide inputs into classes where all values are expected to behave identically.
Test one value from each class.
• Boundary Value Analysis: Test values at and just beyond the edges of input ranges, where defects cluster
most frequently.
• Decision Table Testing: Model complex combinations of conditions and their expected system actions in a
structured table.
• State Transition Testing: Model system behaviour as states and test valid/invalid transitions between them.

Page 9
Software Testing — Study Notes for Graduate Students April 2026

6. Testing Methodologies & Approaches

6.1 Manual vs Automated Testing

Dimension Manual Testing Automated Testing

Speed Slow — human-paced Fast — machine-paced, 24/7

Consistency Variable — human error possible Perfect — identical every run

Initial cost Low — no tool investment High — setup and scripting

Maintenance Low Ongoing — scripts break on changes

Best for Exploratory, UX, ad-hoc Regression, CI/CD, load testing

Coverage scale Limited by time and headcount Can run thousands of cases

6.2 Test-Driven Development (TDD)

TDD is a development practice where a failing test is written before any production code. The developer then
writes the minimum code to pass the test, then refactors. Cycle: Red (fail) → Green (pass) → Refactor.

• Forces modular, testable, single-responsibility code design

• Produces a comprehensive, always-current automated test suite
• Reduces debugging time — defects are caught within seconds of introduction
• Widely practiced in agile, XP (Extreme Programming), and DevOps teams

6.3 Behavior-Driven Development (BDD)

BDD extends TDD by writing acceptance tests in structured plain English using the Given-When-Then format.
Business stakeholders can read and validate these scenarios directly, eliminating ambiguity in requirements.

Scenario: User logs in with valid credentials

Given the user is on the login page

When the user enters valid credentials and clicks Login

Then the user is redirected to the dashboard

Key tools: Cucumber (Java/Ruby), Behave (Python), SpecFlow (.NET), JBehave.

6.4 Continuous Testing in CI/CD

Page 10
Software Testing — Study Notes for Graduate Students April 2026

Continuous testing embeds automated tests at every stage of the delivery pipeline — triggered on every
commit, pull request, or deployment. It is the cornerstone of DevOps and enables teams to release confidently
at high velocity.

• Unit + smoke tests run on every commit (< 5 minutes feedback target)
• Integration + regression tests run on every pull request merge
• Performance and security tests run nightly or on release candidates
• Platform tools: Jenkins, GitHub Actions, GitLab CI/CD, CircleCI, Azure DevOps

Page 11
Software Testing — Study Notes for Graduate Students April 2026

7. Software Testing Life Cycle (STLC)

The STLC defines the sequence of activities performed during testing. Unlike the SDLC, the STLC is
testing-specific and is often run in parallel with development phases.

Phase Key Activities Entry Criteria Deliverable

Requirement
1. Requirement Review specs; identify testable Requirements document
Traceability Matrix
Analysis items; clarify ambiguities available
(RTM)

Define scope, strategy,

2. Test
schedule, resource plan, risk Approved requirements Test Plan document
Planning
analysis

Write test cases, create test Stable requirements & Test cases, test scripts,
3. Test Design
data, design automation scripts test plan test data

4. Environment Configure hardware, software, Test infrastructure

Test environment ready
Setup networks, and test databases available

5. Test Execute tests, log defects, Test environment + build

Test logs, defect reports
Execution retest fixes, update status available

Evaluate exit criteria, generate All planned tests

6. Test Closure Test Summary Report
summary, conduct retrospective executed

7.1 Defect Life Cycle

Status Description

New Defect discovered and logged for the first time

Assigned Assigned to a developer for investigation and fix

Open Developer actively working on the defect

Fixed Developer resolves and marks the defect as fixed

Retest QA verifies the fix in the same build/environment

Closed Fix confirmed valid; defect is formally closed

Reopened Fix inadequate or regression detected; cycle repeats

Rejected / Deferred Not a valid defect, or postponed to a future release

Page 12
Software Testing — Study Notes for Graduate Students April 2026

8. Key Testing Metrics & KPIs

Metrics provide objective evidence of testing quality and process effectiveness. These are the metrics most
commonly referenced in academic literature and industry practice.

Metric Formula / Definition Target

% of code or requirements exercised by at least one > 80% for critical

Test Coverage (%)
test modules

Number of confirmed defects / KLOC (thousand lines < 1 per KLOC

Defect Density
of code) (world-class)

DRE — Defect Removal > 95% considered

(Defects pre-release / Total defects) × 100
Efficiency excellent

> 95% before release

Test Pass Rate (%) (Passed test cases / Total executed) × 100
go/no-go

MTTD — Mean Time To Average time elapsed from defect introduction to

Lower is better
Detect detection

MTTR — Mean Time To

Average time from defect detection to verified fix Lower is better
Repair

Target: 0 Sev-1
Escaped Defects Defects reported by production users after release
escaped

> 70% for regression

Automation Coverage (%) % of total test cases that are automated
suite

8.1 Defect Severity vs Priority — A Common Exam Distinction

Dimension Severity Priority

Business urgency — how soon must it

Definition Technical impact on system functionality
be fixed?

Product Manager / Business

Set by QA / Test Engineer
stakeholder

Example (High Sev,

Crash in a rarely-used admin panel Fix scheduled for next sprint
Low Pri)

Example (Low Sev,

CEO's name misspelled on homepage Fix immediately before press release
High Pri)

Page 13
Software Testing — Study Notes for Graduate Students April 2026

Page 14
Software Testing — Study Notes for Graduate Students April 2026

9. Quick Reference — Tools by Category

Category Popular Tools

Unit Testing JUnit, PyTest, Jest, NUnit, Mocha, PHPUnit

Integration / API Testing Postman, REST-Assured, SoapUI, Karate, Newman

UI / End-to-End Automation Selenium, Playwright, Cypress, Katalon, Appium (mobile)

Performance Testing JMeter, Gatling, k6, Locust, BlazeMeter, LoadRunner

Security Testing OWASP ZAP, Burp Suite, Nessus, Checkmarx, Metasploit

BDD Frameworks Cucumber, Behave, SpecFlow, JBehave, Robot Framework

Test Management TestRail, Zephyr, Xray, Jira, qTest

CI/CD Integration Jenkins, GitHub Actions, GitLab CI, CircleCI, Azure DevOps

Cross-browser / Device BrowserStack, Sauce Labs, LambdaTest, Firebase Test Lab

Code Coverage JaCoCo (Java), Istanbul (JS), [Link] (Python), SonarQube

10. Exam Cheat-Sheet — Key Distinctions to Memorise

Concept Pair Distinction

Smoke vs Sanity Smoke = entire build stability gate. Sanity = specific fix verification.

Severity = technical impact. Priority = business urgency. Independent

Severity vs Priority
axes.

Verification = 'Are we building the product right?' (process). Validation =

Verification vs Validation
'Are we building the right product?' (user need).

Black = no code knowledge. White = full code knowledge. Grey = partial

Black vs White vs Grey Box
(APIs, architecture).

TDD = developer-written unit tests before code. BDD = Given-When-Then

TDD vs BDD
scenarios readable by business.

Retesting = verifying a specific bug fix. Regression = ensuring the fix

Regression vs Retesting
hasn't broken anything else.

Page 15
Software Testing — Study Notes for Graduate Students April 2026

Alpha = internal users, controlled environment. Beta = external users,

Alpha vs Beta Testing
real-world, uncontrolled.

MTBF = reliability (how long between failures). MTTR = recoverability

MTBF vs MTTR
(how fast you fix it).

Load = expected traffic. Stress = beyond limits (gradual). Spike = sudden

Load vs Stress vs Spike
extreme burst.

These notes cover the canonical software testing curriculum as taught at graduate level. For deeper study, refer to:
ISTQB Foundation Level Syllabus, IEEE 829 (Test Documentation Standard), and the OWASP Testing Guide for
security. Good luck with your examinations.

Page 16