Unit-1: Introduction

Faults, Errors, and Failures

This is a fundamental concept (often called the "F-E-F" model) that describes the relationship
between a bug and its outcome.

●​ Fault (Defect/Bug): This is the static mistake in the software code or documentation. It's
the "typo" or "flaw in logic" introduced by a human (e.g., a developer writing $x > 10$
when the requirement was $x >= 10$).
●​ Error: This is the intermediate, incorrect state that a program enters when a fault is
executed. It's the result of the fault. (e.g., when the user enters 10, the program enters
an "incorrect state" where it thinks the input is invalid).
●​ Failure: This is the observable, external symptom of the error. It's what the user sees.
(e.g., the program displays an "Invalid Input" message to the user, which is a failure to
meet the requirement).
○​ Analogy: Fault = A typo in a recipe card. Error = You adding the wrong amount
of an ingredient. Failure = The cake tastes bad.

Basics of software testing

Software testing is the process of evaluating and verifying that a software product or application
does what it's supposed to do. It's an investigation conducted to provide stakeholders with
information about the quality of the software.

Testing objectives
These are the primary goals of the testing process.

●​ Find Defects: The most common objective—to find bugs (faults) so they can be fixed.
●​ Build Confidence: To run tests and demonstrate that the system works, giving the team
and client confidence in its quality.
●​ Verify Requirements: To check that the software meets all specified functional and
non-functional requirements.
●​ Prevent Defects: The process of test design itself (thinking through scenarios) can help
developers identify issues before they even write the code.

Principles of testing
There are 7 key principles that are universally applicable:

1.​ Testing shows the presence of defects, not their absence: If you find 100 bugs,
you've proven 100 bugs exist. If you find 0 bugs, it doesn't mean there are no bugs left,
just that your tests didn't find any.
2.​ Exhaustive testing is impossible: You can never test every single combination of
inputs and paths, except for the most trivial systems. (e.g., testing a 16-digit credit card
field with every possible number is not feasible).
 3.​ Early testing saves time and money: The later a bug is found (e.g., in production vs. in
the requirements phase), the more expensive it is to fix.
4.​ Defect clustering: A small number of modules (often 20%) usually contain the majority
of the defects (often 80%). This is the Pareto Principle.
5.​ Pesticide paradox: If you keep running the exact same set of tests over and over, they
will eventually stop finding new bugs (just like pesticides eventually stop killing insects).
Test cases need to be reviewed and updated.
6.​ Testing is context-dependent: The way you test an e-commerce website (focus on
usability, security) is very different from how you test flight control software (focus on
reliability, safety).
7.​ Absence-of-errors fallacy: Finding and fixing lots of bugs doesn't mean the system is
good. If the software is unusable or doesn't meet the user's actual needs, it's still a
failure, even if it's "bug-free."

Requirements, behavior and correctness

●​ Requirements: A formal description of what the system is supposed to do.
●​ Behavior: What the system actually does when you use it.
●​ Correctness: This is a measure of how closely the system's behavior matches its
requirements. Testing is the process of checking for correctness.

Testing and debugging

These two are often confused but are very different.

●​ Testing: The process of finding a failure. It identifies that something is wrong.

●​ Debugging: The process of locating the fault (bug) in the code that caused the failure
and fixing it. It identifies why something is wrong.

Test metrics and measurements

These are quantifiable numbers used to track the quality of the software and the progress of
testing.

●​ Test Coverage: Percentage of code (or requirements) that has been tested.
●​ Defect Density: Number of defects found per KLOC (Thousand Lines of Code) or per
feature.
●​ Defect Removal Efficiency (DRE): Percentage of bugs found by the test team before
the software is released to the user.
●​ Test Execution Rate: Number of test cases passed / total number of test cases run.

Verification and Validation (V&V)

A critical concept, often summarized as:

●​ Verification: "Are we building the product right?"

○​ This checks if the software conforms to its design and specifications.
 ○​ It's a static process (like reviews, inspections, and walk-throughs) that happens
without running the code.
●​ Validation: "Are we building the right product?"
○​ This checks if the software meets the user's actual needs and business
requirements.
○​ It's a dynamic process (like running the software) that happens by executing the
code.

Types of testing
Testing is broadly categorized into:

●​ Functional Testing: Tests what the system does (e.g., "Does the login button work?").
●​ Non-functional Testing: Tests how well the system does it (e.g., "Is the login page fast?
Secure? Easy to use?").
●​ Structural Testing: (White-box testing) Tests the internal code structure.
●​ Change-Related Testing: (Regression testing) Tests that recent changes haven't
broken anything.

Software Quality and Reliability

●​ Software Quality: A broad term for how well a system meets its requirements and user
expectations. Key attributes include: Reliability, Usability, Efficiency, Maintainability,
and Portability.
●​ Software Reliability: A specific part of quality. It is the probability that the software will
operate without failure for a specified period of time in a specified environment.

Software defect tracking

This is the formal lifecycle of a bug, from discovery to resolution, managed using tools like Jira
or Bugzilla.

1.​ New: A bug is found and reported.

2.​ Assigned: The bug is assigned to a developer.
3.​ Open: The developer is actively working on a fix.
4.​ Fixed: The developer has fixed the code and checked it in.
5.​ Retested: The tester (QA) verifies if the fix works.
6.​ Closed: The fix is confirmed, and the bug is resolved. (Or Reopened if the fix didn't
work).

Unit-2: White Box and Black Box Testing

White Box Testing (WBT)
●​ Also known as Structural, Glass-box, or Clear-box testing.
●​ The tester can see the internal code and structure of the application.
●​ Goal: To test the internal logic, branches, paths, and statements of the code.
 ●​ It is generally performed by developers.

Static testing
●​ A type of testing where the software is not executed.
●​ It's part of Verification.
●​ Methods:
○​ Code Reviews: A developer's code is manually reviewed by peers to find bugs
and design flaws.
○​ Walk-throughs: The author of the code leads a team through the code to explain
the logic.
○​ Inspections: A very formal, data-driven review process with specific roles
(Moderator, Reader, Recorder).

Static analysis tools

●​ These are automated tools that perform static testing.
●​ They scan the source code (without running it) to find potential issues like:
○​ Security vulnerabilities (e.g., SQL injection).
○​ Violations of coding standards (e.g., unused variables).
○​ Potential bugs (e.g., potential null pointer exceptions).
●​ Examples: SonarQube, PMD, Checkstyle.

Structural testing: Unit/Code functional testing

●​ Unit Testing: The lowest level of testing.
●​ Goal: To test individual, isolated components (a single function, method, or class) to
ensure it works as expected.
●​ Developers write unit tests (e.g., using frameworks like JUnit or pytest) to verify their own
code.

Code coverage testing

●​ A metric used in White Box Testing to measure how much of the code was executed by
the test suite.
●​ Statement Coverage: Percentage of lines of code executed. (Weakest).
●​ Branch Coverage (Decision Coverage): Percentage of decision outcomes (e.g., the
true and false branches of an if statement) executed.
●​ Path Coverage: Percentage of all possible paths through the code executed.
(Strongest, but often impossible to achieve 100%).

Code complexity testing

●​ A method to measure the complexity of a piece of code.
●​ Cyclomatic Complexity: The most popular metric. It measures the number of linearly
independent paths through a program's source code.
●​ Formula: $V(G) = E - N + 2$ (where E = Edges, N = Nodes in the control flow graph)
OR simply (Number of Decisions + 1).
 ●​ A high complexity number (e.g., > 10) indicates the code is complex, hard to test, hard to
maintain, and more likely to have defects.

Black Box Testing (BBT)

●​ Also known as Behavioral or Opaque-box testing.
●​ The tester cannot see the internal code. They only know what the system is supposed to
do.
●​ Goal: To test the system's functionality from a user's perspective, based on the
requirements.
●​ It is generally performed by QA testers.

Requirements based testing

●​ A black-box technique where test cases are designed directly from the requirements
specification.
●​ This ensures traceability—every requirement has one or more test cases to verify it.

Boundary Value Analysis (BVA)

●​ A test design technique based on the idea that bugs often cluster at the boundaries
(edges) of input ranges.
●​ Example: If a field accepts numbers 1-100:
○​ Test Cases: 0 (invalid), 1 (valid), 2 (valid), 99 (valid), 100 (valid), 101 (invalid).

Equivalence Partitioning (EP)

●​ A test design technique that divides all possible inputs into "equivalence classes" or
"partitions," where the system is expected to treat all inputs in a partition the same way.
●​ You only need to test one value from each partition.
●​ Example: For the 1-100 field:
○​ Partition 1 (Invalid): $< 1$ (Test with: -5)
○​ Partition 2 (Valid): $1$ to $100$ (Test with: 50)
○​ Partition 3 (Invalid): $> 100$ (Test with: 150)
●​ BVA and EP are almost always used together.

State/graph-based testing
●​ A technique that models the system as a Finite State Machine.
●​ You identify all the states the system can be in and the transitions (inputs or events) that
cause it to move from one state to another.
●​ Goal: To design test cases that cover every state and every transition.
●​ Example: An ATM. States = {Idle, Card Inserted, PIN Entered, Selecting Transaction}.
Transitions = {insert_card, enter_pin, select_withdrawal}.

Model based testing

 ●​ An advanced technique where a model (like a state/graph model) of the system's
behavior is built.
●​ Test cases are then automatically generated from this model by a tool.

Differences between white box and Black box testing

Feature White Box Testing Black Box Testing

Knowledge Requires knowledge of internal Requires no knowledge of internal

code. code.

Basis Based on code structure, design. Based on requirements, specifications.

Performer Usually Developers. Usually QA Testers.

Objective Test internal paths, logic, branches. Test user functionality, inputs/outputs.

Alias Structural, Glass-box. Behavioral, Opaque-box.

Unit-3: Integration, System, and Acceptance Testing

These are the main levels of testing.

Integration Testing
●​ Goal: To test the interfaces and interactions between two or more modules that have
already been unit-tested.
●​ Top-down integration:
○​ Testing starts from the highest-level module (e.g., the UI) and works down.
○​ Missing lower-level modules are replaced by Stubs (dummy code that returns
hardcoded values).
●​ Bottom-up integration:
○​ Testing starts from the lowest-level modules (e.g., database functions) and works
up.
○​ Modules that call the units being tested are replaced by Drivers (dummy code
that passes test data).
 ●​ Bi-directional (Sandwich) integration: A hybrid approach that uses both top-down and
bottom-up methods to get the "best of both worlds."

System integration (Testing)

●​ This is often confused with regular integration testing.
●​ System Integration Testing (SIT) specifically refers to testing the complete system's
integration with other external systems.
●​ Example: Testing your e-commerce website to make sure it communicates correctly with
the external PayPal API or a FedEx shipping API.

Scenario Testing
●​ A test technique that uses real-world scenarios (or "user stories") to test the software
from end-to-end.
●​ Instead of testing one small feature, it tests a complete flow.
●​ Example Scenario: "A user logs in, searches for a product, adds it to the cart, enters a
discount code, and completes the purchase." This one scenario tests login, search, cart,
and payment modules all working together.

Defect Bash
●​ An informal, ad-hoc testing session, usually held before a major release.
●​ The entire team (developers, testers, product managers, designers) "bashes" the
application, trying to find bugs in creative ways.
●​ It's unscripted and relies on collective experience.

Functional versus Non-functional testing

●​ Functional Testing:
○​ Tests what the system does.
○​ Verifies features and business logic.
○​ Examples: Login test, "Add to Cart" test. (Uses black-box techniques).
●​ Non-functional Testing:
○​ Tests how well the system performs.
○​ Verifies characteristics like speed, reliability, security.
○​ Examples: Performance testing, Security testing, Usability testing.

Design/Architecture verification
●​ A static review (Verification) of the high-level system design and architecture documents.
●​ Goal: To find flaws before code is written.
●​ Checks: Does the architecture meet non-functional requirements (like scalability)? Is it
secure? Is it logical?

Deployment testing
 ●​ Testing that happens after the software is built, focusing on the deployment process
itself.
●​ Goal: To ensure the software can be successfully installed, uninstalled, configured, and
upgraded in the production-like (or "Staging") environment.

Beta testing
●​ A type of Acceptance Testing.
●​ The software ("beta version") is released to a limited number of real, external users
outside the company.
●​ Goal: To get feedback from real-world usage and find bugs that the internal test team
missed.

Scalability testing (Non-functional)

●​ Tests the system's ability to "scale up" to handle an increasing amount of load (users,
transactions, data).
●​ Goal: To find the point at which the system's performance starts to degrade or fail.

Reliability testing (Non-functional)

●​ Tests the system's ability to run without failure for a long period of time under normal
conditions.
●​ Goal: To find issues like memory leaks or crashes that only happen after sustained use.
●​ Example: A "soak test" where the system is left running for 48 hours.

Stress testing (Non-functional)

●​ Tests the system beyond its normal operational limits.
●​ Goal: To see how the system fails. Does it crash and corrupt data (bad), or does it fail
gracefully (e.g., "Server busy, please try again") (good)?

Acceptance testing
●​ The final level of testing, performed after System Testing.
●​ Goal: To validate (see Unit 1) that the software meets the business requirements and is
acceptable to the client or user.
●​ Types:
○​ Alpha Testing: Performed internally by employees (not the dev/QA team) at the
developer's site.
○​ Beta Testing: (See above) Performed externally by real users.

Acceptance criteria
●​ The specific, predefined, and measurable conditions that the software must meet to be
accepted by the user.
●​ Example: "The search results page must load in under 2 seconds." "A new user must be
able to register and log in."
Test cases selection and execution
●​ Test Case Selection: The process of choosing a subset of all available test cases to run
in a specific test cycle. This is necessary because you often don't have time to run all
tests. Selection is based on risk, priority, and code changes.
●​ Test Case Execution: The process of running the selected test cases, comparing the
actual result with the expected result, and marking the test as Pass or Fail.

Unit-4: Test Selection & Minimization for Regression

Testing
Regression testing
●​ Goal: To ensure that a new code change (a bug fix or a new feature) has not
accidentally broken any existing functionality.
●​ It involves re-running old tests.
●​ This is the most critical candidate for automation.

Regression test process

1.​ Developer checks in a new code change.
2.​ The tester (or automated system) identifies the impact of the change.
3.​ A subset of the existing test suite (the "regression suite") is selected.
4.​ The selected test cases are executed.
5.​ Any new failures are reported as regressions (bugs).

Initial Smoke or Sanity test

These are quick tests done on a new build before starting detailed testing.

●​ Smoke Test:
○​ A broad but shallow test of the most critical, basic functionality.
○​ Goal: To see if the build is stable enough to even start testing.
○​ Analogy: "Does it catch fire when you turn it on?" (e.g., Can the app start? Can
you log in? Does the main page load?).
○​ If a smoke test fails, the build is rejected immediately.
●​ Sanity Test:
○​ A narrow but deep test of a specific area of functionality, usually one that was just
fixed or changed.
○​ Goal: To quickly check if the new fix actually works and hasn't obviously broken
its surrounding area.

Selection of regression tests

●​ Since running the entire test suite is often too slow, we must select a subset.
●​ Techniques:
 ○​ Prioritization: Run high-priority (P0, P1) tests first.
○​ Risk-Based: Select tests that cover high-risk or critical features.
○​ Change-Based: (Most common) Select tests related to the specific modules of
code that were changed.

Execution Trace
●​ A detailed log of the path of execution a program took while running a specific test case.
●​ It shows which functions were called, in what order, and sometimes the values of
variables. Used for debugging.

Dynamic Slicing
●​ A debugging technique. A "dynamic slice" for a variable at a certain point is the set of all
program statements that actually affected the value of that variable during that one
specific execution.
●​ It helps a developer ignore all the irrelevant code and "slice" down to only the statements
that caused a bug.

Test Minimization
●​ The process of reducing the number of test cases in a test suite (e.g., a regression suite)
without reducing its ability to find bugs.
●​ Goal: To remove redundant test cases (e.g., two tests that cover the exact same code
path and logic).

Tools for regression testing

●​ Any test automation tool can be used.
●​ Examples: Selenium (for web apps), Appium (for mobile apps), JMeter (for
performance regression), TestComplete, Katalon Studio.

Ad hoc Testing
●​ Completely informal, unplanned, and unscripted testing.
●​ The tester relies on their intuition, experience, and creativity to find bugs.
●​ Error Guessing: A common ad-hoc technique where the tester "guesses" where bugs
are likely to be (e.g., "What if I enter a negative number here? Or a special character?").

Pair testing
●​ A technique where two people (e.g., two testers, or a tester and a developer) work
together at one computer.
●​ One person "drives" (uses the mouse and keyboard), and the other "navigates"
(suggests what to test, takes notes, observes).
●​ This often results in finding more bugs due to the two different perspectives.

Exploratory testing
 ●​ A structured approach to unscripted testing. It is not the same as ad-hoc testing.
●​ It is defined as "simultaneous learning, test design, and test execution."
●​ The tester explores the application, and based on what they learn, they dynamically
design and execute new tests in real-time.

Iterative testing
●​ A testing approach used in iterative development models (like Agile and Scrum).
●​ Instead of one big testing phase at the end, testing is done in every iteration (or "sprint").
●​ Each iteration builds on the last, so regression testing is extremely important.

Defect seeding
●​ A technique to measure the effectiveness of a test suite or a tester.
●​ Process:
1.​ A manager or developer intentionally inserts a known number of bugs (called
"seeds") into the code.
2.​ The tester is given the code and told to find all the bugs.
3.​ Management then checks: How many of the real bugs were found? And how
many of the seeded bugs were found?
●​ Formula: (Seeded bugs found / Total seeded bugs) gives an estimate of the tester's (or
test suite's) effectiveness.

Unit-5: Test Management and Automation

Test Planning
●​ The most important activity in test management.
●​ It results in a formal document called a Test Plan.
●​ A Test Plan defines:
○​ Scope: What will be tested and what will not be tested.
○​ Test Strategy: The types of testing to be done (Unit, System, Performance, etc.).
○​ Resources: Who will test? What hardware/software is needed?
○​ Schedule: Milestones and deadlines for testing.
○​ Test Deliverables: What documents will be produced (e.g., Test Cases, Bug
Reports, Test Summary Report).
○​ Entry/Exit Criteria:
■​ Entry Criteria: When can testing start? (e.g., "Build passes smoke test.")
■​ Exit Criteria: When is testing done? (e.g., "95% of tests passed and 0
critical bugs are open.")

Management, Execution and Reporting

●​ Test Management: The overall process of managing the testing effort, tracking progress
against the Test Plan, managing risks, and assigning tasks.
●​ Test Execution: (Covered earlier) The process of running the tests.
 ●​ Test Reporting: The process of communicating test results to stakeholders. This
includes daily status reports, bug reports, and a final Test Summary Report (which
summarizes all testing activities and gives a final recommendation on whether to release
the software).

Software Test Automation

●​ The practice of using special software (automation tools) to execute test cases
automatically and compare the results, without human intervention.
●​ Pros: Fast, repeatable, reliable (for regression), can run 24/7.
●​ Cons: High initial cost and setup time, requires skilled engineers, brittle (can break
easily with UI changes), not good for finding new bugs (only for checking old features).

Scope of automation
●​ You can't (and shouldn't) automate everything.
●​ Good candidates for automation:
○​ Regression tests: Tests that run over and over.
○​ Data-driven tests: Tests that need to run with many different sets of data.
○​ Performance/Load tests: Impossible to do manually.
○​ Smoke tests: To quickly validate a new build.
●​ Bad candidates for automation:
○​ Usability testing: Requires human judgment and feedback.
○​ Exploratory/Ad-hoc testing: By definition, they are unscripted.
○​ Tests that will only be run once.

Design & Architecture for automation

●​ You don't just "record and play" scripts. You build an Automation Framework.
●​ A framework is a set of rules, tools, and libraries that provides a structure for your
automation scripts.
●​ Common Framework Architectures:
○​ Data-Driven: Separates the test script logic from the test data. (e.g., The script is
login(user, pass), and the data comes from an Excel file).
○​ Keyword-Driven: Uses a table of keywords (like "ClickButton", "VerifyText") to
define the test steps. This allows non-technical users to write tests.
○​ Hybrid: A mix of Data-Driven and Keyword-Driven (most common).
○​ Page Object Model (POM): A very popular design pattern (not a framework) for
web automation. Each page of the application is a class, and the elements on the
page (buttons, fields) are objects in that class. This makes tests much easier to
maintain when the UI changes.

Generic requirements for test tool framework

A good automation framework should be:

●​ Reusable: Easy to reuse code and functions.

 ●​ Maintainable: Easy to update when the application changes.
●​ Scalable: Can handle adding thousands of new test cases.
●​ Independent: Tests shouldn't depend on each other (one test's failure shouldn't stop the
others).
●​ Good Reporting: Must generate clear, detailed Pass/Fail reports.
●​ Integratable: Should integrate with CI/CD tools (like Jenkins) and bug trackers (like
Jira).

Test tool selection

●​ How to choose the right automation tool.
●​ Criteria:
○​ Cost: Is it open-source (e.g., Selenium) or commercial (e.g., TestComplete)?
○​ Technology Support: Does it support your application (Web, Mobile, Desktop)?
○​ Language Support: Does your team know the language (Python, Java,
JavaScript)?
○​ Skill Level: Is it easy to learn and use?
○​ Community/Vendor Support: Is there good documentation and help available?

Testing in Object Oriented Systems

●​ Testing OO-based code (like Java or C++) has unique challenges:
○​ Encapsulation: It can be hard to test private methods. You usually have to test
them indirectly via the public methods.
○​ Inheritance: If you change a parent class, you must re-test all of its child classes
(this is a regression testing challenge).
○​ Polymorphism: One method call (e.g., [Link]()) can result in different
behaviors (Dog barks, Cat meows). Test cases must be designed to check all
possible polymorphic behaviors.
○​ Unit: The "unit" in OO testing is no longer a single function; it's the entire class.
Unit tests must instantiate the class and test its methods.