1.
Fundamentals of software testing
Software testing is a crucial phase in the software engineering process. Unlike
development, which is constructive in nature, testing appears to be destructive as it
aims to find faults in the software. Robert Dunn described software as "a working
program remains an elusive thing of beauty," reflecting the difficulty of building
error-free systems. While earlier phases focus on building the product, testing
focuses on breaking it — not to destroy it, but to improve its quality by identifying
errors before the software is released.
Objectives of Testing
Glen Myers, in his book on software testing, outlines key objectives: to execute the
program with the intent of finding errors, to design test cases that have a high
probability of discovering bugs, and to measure a test’s success based on the errors
it uncovers. Testing should not be about confirming that the software works; rather,
it should aim to expose faults and weaknesses. This mindset helps in creating more
reliable software.
Test Planning and Requirement Traceability
All test cases must be clearly linked to customer requirements, as the most severe
defects are those that prevent the software from meeting user expectations.
Planning for testing should begin early—ideally right after the requirements phase.
Detailed test case design can start once the design model is complete, even before
the code is written. This early planning reduces errors in later stages and improves
overall software quality.
Testing Strategy and Pareto Principle
A commonly followed strategy is to begin testing “in the small,” focusing on
individual components or modules, and gradually move to “in the large,” which
includes integration and system testing. The Pareto principle also applies to
testing—about 80% of the bugs are often found in just 20% of the code.
Identifying and prioritizing these error-prone components improves the efficiency
of the testing process.
Limitations of Exhaustive Testing
It is practically impossible to test every path or input in a program due to the sheer
number of possibilities, especially in complex systems. However, carefully
designed test cases can provide sufficient logic coverage to ensure confidence in
the system. To improve effectiveness, testing should ideally be conducted by an
independent team rather than the developers themselves, as they are more likely to
approach the system objectively.
Testability
Meaning and Importance
Testability refers to how easily software can be tested. High testability means
defects can be detected quickly, tests can be automated, and results are easier to
observe. Improving testability leads to more effective and efficient testing.
Key Characteristics of Testable Software
Operability ensures that the software runs smoothly, which simplifies testing.
Observability allows testers to see system outputs, internal states, and errors
clearly. Controllability means testers can control inputs and execution paths,
making automation easier. Decomposability supports testing of individual
modules independently, helping isolate problems. Simplicity—both in
functionality and structure—makes testing faster and easier. Stability ensures that
frequent changes do not disrupt existing test cases. Understandability is enhanced
through good documentation and a clear system design, allowing testers to work
efficiently.
Qualities of a Good Test
A good test case is one that has a high chance of finding an error. It should not be
redundant, must be the best among alternatives, and should maintain a
balance—not too simple and not overly complex. Well-designed test cases increase
the chances of finding critical defects early in the process.
2. White box testing:
Software testing is the process of verifying and validating that a software or
application is bug-free, meets technical requirements as per its design and
development, and fulfills user requirements effectively and efficiently, including
handling of all exceptional and boundary cases.
It not only helps in identifying faults in software but also suggests ways to improve
efficiency, accuracy, and usability. The aim is to measure specifications,
functionality, and performance.
Steps in Software Testing:
1. Verification:
Ensures that the software correctly implements specific functions.
→ “Are we building the product right?”
2. Validation:
Ensures that the built software meets customer requirements.
→ “Are we building the right product?”
Types of Software Testing:
1. Manual Testing:
○ Testing is done manually without automation tools.
○ Involves stages such as Unit Testing, Integration Testing, System
Testing, and User Acceptance Testing.
○ Includes exploratory testing using test plans and test scenarios.
2. Automation Testing:
○ Scripts and tools are used to automate manual testing processes.
○ Useful for regression, load, performance, and stress testing.
○ Improves accuracy and efficiency, and reduces time and cost.
Testing Techniques:
1. Black Box Testing:
○ Tester does not have access to source code.
○ Focuses on software functionality rather than internal logic.
2. White Box Testing:
○ Tester has full knowledge of internal code and logic.
○ Ensures all internal operations are performed as expected.
Levels of Software Testing:
1. Unit Testing:
Testing individual units/components of the software.
2. Integration Testing:
Testing combined units/modules to identify interface faults.
3. System Testing:
Testing the complete integrated system for compliance with requirements.
4. Acceptance Testing:
Testing the system’s acceptability to ensure it meets business requirements.
White Box Testing
White Box Testing is a technique where the internal structure, logic, and code of
the software are tested. It is also called Clear Box, Glass Box, or Structural Testing.
Key Characteristics:
● Code is visible to the tester.
● Focus on logic, loops, conditions, paths, and statements.
● Used primarily in unit testing.
Types of White Box Testing:
● Unit Testing
● Memory Leak Testing
● White Box Penetration Testing
● White Box Mutation Testing
Advantages:
● Optimizes code by finding hidden errors.
● Test cases can be automated.
● Allows early testing in SDLC.
● Ensures complete code coverage.
Disadvantages:
● Requires programming knowledge.
● Time-consuming and complex.
● Developers may not perform it thoroughly.
Basis Path Testing
Basis Path Testing is a White Box Testing technique that uses control flow graphs
to identify independent paths through a program.
Objective: To achieve maximum path coverage using minimum test cases.
Steps:
1. Draw a control flow graph of the program.
2. Calculate Cyclomatic Complexity:
V(G)=E−N+2PV(G) = E - N + 2PV(G)=E−N+2P,
where E = edges, N = nodes, P = number of connected components.
3. Identify independent paths.
4. Generate test cases to execute each path.
Example Paths: For a program with decision nodes:
● Path 1: 1 → 2 → 3 → 5 → 6 → 7
● Path 2: 1 → 2 → 4 → 5 → 6 → 7
● Path 3: 1 → 6 → 7
Advantages:
● Reduces redundant test cases.
● Focuses on logic and flow.
● Ensures each statement is executed at least once.
Control Structure Testing
This testing focuses on validating different control structures in the code to ensure
program logic works correctly.
Types:
1. Condition Testing:
● Ensures logical and decision-making statements are error-free.
● Common conditions:
○ Relational: E1 op E2
○ Simple with NOT operator: ~E1
○ Compound conditions: (E1 & E2) | (E2 & E3)
○ Boolean expressions: A | B
2. Data Flow Testing:
● Analyzes the flow of variables through definitions and uses.
● Each statement is given a unique number.
● Definitions: DEF(S) = {X | X is defined at statement S}
Uses: USE(S) = {X | X is used at stateme
3. Loop Testing:
● Verifies the correctness of loop constructs.
Types of Loops Tested:
● Simple Loop:
○ Test for 0, 1, 2, n-1, n, n+1 iterations
● Concatenated Loops:
○ If independent, test separately like simple loops
○ If dependent, test as nested loops
● Nested Loops:
○ Start from the innermost loop and test outward.
3. Black-Box Testing
Black-box testing, also known as behavioral testing, is focused on evaluating the
functional requirements of the software without looking at its internal structure or
workings. In black-box testing, the tester is concerned only with whether the
system performs as expected from a functional perspective.
The aim is to derive test cases from the functional requirements, enabling testers to
evaluate if all the functional requirements of the software are fully exercised.
Categories of Errors in Black-box Testing
Black-box testing helps identify errors in the following areas:
1. Incorrect or Missing Functions
2. Interface Errors
3. Errors in Data Structures or External Database Access
4. Behavior or Performance Errors
5. Initialization and Termination Errors
Key Testing Questions
● How is functional validity tested?
● How are system behavior and performance tested?
● What classes of input will make good test cases?
● Is the system sensitive to certain input values?
● How are the boundaries of a data class isolated?
● What data rates and data volumes can the system tolerate?
Methods of Black-box Testing
1. Graph-Based Testing Methods
2. Equivalence Partitioning
3. Boundary Value Analysis
4. Orthogonal Array Testing
1. Graph-Based Testing Methods
Graph-based testing starts by understanding the objects modeled in software and
the relationships that connect them. Once this is accomplished, tests are defined to
verify if all objects maintain the expected relationships with each other.
● Graph Representation:
A graph consists of nodes (representing objects) and links (representing
relationships between objects).
○ Directed Links: Represent a one-way relationship.
○ Bidirectional Links: Indicate a two-way relationship.
○ Parallel Links: Used when multiple relationships exist between
nodes.
Behavioral testing methods using graphs include:
● Transaction Flow Modeling: Nodes represent transaction steps, and links
show logical connections between steps.
● Finite State Modeling: Nodes represent states, and links represent
transitions between states.
● Data Flow Modeling: Nodes represent data objects, and links represent
transformations between them.
● Timing Modeling: Nodes represent program objects, and links indicate
sequential connections with execution time requirements.
2. Equivalence Partitioning
Equivalence Partitioning divides the input domain of a program into classes of
data. From these classes, test cases can be derived.
● Equivalence Class: A set of valid or invalid inputs that should behave
similarly when processed by the software.
● Guidelines for Defining Classes:
○ If an input condition specifies a range, define one valid and two
invalid equivalence classes.
○ If a specific value is required, define one valid and two invalid
equivalence classes.
○ For a set, define one valid and one invalid equivalence class.
○ For Boolean conditions, define one valid and one invalid class.
3. Boundary Value Analysis (BVA)
Boundary Value Analysis is based on the observation that errors tend to occur at
the boundaries of input domains rather than in the center. Test cases are designed to
test the boundary values of these input conditions.
● Guidelines for BVA:
1. For a range [a, b], test values at a, b, just above a, and just below b.
2. For a set of values, test the minimum, maximum, and values just
above and below these.
3. For output conditions, test the boundaries of permissible outputs.
4. Test internal data structures at their boundary limits (e.g., a table with
a maximum of 100 entries).
Orthogonal Array Testing
Orthogonal Array Testing is a statistical method used to reduce the number of test
cases required while ensuring adequate test coverage. It systematically tests
combinations of input conditions to cover all interactions between them.
4. Regression testing:
Regression testing is conducted after any new module is added during integration
testing. It ensures that the changes made do not introduce unintended side effects
that affect previously working functionality.
Key Aspects:
● New data flow paths, I/O operations, or control logic introduced by changes
can cause issues with existing functionality.
● Regression testing involves re-executing a subset of previously run tests to
verify that the changes do not negatively impact the software.
Methods:
● Regression tests can be run manually or using automated capture/playback
tools, which record test cases and results for future comparison.
Regression Test Suite:
The suite includes three types of test cases:
1. Representative Sample Tests: These test all major functions of the
software.
2. Focused Tests: These test areas likely affected by the change.
3. Change-Specific Tests: These test the specific components that were
altered.
Challenges:
● As integration testing progresses, the number of regression tests may grow.
● The regression test suite should only include tests that address relevant
errors in major program functions, avoiding the need to re-run all tests.
Benefits and Drawbacks:
Benefits:
● Ensures stability of existing functionality after changes.
● Can be automated, saving time on repeated test execution.
Drawbacks:
● In agile development, frequent changes can lead to high overhead if
regression tests are not managed efficiently.
Uses:
● Tracks software quality, such as code size or execution time, especially in
complex software like compilers.
Criteria for Test Selection:
● Bug fixes and how they affect the system.
● Areas with frequent defects or recent code changes.
● High visibility areas that impact end-users.
Smoke Testing:
Regression testing is often combined with smoke testing. If any test case in the
smoke test fails, the build is considered unstable, and developers/managers are
notified.
5. Unit Testing and Integration Testing
Unit Testing:
● Unit testing focuses on verifying the smallest units of software—such as
individual modules or components.
● The goal is to ensure that each unit operates correctly in isolation before
integration with other parts of the software.
Unit Test Considerations:
1. Module Interface: Verify that data properly flows into and out of the unit.
2. Local Data Structure: Ensure the integrity of data stored temporarily
within the module.
3. Boundary Conditions: Test the module’s operation at established limits
(maximum and minimum values).
4. Independent Paths: Ensure that all statements in the module are executed at
least once during testing.
5. Error Handling: Test how the module handles errors, ensuring clear error
messages and correct exception handling.
Common Errors Found During Unit Testing:
● Computation Errors: Incorrect arithmetic precedence, mixed-mode
operations, or initialization issues.
● Comparison Errors: Incorrect data type comparisons, improper logical
operators, or faulty loop termination.
● Boundary Errors: Failures related to processing elements at the boundaries
of arrays or data structures.
Unit Test Procedures:
● Before Coding: Establish test cases based on design specifications, coupled
with expected results.
● Post-Coding: Develop drivers and stubs (dummy modules) to simulate the
full environment for isolated testing of the component.
● Test Environment: Typically, drivers and stubs are used during testing but
are not part of the final product. Testing can be postponed until integration if
necessary.
Integration Testing:
● Integration testing focuses on constructing the software program structure
and identifying errors at the points where different components interact.
● The aim is to ensure that the various components, which have passed unit
tests, work correctly when integrated.
Types of Integration Testing:
1. Non-Incremental (Big Bang) Integration: All components are integrated at
once, which makes isolating and correcting errors challenging.
2. Incremental Integration: The program is built and tested in small
increments, making it easier to identify and fix errors.
Top-Down Integration Testing:
● Description: Modules subordinate to the main control module are integrated
incrementally.
1. Depth-First: Integrates components along a major control path.
2. Breadth-First: Integrates all components at a given level before
moving to the next level.
● Process:
1. The main control module is tested using stubs for lower-level
modules.
2. Components are integrated in a specific sequence (depth-first or
breadth-first).
3. Regression testing may follow each integration step.
● Problems: Logistic issues may arise when upper-level modules require
lower-level modules to function, causing a delay in testing.
Bottom-Up Integration Testing:
● Description: Integration starts from the lowest-level components and works
upward.
● Process:
1. Low-level components are grouped into clusters and tested with a
driver.
2. As testing progresses, drivers are removed and clusters are integrated
upward.
● Advantages: Requires fewer stubs and simplifies testing of lower-level
modules.
Regression Testing:
● Regression testing ensures that new changes, such as bug fixes or module
additions, do not negatively affect existing functionality.
● Test Types:
1. Representative Samples: Tests that cover all major software
functions.
2. Change-Specific Tests: Tests focusing on parts likely affected by the
changes.
3. Change Component Tests: Tests the components that were directly
altered.
Benefits:
● Ensures changes do not introduce unintended errors or side effects.
● Can be automated to save time and effort.
Smoke Testing:
● Smoke testing is used to check whether the software build is stable enough
for further detailed testing.
● It is often performed on new builds to identify critical errors early in the
development process.
Benefits:
● Quick Assessment: Detects “showstopper” errors that could delay the
project.
● Improved Product Quality: Identifies both functional and architectural
defects early.
● Simplifies Error Diagnosis: Helps isolate errors related to recently added
software components.
Critical Module Identification:
● Criteria:
○ Multiple Requirements: Addresses several software requirements.
○ High Control: Has significant control over the program’s structure.
○ Complexity: Error-prone or complex in nature.
○ Performance Demands: Has specific performance needs.
Importance of Testing Critical Modules Early:
● Critical modules should be tested early in the integration process to ensure
the system works as expected, and regression tests should focus on these
modules to prevent errors from propagating.
Integration Test Documentation:
Test Specification:
● A comprehensive document that includes the overall plan for integration and
specific test cases.
● It should include:
○ Test Plan: The overall strategy for integration.
○ Test Procedures: A step-by-step approach for integration testing.
○ Test Phases: User interaction, data manipulation, display processing,
etc.
○ Performance Tests: To verify that the system performs within
acceptable parameters.
○ Expected Results: Listing of test cases and their expected outcomes.
Test Environment:
● Describes the environment in which testing will occur, including software
resources like stubs and drivers. The environment should mimic the
production setup as closely as possible to detect integration issues
effectively.
5. Validation Testing
● Objective: Ensures that the developed software meets the client’s needs and
business requirements.
● Test Plan: Includes functional, performance, documentation, usability, and
compatibility testing.
● Validation Process: After each test, either the software is accepted, or a
deficiency list is created.
● Alpha Testing: Conducted at the developer's site, where end users use the
software under observation.
● Beta Testing: Happens at end-user sites without developer supervision. The
developer fixes the issues reported by users.
● Acceptance Testing: Performed by customers to uncover errors before the
software is accepted, often in a formal, contract-driven setting.
6. System Testing
1. Recovery Testing: Ensures the system can recover from failures, either
automatically or with human intervention.
2. Security Testing: Validates that the system’s security mechanisms can
protect against attacks from external or internal threats.
3. Stress Testing: Tests the system under abnormal conditions to see how it
performs under extreme loads.
4. Performance Testing: Assesses the software’s runtime performance,
including resource utilization and system stability during full integration.
5. Deployment Testing: Verifies software functionality in different
environments and installation processes, ensuring compatibility across
platforms and configurations.
7. Debugging
Debugging is the process of identifying, analyzing, and fixing defects or problems
in software that prevent it from working as expected. It is crucial for ensuring
software quality and effectiveness before testing begins.
Debugging Process
● Execution of Test Cases: Debugging begins after a test case reveals an error
in the system.
● Outcomes: After identifying an error, there are two possible outcomes:
○ The cause is found and fixed.
○ The cause cannot be identified, leading to further investigation.
Why Debugging is Difficult
● Geographically Remote Symptoms and Causes: Symptoms may occur in
one part of the program, while the root cause lies elsewhere.
● Masking: The symptoms may disappear temporarily when another error is
corrected.
● Non-errors: Some issues arise due to human error or inaccuracies (e.g.,
round-off errors).
● Timing Issues: Problems might be caused by timing issues, such as delays
in real-time systems.
● Intermittent Problems: Problems may not occur consistently, making them
hard to reproduce, especially in embedded systems.
● Distributed Causes: In systems with multiple tasks or processors, causes of
problems may be spread across different components.
Popular Debugging Approaches
1. Brute Force Method
○ Description: This is the most basic method, involving the insertion of
print statements in the code to track variable values at various stages.
It's inefficient but useful in simple cases.
2. Backtracking
○ Description: This involves tracing the code backward from where the
error occurs to locate the origin of the issue.
3. Cause Elimination Method
○ Description: A list of potential causes for the error is created, and
each is tested to eliminate possibilities. This method systematically
narrows down the cause.
4. Program Slicing
○ Description: Program slicing narrows the search area by focusing on
parts of the code that affect a particular variable. This reduces the
complexity of tracing the error by examining only the relevant
sections of code.
8. Reverse and Forward Engineering
Forward Engineering
Forward Engineering refers to the process of developing an application based on
specific requirements. It is also known as Renovation and Reclamation. This
method requires high proficiency and skill, and it typically takes more time to
construct or develop the application since it involves creating the system from
scratch according to predefined specifications.
Reverse Engineering
Reverse Engineering, also known as backward engineering, is essentially the
opposite of forward engineering. In reverse engineering, information is gathered
from an existing application or system, which is then analyzed to extract
knowledge or its architecture. This process is faster than forward engineering as it
does not involve building a system from scratch but rather deconstructing an
existing one. Reverse engineering focuses on understanding and analyzing the
system rather than modifying it, making it a valuable tool for system analysis or
improvements.
Key Differences
● Forward Engineering: Builds the system from the ground up based on
requirements.
● Reverse Engineering: Analyzes and extracts information from an existing
system to understand its components or architecture.
These methods play important roles in the process of re-engineering, where
existing systems are either rebuilt or improved based on collected knowledge.
9. Reengineering Process Model:
Reengineering is the process of rebuilding software products with the aim of
improving functionality, performance, reliability, and maintainability. It involves
activities like inventory analysis, document restructuring, reverse engineering,
program and data restructuring, and forward engineering. The end result of
reengineering is typically an improved software product or business process.
Reengineering Process Model : The software reengineering process model defines
six activities that typically occur in a linear sequence. However, this sequence may
not always be strictly followed:
1. Inventory Analysis
This involves creating an inventory that serves as a detailed description of
the project, similar to a spreadsheet. It organizes information based on
various criteria like business importance, current maintainability, and
longevity. This helps identify candidates for reengineering and is revisited
periodically for updates.
2. Document Restructuring
Documentation is critical to the SDLC (Software Development Life Cycle).
If documentation is inadequate, it leads to restructuring. There are three
possible alternatives for document restructuring:
○ Retain weak documentation
○ Update existing documentation
○ Rewrite documentation for critical systems focusing on the essential
minimum.
3. Reverse Engineering
Reverse engineering involves analyzing a program to create a higher-level
abstraction of its design. This can involve design recovery, where tools
extract data, architectural, and procedural design information from the
existing codebase.
4. Code Restructuring
In this phase, the source code is analyzed using restructuring tools. Any
violations of structured programming constructs are corrected. The code is
restructured or rewritten using modern programming languages, and the
restructured code is tested to ensure no issues arise. Documentation is also
updated.
5. Data Restructuring
Data restructuring begins with reverse engineering the existing data
architecture. The data structures are reviewed for quality and weak data
models are reengineered. This is important as the data architecture impacts
program architecture and the algorithms involved.
6. Forward Engineering
Also known as reclamation or renovation, forward engineering involves
recovering design information from the existing source code and using it to
improve the system’s overall quality and performance.
10. BPR Model
BPR Model – Business Process Reengineering
Definition:
Business Process Reengineering (BPR) is the fundamental rethinking and radical
redesign of business processes to achieve major improvements in performance
metrics such as cost, quality, service, and speed.
Purpose of BPR:
To enhance organizational performance by improving the efficiency and
effectiveness of existing processes, along with systems and structures.
Key Components of BPR:
● Strategy
● Organization
● Processes
● Technology
● Culture
Phases of BPR:
1. Analysis Phase
2. Design Phase
3. Implementation Phase
Benefits of BPR:
● Streamlines business processes
● Reduces operational costs
● Improves profitability
● Boosts employee productivity
● Enhances customer satisfaction
7 Principles of BPR (Hammer & Champy):
1. Organize around outcomes, not tasks
2. Identify and prioritize processes for redesign
3. Integrate data processing with actual work
4. Treat dispersed resources as centralized
5. Link parallel activities instead of integrating later
6. Move decisions to where the work happens
7. Capture information once, at the source
Steps in BPR Implementation:
1. Define vision and goals
2. Form a skilled team
3. Analyze current processes
4. Redesign the processes
5. Implement the reengineered processes
Popular BPR Methodologies:
1. Hammer/Champy Methodology:
● CEO communicates the vision
● Identify and select key processes
● Analyze current vs. desired performance
● Redesign creatively
● Implement redesigned processes
2. Davenport Methodology:
● Define vision and objectives
● Select up to 15 key processes
● Set performance benchmarks
● Use IT tools
● Prototype and improve
● Full implementation
3. Manganelli/Klein Methodology:
● Set goals and prep team
● Select and study key processes
● Define IT support and work environments
● Implement reengineered processes
4. Kodak Methodology:
● Plan and organize the project
● Form team and design process model
● Redesign and pilot test
● Organization-wide implementation
● Continuously handle obstacles
Advantages of BPR:
● Tight integration between business functions
● Unified data views and consistent reporting
● Streamlined and process-oriented systems
● Rich functionality with templates and models
Disadvantages of BPR:
● Not suitable for every business (depends on size/resources)
● No immediate results; time-consuming process