MODULE 4 Testing & Maintenance

MODULE 4
TESTING & MAINTENANCE
Ch 10: Testing

Testing
The main objective of software testing is to detect and remove as many defects as possible
from a program. Although exhaustive testing is not feasible because the input domain of most
programs is extremely large, testing remains a practical approach to improve software
reliability. For example, testing a function that takes floating-point numbers as input would
require infinite combinations, making complete testing impossible. Despite this limitation,
systematic testing significantly reduces defects by identifying a large percentage of errors,
thereby improving software quality and ensuring that the software performs as expected under
various conditions.

Basic Concepts and Terminologies

Testing a program involves executing it with a carefully selected set of test inputs and observing
its behavior. If the program does not behave as expected, the input conditions and failure details
are recorded to help developers reproduce and fix the issue. This process is illustrated
conceptually in Figure 10.1, where a tester provides inputs, observes outputs, and reports
failures. Recording the conditions that cause failures is crucial for effective debugging because
some errors may only appear under specific scenarios, such as when a network connection is
active.

Key Terminologies

• Mistake: A human action that introduces a defect, such as failing to initialize a variable
or ignoring exceptional conditions like division by zero.

• Error/Fault/Bug/Defect: These terms are used interchangeably in software testing and

refer to incorrect program elements caused by mistakes during development. For
instance, calling the wrong function is an example of an error.

• Failure: An incorrect program behavior observed during execution, such as incorrect

results, crashes, or inappropriate actions (e.g., a robot failing to avoid an obstacle).
Importantly, not every error in the code causes a failure, as it may remain dormant
unless triggered by specific inputs.

Prof. Rajendra M. Jotawar MAC, AIT, Bangalore 1

MODULE 4 Testing & Maintenance

Example: A designer might overlook a requirement during the design phase, leading to its
absence in the code and causing a program error. This illustrates how mistakes in early stages
propagate into errors during implementation.

Verification versus Validation

Verification and validation (V&V) are two complementary techniques aimed at defect
detection:

• Verification: Ensures that outputs of a development phase meet the specifications of

the previous phase. It focuses on correctness at each stage, such as checking if design
documents conform to the requirements specification. Verification techniques include
reviews, simulation, formal verification, and testing of individual modules. It does not
require program execution.

• Validation: Confirms that the fully developed software meets customer requirements.
It involves executing the software and observing whether it performs as intended.
System testing is a key validation activity.
Verification emphasizes phase containment of errors, meaning defects should be identified
and corrected as early as possible. Detecting a design error during the design phase is
significantly cheaper than fixing it during testing, which would otherwise require redesign,
recoding, and retesting. Therefore, both verification and validation are essential bug filters, and
together they ensure the development of a reliable software product.

Testing Activities
The testing process consists of four major activities:

1. Test Suite Design: Creating a set of test cases based on various test case design
techniques.

2. Executing Test Cases: Running each test case and comparing actual results with
expected results. Any mismatches indicate failures.

3. Error Localization: Analyzing failure symptoms to identify the underlying erroneous

code.

4. Error Correction: Debugging the identified errors and modifying the code to eliminate
defects.

Among these activities, debugging is often the most time-consuming because it involves
analyzing program behavior and locating the root cause of defects (as shown in Figure 10.2).

Prof. Rajendra M. Jotawar MAC, AIT, Bangalore 2

MODULE 4 Testing & Maintenance

Why Design Test Cases?

A critical question in software testing is whether testing software with random inputs is
sufficient. The answer is no, because random testing is both costly and ineffective. Random
test inputs may result in redundant test cases that detect the same error multiple times without
uncovering new ones.

For example, consider a faulty code snippet for finding the maximum of two integers:

if (x > y) max = x;
else max = x;

A test suite with inputs (3,2) and (2,3) can detect the bug, but adding more random test cases
such as (4,3) or (5,1) will not reveal any additional errors. Thus, a carefully designed minimal
test suite is more effective than a large, randomly generated one.
To achieve this, systematic test case design techniques are used, primarily categorized into:

• Black-Box Testing: Focuses on functional behavior based on the program’s input-

output specification, without knowledge of its internal structure.

• White-Box Testing: Relies on an understanding of the program’s internal logic and

structure, making it possible to design tests that cover specific paths or branches in the
code.

Both approaches complement each other and are used together to ensure comprehensive
testing.

Testing in the Large versus Testing in the Small

Software testing occurs at three major levels:
• Unit Testing: Tests individual functions or modules in isolation and is referred to as
testing in the small. It is usually performed during the coding phase.

Prof. Rajendra M. Jotawar MAC, AIT, Bangalore 3

MODULE 4 Testing & Maintenance

• Integration Testing: Combines tested modules incrementally and tests them as a group
to verify their interaction.

• System Testing: Validates the entire integrated system against its requirements and is
referred to as testing in the large.

Testing modules individually before integration simplifies debugging because identifying the
source of a defect is easier in isolated modules than in an integrated system. Once all modules
are tested independently, integration and system testing ensure the overall system functions
correctly.

Unit Testing
Unit testing is the process of testing individual modules of a software system in isolation after
they have been coded and reviewed. This is typically performed by the developer who
implemented the module, usually during the coding phase. The primary objective of unit testing
is to verify that each module performs its intended functionality correctly before it is integrated
with other modules.

Before conducting unit testing, two essential preparations must be completed:

1. Designing Unit Test Cases: Test cases must be carefully designed to validate the
correctness of the module under different input conditions.

2. Developing a Test Environment: A suitable environment must be set up to execute the

module and verify its behavior.

Driver and Stub Modules

A single module cannot be tested in isolation without the supporting environment that mimics
its interaction with other modules. To create this environment, certain additional code
components are required:
• The procedures of other modules that the module under test calls.

• Non-local data structures that the module accesses.

• A procedure to call the functions of the module under test with appropriate parameters.
However, during the early stages of development, the modules that interact with the one being
tested may not yet be implemented or tested. To address this issue, driver and stub modules
are introduced to simulate the required environment and facilitate testing.

Stub Modules
A stub is a dummy procedure that simulates the behavior of a lower-level module that is called
by the module under test. A stub has the same input/output parameters as the real procedure it
represents but contains only a simplified implementation. For example, a stub may simply
return predefined values from a table rather than executing the full functionality of the actual

Prof. Rajendra M. Jotawar MAC, AIT, Bangalore 4

MODULE 4 Testing & Maintenance

module. This allows the module under test to be executed without requiring the actual
dependent modules.

Driver Modules
A driver is a piece of code that simulates a higher-level module that calls the module under
test. It is responsible for:

• Initializing and managing any non-local data structures used by the module under test.

• Invoking the functions of the module under test with appropriate test inputs.

• Capturing and displaying the output to verify correctness.

In essence, drivers provide the mechanism to test a module in isolation, while stubs provide
the behavior of dependent modules that are not yet available. Together, drivers and stubs form
the testing environment necessary for effective unit testing, as illustrated in Figure 10.3.

Black-Box Testing
Black-box testing is a testing approach in which test cases are designed by examining only the
input and output of a program, without any knowledge of its internal design or code. This
method focuses purely on verifying whether the software meets its functional specifications.
Black-box testing is widely used because it evaluates the software from the end-user’s
perspective and helps ensure that the program behaves as expected under various input
conditions.
Two important techniques are used for designing black-box test cases:

1. Equivalence Class Partitioning (ECP)

2. Boundary Value Analysis (BVA)

Equivalence Class Partitioning (ECP)

Equivalence class partitioning involves dividing the input domain of a program into a set of
equivalence classes. Each equivalence class represents a group of inputs that are expected to

Prof. Rajendra M. Jotawar MAC, AIT, Bangalore 5

MODULE 4 Testing & Maintenance

be processed similarly by the program. The main principle behind this technique is that testing
any single input from an equivalence class is as effective as testing any other input from the
same class.

Equivalence classes can be defined by analyzing the input and output data of a program. The
general rules for creating equivalence classes include:

• If the input data can be expressed as a range of values, define one valid equivalence
class and two invalid equivalence classes.
Example: For inputs in the range 1 to 10, the valid class is [1, 10], while the invalid
classes are (−∞, 0) and (11, +∞).

• If the input data consists of discrete values, define one valid equivalence class for
acceptable values and one invalid equivalence class for all other values.
Example: If the valid inputs are {A, B, C}, then the invalid equivalence class is all
values except {A, B, C}.

Example:
For a software that calculates the square root of an integer in the range 0 to 5000, the
equivalence classes are:
• Negative integers (invalid)

• Integers between 0 and 5000 (valid)

• Integers greater than 5000 (invalid)

A suitable test suite would be: {−5, 500, 6000}.

Boundary Value Analysis (BVA)

Boundary value analysis is a test design technique that focuses on the boundary values of
equivalence classes, as programming errors frequently occur at these edges. Many defects arise
because programmers mistakenly use relational operators such as < instead of <=, or vice versa.

BVA involves creating test cases using values at the boundaries of each equivalence class.

• If an equivalence class is a range of values, test cases should include its boundary
values.

• If an equivalence class consists of discrete values, boundary testing is not applicable.

Example:
For a function that calculates the square root of integers from 0 to 5000, the equivalence classes
are:

• Negative integers

• Integers from 0 to 5000

• Integers greater than 5000

The boundary value test suite would include: {0, −1, 5000, 5001}.

Prof. Rajendra M. Jotawar MAC, AIT, Bangalore 6

MODULE 4 Testing & Maintenance

This ensures that the program handles the transition between valid and invalid input values
correctly.

Summary of the Black-Box Test Suite Design Approach

The process of designing a black-box test suite can be summarized as follows:

1. Examine the program’s input and output values.

2. Identify all possible equivalence classes.

3. Select one representative test case from each equivalence class.

4. For equivalence classes defined as ranges, add test cases for their boundary values.

Black-box testing is an intuitive and effective approach for detecting functional errors. The
most critical step is the correct identification of equivalence classes. While this may be
challenging initially, with practice it becomes straightforward. Once the equivalence classes
are identified, designing test cases for equivalence classes and their boundaries becomes a
systematic and almost mechanical process.

White-Box Testing
White-box testing is a structural testing technique in which test cases are designed based on an
analysis of the source code. Unlike black-box testing, which focuses on input-output behavior,
white-box testing ensures that internal logic, code paths, and program structure are thoroughly
verified. It is commonly used during unit testing but can also be applied in other stages of
software testing to ensure that every code element functions correctly and no hidden defects
remain.

Basic Concepts
White-box testing strategies can be broadly categorized into coverage-based testing and fault-
based testing:
• Coverage-based testing focuses on ensuring that specific elements of the source code,
such as statements, branches, or paths, are executed during testing. Popular examples
include statement coverage, branch coverage, multiple condition coverage, and
path coverage. The testing criterion in this context defines the specific program
elements that must be executed to consider the test suite adequate. For example, if a
strategy aims to execute every statement at least once, its testing criterion is statement
coverage.

• Fault-based testing, on the other hand, is aimed at detecting specific classes of faults
that are likely to occur in the program. An example of this approach is mutation testing,
where intentional small changes (mutations) are made in the program to check whether
the test cases can detect them.

Prof. Rajendra M. Jotawar MAC, AIT, Bangalore 7

MODULE 4 Testing & Maintenance

White-box testing strategies are also compared based on their strength. A strategy is said to be
stronger if it covers all the program elements of a weaker strategy and includes additional
elements. If two strategies each cover unique elements not covered by the other, they are
considered complementary, meaning both should be used together for better testing coverage.

Statement Coverage
Statement coverage is one of the simplest white-box testing techniques. Its goal is to design
test cases that execute every statement in the program at least once. The idea behind this method
is straightforward: if a statement is never executed, there is no way to confirm whether it
contains a defect. For example, a line of code may contain an incorrect arithmetic operation
that would never be discovered unless executed during testing.

However, statement coverage has limitations. Executing a statement only once does not
guarantee that it works correctly for all possible input values. Despite this drawback, statement
coverage is still an intuitive and useful starting point for white-box testing.

Branch Coverage
Branch coverage (also known as edge coverage) ensures that each branch or decision point
in the program’s control flow is executed for both true and false outcomes. This means every
possible decision outcome must be tested at least once.
Branch coverage improves upon statement coverage because it tests decision-making logic and
ensures that both sides of every conditional statement are validated. This helps in detecting
logical errors that might be missed with statement coverage alone.

Multiple Condition Coverage

Multiple condition (MC) coverage is a stronger technique than branch coverage. It requires
that every individual condition in a composite logical expression be tested with both true and
false values.

For example, for the condition ((c1 AND c2) OR c3), test cases should ensure that c1, c2, and
c3 each assume true and false values independently. This approach provides a higher level of
confidence in the correctness of the code but comes at the cost of a rapidly increasing number
of test cases. For n conditions, 2ⁿ test cases may be required, making it practical only for small
n.

Path Coverage
Path coverage focuses on testing all linearly independent paths in a program. It is based on
the program’s control flow graph (CFG), which represents the flow of control between
different program statements.

Prof. Rajendra M. Jotawar MAC, AIT, Bangalore 8

MODULE 4 Testing & Maintenance

• Control Flow Graph (CFG): Each statement in the program is represented as a node,
and edges represent control transfers between statements. CFGs allow visualization of
sequence, selection (branching), and iteration (loops).

• Path: A path is a sequence of nodes and edges from the program’s entry point to an exit
point.

Since loops can create infinite paths, testing all paths is impractical. Instead, testers focus on
linearly independent paths (also called basis paths)—paths that add at least one new edge
not present in previously identified paths.
The number of linearly independent paths can be determined using McCabe’s Cyclomatic
Complexity, which provides an upper bound for the required number of tests. Path coverage
is more rigorous than statement or branch coverage, ensuring that the program’s logic is tested
thoroughly.

Data Flow-Based Testing

Data flow testing focuses on the definitions and uses of variables in a program. It involves
identifying how variables are defined (DEF) and used (USES) across different statements.

• A definition-use (DU) chain represents a link between a statement where a variable is

defined and another statement where it is used.

• All-definitions criterion: Every definition of a variable should be tested by at least one

path where that definition is used.

• All-uses criterion: Every definition-use pair must be covered.

• All definition-use paths criterion: All possible definition-use paths (excluding cycles
or including only simple cycles) should be tested.
Data flow testing is especially helpful in programs with nested conditional and loop structures
because it focuses on variable usage correctness and detects data-related errors that other
techniques may miss.

Mutation Testing
Mutation testing is a fault-based white-box testing technique that checks the effectiveness of
a test suite in detecting real programming errors. In this approach, small changes called
mutations are introduced into the program to create mutants. These mutations could include
changes like modifying arithmetic or logical operators, deleting statements, or changing
variable values.

The mutated program is then tested against the existing test suite:

• If a test case detects the mutation, the mutant is considered killed.

Prof. Rajendra M. Jotawar MAC, AIT, Bangalore 9

MODULE 4 Testing & Maintenance

• If no test case detects it, the mutant is alive, and additional tests are designed to kill it—
unless the mutant is equivalent to the original program (no observable behavioral
change).

Mutation testing is highly effective but computationally expensive because it generates many
mutants, each requiring testing. Automated tools are often used to perform mutation testing
efficiently.

Conclusion:
White-box testing is an essential part of software testing because it verifies internal code
behavior and improves software quality. Combining various strategies—such as statement
coverage, branch coverage, path coverage, data flow testing, and mutation testing—provides a
stronger guarantee of correctness and helps in identifying defects that would be missed by
black-box testing alone.

Integration Testing
Integration testing is performed after individual modules have successfully passed unit testing.
While unit testing verifies that each module works in isolation, integration testing ensures that
modules interact correctly with one another. The primary goal is to identify interface-related
errors, such as incorrect parameter passing, mismatched data types, or improper
communication between modules.

Integration testing is conducted in a planned and incremental manner. An integration plan

specifies the sequence in which modules will be combined and tested. This plan is often guided
by the module dependency graph (structure chart), which illustrates how modules call one
another. There are several common strategies for integration testing:

Big-Bang Integration Testing

In this approach, all unit-tested modules are integrated at once, and the entire system is tested
in a single step. While this approach is simple, it is impractical for large systems because
debugging becomes extremely difficult. If an error occurs, identifying the module responsible
for the failure is complex and time-consuming. Therefore, big-bang testing is usually limited
to very small projects.

Bottom-Up Integration Testing

In this approach, testing starts with the lowest-level modules. Modules that form subsystems
are integrated and tested first, and these tested subsystems are then combined to form higher-
level subsystems. This approach primarily requires test drivers (programs that simulate higher-
level modules). One major advantage is that low-level modules—often responsible for critical
operations such as I/O—are thoroughly tested at every stage. However, for large systems with
many low-level modules, this method can become complex and time-consuming.

Prof. Rajendra M. Jotawar MAC, AIT, Bangalore 10

MODULE 4 Testing & Maintenance

Top-Down Integration Testing

This approach starts with the root module and progressively integrates lower-level modules.
Program stubs are used to simulate the behavior of modules that are not yet implemented.
One advantage of this approach is that it requires only stubs, which are simpler to write than
test drivers. However, it can be challenging to adequately test top-level modules early because
many lower-level modules responsible for I/O or data processing may not yet be available.

Mixed (Sandwiched) Integration Testing

This is a hybrid of top-down and bottom-up testing. It allows integration testing to begin as
soon as any module becomes available, rather than waiting for all top-level or bottom-level
modules to be ready. Both stubs and drivers are used. This flexibility makes it one of the most
widely used approaches in real-world projects.

Phased vs. Incremental Integration Testing

• Incremental integration testing adds one module at a time to the partially integrated
system. This makes it easier to trace and debug errors since any defect is likely to be in
the most recently added module or its interface.
• Phased integration testing integrates a group of related modules together in each step.
While this requires fewer integration steps, debugging is harder because multiple
modules are added at once.

A degenerate case of phased integration is big-bang testing, where all modules are combined
in one step. Incremental integration is generally preferred because it simplifies debugging and
improves test effectiveness.

System Testing
Once all modules have been integrated successfully, system testing is conducted to validate
the complete software system against its Software Requirements Specification (SRS). Unlike
integration testing, which focuses on interfaces between modules, system testing ensures that
the overall system satisfies both functional and non-functional requirements.
System testing is independent of the development approach—whether the system is object-
oriented or procedural—because the test cases are derived from the SRS, not the code. There
are three main types of system testing based on who performs the testing:

1. Alpha Testing
Alpha testing is performed by the development team’s test engineers in-house. Its goal is to
identify and fix defects before the product is released to external users.

2. Beta Testing
Beta testing is carried out by a small group of external users (friendly customers) who use the
software in a real-world environment. Feedback from beta testing is used to fix remaining
issues before the official release.

Prof. Rajendra M. Jotawar MAC, AIT, Bangalore 11

MODULE 4 Testing & Maintenance

3. Acceptance Testing
Acceptance testing is conducted by the customer to determine whether the software meets all
specified requirements and is ready for deployment.

Before system testing begins, smoke testing is performed to ensure that the software is stable
enough for rigorous testing.

Smoke Testing
Smoke testing is a preliminary test that checks whether the basic functionalities of the software
work correctly. If the system fails smoke testing, further testing is postponed until critical issues
are fixed. For example, in a library automation system, smoke testing might verify that books
can be added, deleted, borrowed, and returned without errors.

Performance Testing
Performance testing ensures that the software meets its non-functional requirements. Various
types of performance testing include:
• Stress Testing: Evaluates system behavior under extreme conditions or peak loads. For
example, testing a system designed for 60 users by simulating 70 concurrent users.

• Volume Testing: Checks if the system can handle large volumes of data (e.g., verifying
if a compiler’s symbol table overflows with very large programs).

• Configuration Testing: Tests the system in different hardware or software

configurations to ensure compatibility.

• Compatibility Testing: Verifies whether the software integrates and communicates

correctly with external systems such as databases or servers.

• Regression Testing: Ensures that recent bug fixes or enhancements do not introduce
new defects into previously working functionality.

• Recovery Testing: Evaluates how the system responds to faults such as power failures
or resource loss, and whether it can recover gracefully.

• Maintenance Testing: Ensures that diagnostic and maintenance tools function

correctly.

• Documentation Testing: Checks if user manuals, maintenance manuals, and technical

documentation are complete and accurate.

• Usability Testing: Validates the user interface (UI) to ensure it is user-friendly and
meets the requirements outlined in the SRS.

• Security Testing: Ensures that the system is protected from unauthorized access and
vulnerabilities through techniques like penetration testing and password cracking.

Prof. Rajendra M. Jotawar MAC, AIT, Bangalore 12

MODULE 4 Testing & Maintenance

Error Seeding
Error seeding is a technique used to estimate the number of residual errors in the software and
to assess the effectiveness of the testing process. In this method, a known number of artificial
errors are deliberately inserted into the program. During testing, the number of seeded errors
detected is compared with the number of actual errors found. This data is then used to:
• Estimate the number of remaining errors in the software.

• Evaluate the effectiveness of the current testing strategy.

Conclusion:
Integration testing validates module interactions, while system testing validates the overall
system behavior. These testing levels ensure software quality, reliability, and compliance with
requirements before deployment.

Regression Testing
Regression testing is a critical software testing activity that spans across unit testing,
integration testing, and system testing. Unlike these testing levels, which focus on specific
aspects of the software (individual modules, module interfaces, or the overall system),
regression testing is a separate dimension of testing aimed at ensuring that recent changes or
fixes do not negatively impact the existing functionality of the software.

Whenever software is modified—whether it is to fix a bug, add a new feature, or remove an

obsolete function—there is a risk of introducing new defects in the unchanged parts of the
system. Regression testing addresses this risk by re-running previously executed test cases to
verify that the software continues to function as intended.

Key Objectives of Regression Testing

1. Detect unintended side effects: Ensure that code modifications do not break existing
functionality.

2. Validate bug fixes: Confirm that previously reported defects have been resolved
without affecting other areas.

3. Ensure software stability: Provide confidence that the system remains reliable after
updates.

4. Maintain software quality over iterations: Support continuous development and

integration in large-scale projects.

When Regression Testing is Performed

Regression testing is typically carried out in the following scenarios:

• After bug fixes or defect resolutions.

Prof. Rajendra M. Jotawar MAC, AIT, Bangalore 13

MODULE 4 Testing & Maintenance

• After enhancements or feature additions to an existing module.

• After code refactoring or optimization (to ensure functionality remains unchanged).

• After environment changes, such as OS updates, database upgrades, or integration

with third-party components.

• As part of regular maintenance cycles in software development.

Approaches to Regression Testing

Regression testing does not always require running the entire test suite. Instead, the scope of
testing is determined by the type and extent of changes:

1. Retest All
o Involves executing the entire existing test suite regardless of the scope of the
change.

o Guarantees complete coverage but is time-consuming and expensive,

especially for large systems.

o Commonly used in critical systems (e.g., medical or aerospace software).

2. Selective Regression Testing

o Only the tests related to the modified components or modules are re-executed.

o More cost-effective but requires careful impact analysis to identify the affected
areas.

3. Test Case Prioritization

o Test cases are prioritized based on risk, frequency of use, or critical
functionality.

o High-priority tests are executed first, followed by lower-priority ones if time

permits.

o Often used in Agile or Continuous Integration (CI) environments.

Regression Testing in Different Testing Levels

• During Unit Testing: If a single function or module is modified, regression testing
ensures that the change does not impact the module's other functionalities.

• During Integration Testing: When modules are re-integrated, regression testing

validates that the interfaces still work correctly.
• During System Testing: After any change, regression testing ensures the software as a
whole still conforms to its requirements.

Prof. Rajendra M. Jotawar MAC, AIT, Bangalore 14

MODULE 4 Testing & Maintenance

Automation in Regression Testing

Regression testing is highly suitable for automation because:

• It involves executing repetitive test cases multiple times.

• Automated regression testing tools (e.g., Selenium, JUnit, TestNG) can reduce effort
and improve speed.

• Continuous Integration (CI) pipelines often include automated regression testing to

provide rapid feedback on every change.

Example: In an e-commerce system, if the "Add to Cart" function is modified, automated

regression tests can be used to recheck core functionalities such as payment processing, order
confirmation, and discount calculations to ensure no unintended issues arise.

Challenges in Regression Testing

• Time and cost overhead for large projects if the test suite is very extensive.

• Test case maintenance, as changes in the software often require updating regression
test cases.

• Impact analysis complexity, especially in systems with numerous interdependencies.

• Tool selection and setup for automated regression testing can require significant initial
investment.

Benefits of Regression Testing

• Improves software stability and reliability.

• Detects unintended side effects early.

• Reduces the risk of introducing defects into production.

• Supports continuous development and Agile methodologies effectively.

• Builds confidence for both developers and customers before every release.

Conclusion:
Regression testing acts as a safety net that ensures software quality is preserved during every
stage of evolution. By combining selective test execution, prioritization, and automation,
regression testing enables faster and more reliable software releases, minimizing the risks
associated with frequent changes.

Prof. Rajendra M. Jotawar MAC, AIT, Bangalore 15

MODULE 4 Testing & Maintenance

Debugging
Debugging is the process of identifying, analyzing, and fixing errors (or bugs) in a software
program. After a failure has been detected during testing or execution, the next step is to locate
the faulty program statements that caused the failure and correct them.

Debugging is one of the most time-consuming activities in software development and requires
not only technical skills but also logical reasoning, patience, and experience.

Debugging Approaches
Several techniques can be used to locate the source of a software error. Each has its strengths
and weaknesses, and often, programmers use a combination of them:

1. Brute Force Method

• This is the most common but also the least efficient approach.

• How it works: Developers insert print statements (or use logging) throughout the
program to display intermediate variable values and execution points, hoping to identify
where things go wrong.

• Symbolic Debuggers: Tools such as GDB, Visual Studio Debugger, or Eclipse

Debugger enhance this approach by allowing breakpoints, watchpoints, and variable
inspections.

• Single Stepping: Developers execute one instruction at a time while checking if the
program state matches the expected result.

Advantages:

• Simple and requires minimal setup.

• Useful for small programs or quick checks.

Disadvantages:

• Time-consuming for large programs.

• May clutter code with debug statements.

• Does not scale well for complex software.

2. Backtracking
• Start from the point of failure and trace the program execution backward to the source
of the error.
• This approach works well for small, linear programs.

Prof. Rajendra M. Jotawar MAC, AIT, Bangalore 16

MODULE 4 Testing & Maintenance

• However, in large programs with many branching paths, backtracking may become
unmanageable.

Advantages:

• Logical and straightforward for localized errors.

Disadvantages:

• Difficult to apply in programs with complex control flows and multiple modules.

3. Cause Elimination Method

• Based on the principle of hypothesis and testing.
• The programmer notes the error symptoms (e.g., an incorrect variable value) and
develops a list of possible causes.
• Each suspected cause is systematically tested and eliminated until the actual cause is
identified.

• Related technique: Software Fault Tree Analysis (SFTA) – builds a logical tree to
identify root causes of the error.

Advantages:
• Reduces random trial-and-error debugging.

• More structured than brute force.

Disadvantages:

• Requires experience and deep understanding of the software’s logic.

4. Program Slicing
• A more systematic and focused method than backtracking.

• A slice of a program is defined as the set of statements that may affect the value of a
particular variable at a specific point in the program.

• By analyzing only the relevant slice, the search space is reduced.

Advantages:

• Highly effective for large programs.

• Reduces unnecessary examination of unrelated code.

Disadvantages:
• Requires tools or frameworks to perform automated slicing effectively.

Prof. Rajendra M. Jotawar MAC, AIT, Bangalore 17

MODULE 4 Testing & Maintenance

Debugging Guidelines
Effective debugging is not just about finding errors but also preventing the introduction of new
ones. Some general guidelines include:

1. Understand the Program Design

o Debugging is faster and more accurate if the programmer has a clear

understanding of the software’s architecture, logic, and data flow.

o Partial knowledge often leads to wasted effort.

2. Do Not Fix Only the Symptoms

o Novice programmers often fix error symptoms (e.g., changing a variable value)
without addressing the underlying root cause.

o This may result in temporary fixes or additional errors.

3. Use Automated Debugging Tools

o Tools like IDEs with built-in debuggers, static analyzers, and profiling tools
can speed up error detection.

4. Regression Testing After Fixes

o Every bug fix has the potential to introduce new bugs.

o After every debugging session, regression testing should be performed to

ensure existing functionality remains intact.

5. Debug in Small Increments

o Modify and test one section of code at a time.

o Large simultaneous changes make it harder to identify the source of new errors.

6. Keep Detailed Logs

o Logging is an important part of debugging, especially for production

environments where interactive debugging may not be possible.

7. Collaborate When Necessary

o Sometimes a fresh perspective from another developer can help identify errors
faster.

Real-World Example
Suppose a banking application shows negative balances for certain transactions.

• Brute Force: Add print statements to track balance updates.

• Backtracking: Trace from the failed transaction back to the point where the balance
was updated incorrectly.

Prof. Rajendra M. Jotawar MAC, AIT, Bangalore 18

MODULE 4 Testing & Maintenance

• Cause Elimination: Test if it’s a concurrency issue, incorrect fee calculation, or data
type overflow.

• Program Slicing: Focus only on the code slice that updates the balance variable.

Conclusion:
Debugging is an iterative and structured process that should be combined with regression
testing and best practices. Using a mix of debugging approaches, tools, and guidelines
significantly improves software reliability and reduces error correction costs.

Program Analysis Tools

Program analysis tools are automated software tools that evaluate a program’s quality and
characteristics. These tools help developers, testers, and project managers identify potential
problems early and ensure adherence to coding standards.
A program analysis tool can analyze either:
• Source Code (before execution), or

• Executable Code (during or after execution).

These tools typically provide reports on:

• Program size and complexity

• Adequacy of comments and documentation

• Adherence to coding standards

• Potential errors or bad practices

• Test coverage and adequacy

Types of Program Analysis Tools

Program analysis tools are classified into two major categories:

1. Static Analysis Tools (analyze without execution)

2. Dynamic Analysis Tools (analyze during execution)

Static Analysis Tools

Static program analysis tools evaluate the program without executing it. They examine the
source code to compute various metrics and detect issues before runtime.
Key Functions of Static Analysis Tools

• Check adherence to coding standards

Prof. Rajendra M. Jotawar MAC, AIT, Bangalore 19

MODULE 4 Testing & Maintenance

o Ensures proper naming conventions, indentation, and code structure.

• Identify programming errors such as:

o Uninitialized variables

o Mismatched function parameters

o Variables declared but never used

o Dead code

• Compute software metrics like:

o Size metrics: Lines of Code (LOC)

o Cyclomatic complexity: Measures program complexity based on decision

paths.

o Halstead metrics: Measures effort, difficulty, and volume of a program.

• Generate Kiviat (Radar) Charts

o Summarizes metrics visually (e.g., complexity, comments percentage, LOC).

• Code conformance review

o Automated checks for compliance with development guidelines.

Example: Tools like SonarQube, Pylint, and ESLint perform static analysis for code quality.

Limitations of Static Analysis

• Cannot analyze runtime behavior (e.g., memory leaks, concurrency issues).

• Struggles with dynamic memory references (e.g., pointer arithmetic in C/C++).

• Requires manual interpretation of results.

Dynamic Analysis Tools

Dynamic program analysis tools evaluate software during execution to study its runtime
behavior.

Key Functions of Dynamic Analysis Tools

• Execution trace collection:

o Code is instrumented to log runtime details (variable values, executed

branches).

• Coverage Analysis:

o Measures statement, branch, and path coverage.

o Ensures all parts of the code are tested.

• Performance Analysis:

Prof. Rajendra M. Jotawar MAC, AIT, Bangalore 20

MODULE 4 Testing & Maintenance

o Identifies performance bottlenecks and memory leaks.

• Test Optimization:

o Detects redundant test cases.

o Helps design additional test cases for uncovered paths.

• Evidence generation:

o Produces reports (histograms or pie charts) to demonstrate adequate testing.

Example: Tools like JaCoCo (Java), Valgrind (C/C++), and Clover provide dynamic
analysis capabilities.

Benefits of Dynamic Analysis

• Detects runtime errors (e.g., null pointer dereferences, memory corruption).

• Provides empirical data about program behavior.

• Helps in validating the effectiveness of a test suite.

Static vs Dynamic Analysis Tools (Comparison)

Aspect Static Analysis Dynamic Analysis

Execution Analyzes code without running it Analyzes code while running it

Focus Code structure, style, metrics Runtime behavior, test coverage

Output Code quality reports, Kiviat charts Coverage reports, performance data

Detects Syntax violations, unused variables Memory leaks, concurrency issues

Tools SonarQube, ESLint, Pylint Valgrind, JaCoCo, Clover

Conclusion
Program analysis tools improve software quality by combining static checks for code
correctness with dynamic analysis for runtime validation. Using both approaches ensures
early detection of errors, better test coverage, and overall maintainability of software.

Symbolic Execution
Symbolic Execution is a program analysis technique that involves executing a program using
symbolic inputs instead of actual data values. Instead of assigning concrete values to program
variables, symbolic execution represents inputs as symbolic variables and then determines how
these symbolic inputs propagate through the program. This allows the tool to generate

Prof. Rajendra M. Jotawar MAC, AIT, Bangalore 21

MODULE 4 Testing & Maintenance

expressions for program paths and automatically derive test cases that cover different program
paths.

During symbolic execution, the program is explored path by path. For every branch condition,
the tool evaluates the condition using symbolic variables and generates a path condition—a
set of constraints that must be satisfied for that path to be executed. A constraint solver is then
used to determine whether a feasible set of input values exists for these constraints.
For example, if a branch has a condition if (x > 10), the symbolic executor will generate two
paths: one with the constraint x > 10 and another with x ≤ 10. These constraints are later solved
to produce concrete test inputs.

Advantages of Symbolic Execution

• Automated test generation: Helps in generating precise test cases to achieve high path
coverage.

• Bug detection: Detects runtime errors like assertion violations or division by zero early.
• Increased confidence in software correctness: Ensures that many execution paths are
validated systematically.
Limitations of Symbolic Execution
• Path explosion problem: The number of possible paths grows exponentially with
program complexity.
• Handling external dependencies: Difficulties arise when analyzing programs with
complex libraries or system calls.

• Resource-intensive: Requires significant memory and CPU resources for constraint

solving in large programs.

Applications of Symbolic Execution

• Unit test generation and test coverage improvement.

• Formal verification of security-critical software.

• Bug localization by examining infeasible paths and constraint failures.

Model Checking
Model Checking is a formal verification technique used to verify whether a system model
satisfies a given specification, typically expressed in temporal logic. Unlike symbolic
execution, which focuses on analyzing program paths, model checking systematically explores
all possible states of a system to ensure correctness.
In model checking, the system is first represented as a state-transition model (e.g., finite state
machines). The properties to be verified are then expressed using formal specifications, such
as Linear Temporal Logic (LTL) or Computational Tree Logic (CTL). The model checker
automatically traverses all states and verifies whether the properties hold. If a property is

Prof. Rajendra M. Jotawar MAC, AIT, Bangalore 22

MODULE 4 Testing & Maintenance

violated, the tool provides a counterexample trace showing the sequence of events that lead
to the violation, which greatly helps in debugging.

For example, in a concurrent program, a model checker might verify that “it is never possible
for two threads to access the same resource simultaneously,” ensuring mutual exclusion.

Advantages of Model Checking

• Exhaustive verification: Ensures that every possible system state is checked.

• Automatic counterexample generation: Provides concrete traces for debugging.

• Applicable to concurrency and real-time systems: Effectively detects deadlocks,

race conditions, and safety violations.

Limitations of Model Checking

• State explosion problem: The number of states grows exponentially with system
complexity.

• Abstraction needed: Requires careful abstraction to simplify the model without losing
accuracy.
• Tool dependence: Effective model checking depends on powerful tools such as SPIN,
NuSMV, or UPPAAL.

Applications of Model Checking

• Verification of communication protocols and distributed systems.

• Safety-critical software verification in aerospace, automotive, and medical domains.

• Detecting concurrency issues such as deadlocks or race conditions.

Conclusion
• Symbolic Execution is best suited for generating path-specific test cases and analyzing
smaller program units.

• Model Checking is preferred for exhaustive verification of system-wide behavior,

especially for concurrent or safety-critical applications.

• Both techniques complement each other and are widely used in modern software
verification to improve reliability and correctness.

Prof. Rajendra M. Jotawar MAC, AIT, Bangalore 23

MODULE 4 Testing & Maintenance

Question Bank Module 4

Sl. Q.
Question Marks BL
No. No.

1 Q1 Define mistake, error, defect, and failure with suitable examples. 8 BL1

2 Q2 Differentiate between Verification and Validation with examples. 8 BL2

3 Q3 Explain drivers and stubs used in unit testing with an example. 8 BL3

What is equivalence class partitioning (ECP)? Illustrate with

4 Q4 8 BL3
example.

5 Q5 Explain Boundary Value Analysis (BVA) with a suitable example. 8 BL3

List and explain the different levels of testing (unit, integration,

6 Q6 8 BL2
system).

7 Q7 Compare black-box testing and white-box testing with examples. 8 BL3

8 Q8 Explain statement coverage and branch coverage with examples. 8 BL4

What is regression testing? State its objectives and when

9 Q9 8 BL4
performed.

Write short notes on: (a) Alpha, Beta, and Acceptance testing (b)
10 Q10 8 BL3
Smoke testing

With examples, explain the major activities of testing (test suite

11 Q11 10 BL4
design, execution, error localization, error correction).

Discuss white-box testing strategies: statement, branch, multiple

12 Q12 10 BL4
condition, path coverage.

Explain path coverage using CFG. Define McCabe’s Cyclomatic

13 Q13 10 BL4
Complexity.

Explain data flow-based testing with DU chain, all-definitions,

14 Q14 10 BL4
and all-uses criteria.

What is mutation testing? Explain how it is performed with an

15 Q15 10 BL4
example.

Prof. Rajendra M. Jotawar MAC, AIT, Bangalore 24