SOFTWARE ENGINEERING

S U S H R E E S ATA PAT H Y
D E PA RT M E N T O F C O M P U T E R S C I E N C E
S I L I C O N U N I V E R S I T Y, B H U B A N E S WA R
 Module 4
SOFTWARE TESTING
 Syllabus
Module Topics
Module – 1 Software Engineering: Introduction, Evolving role of software, Legacy software, Software myths, Process
framework, CMM, Life-cycle models, Waterfall model, Incremental models, Evolutionary models,
Specialized models, Unified process, Agile Process Models (Extreme programming, Crystal, Scrum)
Module – 2 Requirements Engineering: Types of Requirements, Functional and non-functional requirements, The
software requirements document, Requirements - specification, engineering processes, elicitation &
analysis, validation, and management; Decision Trees and Decision Tables, Formal Specification
(Axiomatic specs for Stacks & Queues)
Module – 3 Software Project Management: Software project planning process, Project estimation (Cost, Time, Effort),
Decomposition techniques, Empirical estimation models, The Make/Buy decision, Project scheduling, Task
network, Critical Path method, PERT Scheduling, Earned Value analysis
Module – 4 Design Engineering: Function-oriented Software Design (DFD Structure charts), Object-oriented Design
using UML, User Interface design; Software Testing: Testing strategies, Types of testing, Black-Box testing
, White-box testing, Basis Path testing, Control Structure testing, Reliability testing, Security testing.
Module – 5 Advanced Topics: Testing web-apps, Formal methods, Risk Management, Configuration Management, Re-
Engineering Security Engineering.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 3

Coding and Testing
•Coding is undertaken once the design phase is complete and the design
documents have been successfully reviewed.

•During unit testing, each module is tested in isolation from other modules.

•After all the modules of a system have been coded and unit tested, the
integration and system testing phase is undertaken.
•

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 4

Coding and Testing
•Integration and testing of modules is carried out according to an integration
plan.
•During each integration step, a number of modules are added to the partially
integrated system and the resultant system is tested.
•The full product takes shape only after all the modules have been integrated
together.
•System testing is conducted on the full product. During system testing, the
product is tested against its requirements as recorded in the SRS document.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 5

Coding
•The objective of the coding phase is to transform the design of a system into
code in a high-level language, and then to unit test this code.
•It is mandatory for the programmers to follow the coding standards. Compliance
of their code to coding standards is verified during code inspection. Any code
that does not conform to the coding standards is rejected during code review
and the code is reworked by the concerned programmer. In contrast, coding
guidelines provide some general suggestions regarding the coding style to be
followed but leave the actual implementation of these guidelines to the
discretion of the individual developers.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 6

Code review
•Testing is an effective defect removal mechanism.
•However, testing is applicable to only executable code.
•Review is a very effective technique to remove defects from source code.
•In fact, review has been acknowledged to be more cost-effective in removing
defects as compared to testing.
•Code review for a module is undertaken after the module successfully compiles.
That is, all the syntax errors have been eliminated from the module.
•Code review has been recognised as an extremely cost-effective strategy for
eliminating coding errors and for producing high quality code as it directly
detects and eliminated errors.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 7

Code review
•Types of code reviews
•Code walkthrough – the main objective of code walkthrough is to discover the
algorithmic and logical errors in the code.

•Code inspection – The principal aim of code inspection is to check for the
presence of some common types of errors that usually creep into code due to
programmer mistakes and oversights and to check whether coding standards
have been adhered to.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 8

Testing
•The aim of program testing is to help realiseidentify all defects in a program.
•However, in practice, even after satisfactory completion of the testing phase, it
is not possible to guarantee that a program is error free.
•This is because the input data domain of most programs is very large, and it is
not practical to test the program exhaustively with respect to each value that
the input can assume.
•However, careful testing can expose a large percentage of the defects existing in
a program, and therefore provides a practical way of reducing defects in a
system.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 9

How to test a program?
•Testing a program involves executing the program with a set of test inputs and
observing if the program behaves as expected.
•If the program fails to behave as expected, then the input data and the
conditions under which it fails are noted for later debugging and error
correction.

•The tester has been shown as a stick icon, who inputs several test data to the
system and observes the outputs produced by it to check if the system fails on
some specific inputs. Unless the conditions under which a software fails are
noted down, it becomes difficult for the developers to reproduce a failure
observed by the testers.
SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 10
Terminologies
•A mistake is essentially any programmer action that later shows up as an
incorrect result during program execution.
•A programmer may commit a mistake in almost any development activity.
•For example, during coding a programmer might commit the mistake of not
initializing a certain variable, or might overlook the errors that might arise in
some exceptional situations such as division by zero in an arithmetic operation.
•Both these mistakes can lead to an incorrect result.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 11

Terminologies

•An error is the result of a mistake committed by a developer in any of the

development activities.
•One example of an error is a call made to a wrong function.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 12

Terminologies
•A failure of a program essentially denotes an incorrect behaviour exhibited by
the program during its execution.
•An incorrect behaviour is observed either as an incorrect result produced or as
an inappropriate activity carried out by the program.
•Every failure is caused by some bugs present in the program.
•In other words, we can say that every software failure can be traced to some
bug or other present in the code.
•Example
• The result computed by a program is 0, when the correct result is 10.
• A program crashes on an input.
• A robot fails to avoid an obstacle and collides with it.
SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 13
Terminologies
•Failure

•In the above code, if the variable roll assumes zero or some negative value
under some circumstances, then an array index out of bound type of error
would result.
•Positive values will cause no failures
•Thus, even if an error is present in the code, it does not show up as an error
since it is unreachable for normal input values.
SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 14
 Terminologies
•A test case is a triplet [I , S, R], where
• I is the data input to the program under test,
• S is the state of the program at which the data is to be input, and
• R is the result expected to be produced by the program.
•The state of a program is also called its execution mode.
•A test case is a set of test inputs, the mode in which the input is to be applied, and the
results that are expected during and after the execution of the test case.
•As an example, consider the different execution modes of a certain text editor software. The text editor can
at any time during its execution assume any of the following execution modes—edit, view, create, and
display.
•[input: “abc”, state: edit, result: abc is displayed], which essentially means that the input abc needs to be
applied in the edit mode, and the expected result is that the string a b c would be displayed.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 15

Terminologies
•A test scenario is an abstract test case in the sense that it only identifies the
aspects of the program that are to be tested without identifying the input, state,
or output.
•A test case can be said to be an implementation of a test scenario.
•In the test case, the input, output, and the state at which the input would be
applied is designed such that the scenario can be executed.
•An important automatic test case design strategy is to first design test scenarios
through an analysis of some program abstraction (model) and then implement
the test scenarios as test cases.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 16

Terminologies

•A test script is an encoding of a test case as a short program. Test scripts are
developed for automated execution of the test cases.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 17

Terminologies
•A test case is said to be a positive test case if it is designed to test whether the
software correctly performs a required functionality.
•A test case is said to be negative test case, if it is designed to test whether the
software carries out something, that is not required of the system.
•As one example each of a positive test case and a negative test case, consider a
program to manage user login. A positive test case can be designed to check if a
login system validates a user with the correct user name and password. A
negative test case in this case can be a test case that checks whether the the
login functionality validates and admits a user with wrong or bogus login user
name or password.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 18

Terminologies
•A test suite is the set of all test that have been designed by a tester to test a
given program.

•Testability of a requirement denotes the extent to which it is possible to

determine whether an implementation of the requirement conforms to it in
both functionality and performance. In other words, the testability of a
requirement is the degree to which an implementation of it can be adequately
tested to determine its conformance to the requirement.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 19

Terminologies
•A failure mode of a software denotes an observable way in which it can fail. In
other words, all failures that have similar observable symptoms, constitute a
failure mode. As an example of the failure modes of a software, consider a
railway ticket booking software that has three failure modes—failing to book an
available seat, incorrect seat booking (e.g., booking an already booked seat), and
system crash.
•Equivalent faults denote two or more bugs that result in the system failing in the
same failure mode. As an example of equivalent faults, consider the following
two faults in C language—division by zero and illegal memory access errors.
These two are equivalent faults, since each of these leads to a program crash.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 20

Verification vs Validation
•Similarity : both have the objective to help remove errors in a software.

Verification Validation
Aim phase containment of errors to check whether the deliverable
software is error free.

Objective Determines whether the Determines whether a fully

output/work product of one developed software conforms to
phase of software development its requirements specification.
conforms to that of its previous
phase
Technique Informal techniques like review, primarily based on product
simulation, formal verification, testing.
and testing. Formal verification
usually involves use of theorem
proving techniques or use of
automated tools such as a model
checker.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 21

Verification vs Validation
Verification Validation
Type of testing unit and integration testing can system testing can be considered
be considered as verification as a validation step
steps
Need of execution does not require execution of the requires execution of the
software software

When to do carried out during the carried out to check if the right as
development process to check if required by the customer has
the development activities are been
proceeding alright developed.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 22

Testing activities
•Test suite design: The set of test cases using which a program is to be tested is
designed possibly using several test case design techniques.
•Running test cases and checking the results to detect failures: Each test case is
run and the results are compared with the expected results. A mismatch
between the actual result and expected results indicates a failure. The test cases
for which the system fails are noted down for later debugging.
•Locate error: In this activity, the failure symptoms are analysed to locate the
errors. For each failure observed during the previous activity, the statements
that are in error are identified.
•Error correction: After the error is located during debugging, the code is
appropriately changed to correct the error.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 23

Testing activities

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 24

Why design test cases?
•When test cases are designed based on random input data, many of the test
cases do not contribute to the significance or effectiveness of the test suite, That
is, they do not help detect any additional defects not already being detected by
other test cases in the suite.
•This code segment has a simple programming error:
• if (x>y) max = x;
• else max = x;
•For the given code segment, the test suite {(x=3,y=2);(x=2,y=3)} can detect the
error, whereas a larger test suite {(x=3,y=2);(x=4,y=3); (x=5,y=1)} does not detect
the error. All the test cases in the larger test suite help detect the same error,
while the other error in the code remains undetected.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 25

Why design test cases?
•exhaustive testing of almost any nontrivial system is impractical due to the fact
that the domain of input data values to most practical software systems is either
extremely large or countably infinite.

•A minimal test suite is a carefully designed set of test cases such that each test
case helps detect different errors. This is in contrast to testing using some
random input values.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 26

Approaches of test case design
•There are essentially two main approaches to systematically design test cases:
• Black-box approach
• White-box (or glass-box) approach

Black-box approach White-box approach

Design designed solely based on Designed based on the analysis of
an analysis of the input/out the code
behaviour (that is, functional
behaviour)
Knowledge of internal structure Does not require Requires
Also called Functional testing Structural testing

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 27

Levels of testing
• Unit testing
• Integration testing
• System testing

•During unit testing, the individual functions (or units) of a program are tested.
•Unit testing is referred to as testing in the small, whereas integration and system
testing are referred to as testing in the large.
•After testing all the units individually, the units are slowly integrated and tested
after each step of integration (integration testing).
•Finally, the fully integrated system is tested (system testing).

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 28

Unit testing
•Unit testing is undertaken after a module has been coded and reviewed.
•This activity is typically undertaken by the coder of the module himself in the
coding phase.
•Before carrying out unit testing, the unit test cases have to be designed and the
test environment for the unit under test has to be developed.
•Why unit testing?
• other modules with which this module needs to interface may not be ready.
• good idea to first test the module in isolation before integration because it makes debugging
easier

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 29

Driver and Stub Modules
•In order to test a single module, we need a complete environment to provide all
relevant code that is necessary for execution of the module.
•That is, besides the module under test, the following are needed to test the
module:
• The procedures belonging to other modules that the module under test calls.
• Non-local data structures that the module accesses.
• A procedure to call the functions of the module under test with appropriate parameters.

•Modules required to provide the necessary environment (which either call or

are called by the module under test) are usually not available until they too have
been unit tested.
•In this context, stubs and drivers are designed to provide the complete
environment for a module so that testing can be carried out.
SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 30
Driver and Stub Modules
•A stub procedure is a dummy procedure that has
the same I/O parameters as the function called
by the unit under test but has a highly simplified
behaviour.
•For example, a stub procedure may produce the
expected behaviour using a simple table look up
mechanism.
•A driver module should contain the non-local
data structures accessed by the module under
test. Additionally, it should also have the code to
call the different functions of the unit under test
with appropriate parameter values for testing.
SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 31
Black-box Testing
•In black-box testing, test cases are designed from an examination of the
input/output values only and no knowledge of design or code is required.
•The following are the two main approaches available to design black box test
cases:
•Equivalence class partitioning
•Boundary value analysis

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 32

Equivalence Class Partitioning
•In the equivalence class partitioning approach, the domain of input values to the
program under test is partitioned into a set of equivalence classes.

•The partitioning is done such that for every input data belonging to the same
equivalence class, the program behaves similarly.

•The main idea behind defining equivalence classes of input data is that testing
the code with any one value belonging to an equivalence class is as good as
testing the code with any other value belonging to the same equivalence class.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 33

Equivalence Class Partitioning
•The following are two general guidelines for designing the equivalence classes:
1. If the input data values to a system can be specified by a range of values, then
one valid and two invalid equivalence classes need to be defined. For
example, if the equivalence class is the set of integers in the range 1 to 10
(i.e., [1,10]), then the invalid equivalence classes are [−∞,0], [11,+∞].
2. If the input data assumes values from a set of discrete members of some
domain, then one equivalence class for the valid input values and another
equivalence class for the invalid input values should be defined. For example,
if the valid equivalence classes are {A,B,C}, then the invalid equivalence class
is 􀀀 -{A,B,C}, where 􀀀 is the universe of possible input values.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 34

Equivalence Class Partitioning
•Example For a software that computes the square root of an input integer that
can assume values in the range of 0 and 5000. Determine the equivalence
classes and the black box test suite.
•Answer: There are three equivalence classes—The set of negative integers, the
set of integers in the range of 0 and 5000, and the set of integers larger than
5000. Therefore, the test cases must include representatives for each of the
three equivalence classes. A possible test suite can be: {–5,500,6000}.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 35

Equivalence Class Partitioning
•Example 10.7 Design the equivalence class test cases for a program that reads
two integer pairs (m1, c1) and (m2, c2) defining two straight lines of the form
y=mx+c. The program computes the intersection point of the two straight lines
and displays the point of intersection.
•Answer: The equivalence classes are the following:
•Parallel lines (m1 = m2, c1 ≠ c2)
•Intersecting lines (m1 ≠ m2)
•Coincident lines (m1 = m2, c1 = c2)
•Selecting one representative value from each equivalence class, we get the
required equivalence class test suite {(2,2)(2,5),(5,5)(7,7), (10,10) (10,10)}.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 36

Equivalence Class Partitioning
•Example Design equivalence class
partitioning test suite for a function that
reads a character string of size less than five
characters and displays whether it is a
palindrome.
•Answer: The equivalence classes are the
leaf level classes shown in Figure. The
equivalence classes are palindromes, non-
palindromes, and invalid inputs. Now,
selecting one representative value from
each equivalence class, we have the
required test suite: {abc,aba,abcdef}.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 37

Boundary Value Analysis
•A type of programming error that is frequently committed by programmers is
missing out on the special consideration that should be given to the values at
the boundaries of different equivalence classes of inputs.
•Programmers often fail to properly address the special processing required by
the input values that lie at the boundary of the different equivalence classes. For
example, programmers may improperly use < instead of <=, or conversely <= for
<, etc.
•Boundary value analysis-based test suite design involves designing test cases
using the values at the boundaries of different equivalence classes.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 38

Boundary Value Analysis
•To design boundary value test cases, it is required to examine the equivalence
classes to check if any of the equivalence classes contains a range of values.
•For those equivalence classes that are not a range of values (i.e., consist of a
discrete collection of values) no boundary value test cases can be defined. For
an equivalence class that is a range of values, the boundary values need to be
included in the test suite.
•For example, if an equivalence class contains the integers in the range 1 to 10,
then the boundary value test suite is {0,1,10,11}.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 39

Boundary Value Analysis
•Example For a function that computes the square root of the integer values in
the range of 0 and 5000, determine the boundary value test suite.
•Answer: There are three equivalence classes—The set of negative integers, the
set of integers in the range of 0 and 5000, and the set of integers larger than
5000. The boundary value-based test suite is: {0,-1,5000,5001}.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 40

 Summary of the Black-box Test Suite
Design Approach
•Examine the input and output values of the program.
•Identify the equivalence classes.
•Design equivalence class test cases by picking one representative value from
each equivalence class.
•Design the boundary value test cases as follows. Examine if any equivalence
class is a range of values. Include the values at the boundaries of such
equivalence classes in the test suite.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 41

White-box testing
•an important type of unit testing
•A white-box testing strategy can either be coverage-based or fault-based.
•Fault-based testing
•A fault-based testing strategy targets to detect certain types of faults.
•These faults that a test strategy focuses on constitutes the fault model of the
strategy.
•Coverage-based testing
•A coverage-based testing strategy attempts to execute (or cover) certain
elements of a program.
•Popular examples of coverage-based testing strategies are statement coverage,
branch coverage, multiple condition coverage, and path coverage-based testing.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 42

Testing criteria for coverage-based testing
•A coverage-based testing strategy typically targets to execute (i.e., cover) certain
program elements for discovering failures.
•The set of specific program elements that a testing strategy targets to execute is
called the testing criterion of the strategy.
•For example, if a testing strategy requires all the statements of a program to be
executed at least once, then we say that the testing criterion of the strategy is
statement coverage. We say that a test suite is adequate with respect to a
criterion, if it covers all elements of the domain defined by that criterion.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 43

 Stronger vs Weaker Testing
•A white-box testing strategy is said to be
stronger than another strategy, if the
stronger testing strategy covers all program
elements covered by the weaker testing
strategy, and the stronger strategy
additionally covers at least one program
element that is not covered by the weaker
strategy.
•If a stronger testing has been performed,
then a weaker testing need not be carried
out.
•When none of two testing strategies fully
covers the program elements exercised by
the other, then the two are called
complementary testing strategies.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 44

Statement Coverage
•The statement coverage strategy aims to design test cases so as to execute every
statement in a program at least once.
•The principal idea governing the statement coverage strategy is that unless a
statement is executed, there is no way to determine whether an error exists in
that statement.
•It is obvious that without executing a statement, it is difficult to determine
whether it causes a failure due to illegal memory access, wrong result
computation due to improper arithmetic operation, etc.
•It can however be pointed out that a weakness of the statement- coverage
strategy is that executing a statement once and observing that it behaves
properly for one input value is no guarantee that it will behave correctly for all
input values.
SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 45
Statement Coverage
•Example Design statement coverage-based test suite for the following Euclid’s GCD
computation program:
int computeGCD(x,y)
int x,y;
{
1 while (x != y) {
2 if (x>y) then
3 x=x-y;
4 else y=y-x;
5}
6 return x;
}
•Answer: To design the test cases for the statement coverage, the conditional expression
of the while statement needs to be made true and the conditional expression of the if
statement needs to be made both true and false. By choosing the test set {(x = 3, y = 3),
(x = 4, y = 3), (x = 3, y = 4)}, all statements of the program would be executed at least
once.
SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 46
Branch Coverage
•A test suite satisfies branch coverage, if it makes each branch condition in the
program to assume true and false values in turn.
•In other words, for branch coverage each branch in the CFG representation of
the program must be taken at least once, when the test suite is executed.
•Branch testing is also known as edge testing, since in this testing scheme, each
edge of a program’s control flow graph is traversed at least once.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 47

Branch Coverage
•Example Design branch coverage-based test suite for the following Euclid’s GCD
computation program:
int computeGCD(x,y)
int x,y;
{
1 while (x != y) {
2 if (x>y) then
3 x=x-y;
4 else y=y-x;
5}
6 return x;
}
•Answer: The test suite {(x = 3, y = 3), (x = 3, y = 2), (x = 4, y = 3), (x = 3, y = 4)} achieves
branch coverage.
SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 48
Branch Coverage vs Statement Coverage
•It is easy to show that branch coverage-based testing is a stronger testing than
statement coverage-based testing. We can prove this by showing that branch coverage
ensures statement coverage, but not vice versa.
•(a) Branch testing would guarantee statement coverage since every statement must
belong to some branch (assuming that there is no unreachable code).
•(b) To show that statement coverage does not ensure branch coverage, it would be
sufficient to give an example of a test suite that achieves statement coverage, but does
not cover at least one branch. Consider the following code, and the test suite {5}.
•if(x>2) x+=1;
•The test suite would achieve statement coverage. However, it does not achieve branch
coverage, since the condition (x > 2) is not made false by any test case in the suite.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 49

Multiple Condition Coverage
•In the multiple condition (MC) coverage-based testing, test cases are designed
to make each component of a composite condition in an expression to assume
both true and false values.
•For example, consider the composite conditional expression ((c1 .and.c2 ).or.c3).
•A test suite would achieve MC coverage, if all the component conditions c1, c2
and c3 are each made to assume both true and false values.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 50

 Multiple Condition Coverage vs
Branch Coverage
•Branch testing can be considered to be a simplistic condition testing strategy
where only the compound conditions appearing in the different branch
statements are made to assume the true and false values.
•It is easy to prove that condition testing is a stronger testing strategy than
branch testing. For a composite conditional expression of n components, 2n test
cases are required for multiple condition coverage.
•Thus, for multiple condition coverage, the number of test cases increases
exponentially with the number of component conditions. Therefore, multiple
condition coverage-based testing technique is practical only if n (the number of
conditions) is small.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 51

Path Coverage
•A test suite achieves path coverage if it executes each linearly independent
paths ( or basis paths ) at least once.
•A linearly independent path can be defined in terms of the control flow graph
(CFG) of a program.

•Control flow graph (CFG)

•A control flow graph describes how the control flows through the program.
•We can define a control flow graph as the following:
• A control flow graph describes the sequence in which the different
instructions of a program get executed.
• In order to draw the control flow graph of a program, we need to first number
all the statements of a program. The different numbered statements serve as
nodes of the control flow graph. There exists an edge from one node to
another, if the execution of the statement representing the first node can
result in the transfer of control to the other node.
SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 52
 CFG
•A CFG is a directed graph consisting of
a set of nodes and edges (N, E), such
that each node n є N corresponds to a
unique program statement and an
edge exists between two nodes if
control can transfer from one node to
the other.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 53

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 54
Path
•A path through a program is any node and edge sequence from the start node to
a terminal node of the control flow graph of a program.
•A program can have more than one terminal nodes when it contains multiple
exit or return type of statements.
•Writing test cases to cover all paths of a typical program is impractical since
there can be an infinite number of paths through a program in presence of
loops.
•Path coverage testing does not try to cover all paths, but only a subset of paths
called linearly independent paths (or basis paths).

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 55

 Linearly Independent set of Paths or
Basis path set
•A set of paths for a given program is called linearly independent set of paths (or
the set of basis paths or simply the basis set), if each path in the set introduces
at least one new edge that is not included in any other path in the set.

•if we find that a path has one new node compared to all other linearly
independent paths, then this path should also be included in the set of linearly
independent paths. This is because, any path having a new node would
automatically have a new edge.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 56

McCabe’s Cyclomatic Complexity
•McCabe’s cyclomatic complexity defines an upper bound on the number of
independent paths in a program.
•Method 1: Given a control flow graph G of a program, the cyclomatic
•complexity V(G) can be computed as:
•V(G) = E – N + 2
•where, N is the number of nodes of the control flow graph and E is the number
of edges in the control flow graph.
•For the CFG of example shown in Figure previously, E = 7 and N = 6. Therefore,
the value of the Cyclomatic complexity = 7 – 6 + 2 = 3.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 57

McCabe’s Cyclomatic Complexity
•Method 2: the cyclomatic complexity V (G) for a graph G is given by the
following expression:
•V(G) = Total number of non-overlapping bounded areas + 1
•In the program’s control flow graph G, any region enclosed by nodes and edges
can be called as a bounded area.
•The number of bounded areas in a CFG increases with the number of decision
statements and loops.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 58

McCabe’s Cyclomatic Complexity
•Method 3: The cyclomatic complexity of a program can also be easily computed
by computing the number of decision and loop statements of the program. If N
is the number of decision and loop statements of a program, then the McCabe’s
metric is equal to N + 1.
•For the previous figure the no of decision statements are 2, and the cyclomatic
complexity is 2+1 = 3

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 59

•Path 1 (loop executes with x > y):
•1 → 2 → 3 → 5 → 1 → 6
•Path 2 (loop executes with x < y):
•1 → 2 → 4 → 5 → 1 → 6
•Path 4 (loop condition false
initially):
•1 → 6

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 60

Path coverage-based testing
•How is path testing carried out by using computed McCabe’s cyclomatic metric
value?
•Knowing the number of basis paths in a program does not make it any easier to
design test cases for path coverage, only it gives an indication of the minimum
number of test cases required for path coverage. For the CFG of a moderately
complex program segment of say 20 nodes and 25 edges, you may need several
days of effort to identify all the linearly independent paths in it and to design the
test cases. It is therefore impractical to require the test designers to identify all
the linearly independent paths in a code, and then design the test cases to force
execution along each of the identified paths. In practice, for path testing, usually
the tester keeps on forming test cases with random data and executes those
until the required coverage is achieved.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 61

Path coverage-based testing
•The following is the sequence of steps that need to be undertaken for deriving
the path coverage-based test cases for a program:
•1. Draw control flow graph for the program.
•2. Determine the McCabe’s metric V(G).
•3. Determine the cyclomatic complexity. This gives the minimum number of test
cases required to achieve path coverage.
•4. repeat

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 62

 Uses of McCabe’s cyclomatic
complexity metric

•Estimation of structural complexity of code: McCabe’s cyclomatic complexity is a

measure of the structural complexity of a program. The reason for this is that it
is computed based on the code structure (number of decision and iteration
constructs used). Cyclomatic complexity of a program is a measure of the
psychological complexity or the level of difficulty in understanding the program.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 63

 Uses of McCabe’s cyclomatic
complexity metric

•Estimation of testing effort: Cyclomatic complexity is a measure of the maximum

number of basis paths. Thus, it indicates the minimum number of test cases
required to achieve path coverage. Therefore, the testing effort and the time
required to test a piece of code satisfactorily is proportional to the cyclomatic
complexity of the code. To reduce testing effort, it is necessary to restrict the
cyclomatic complexity of every function to seven.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 64

 Uses of McCabe’s cyclomatic
complexity metric

•Estimation of program reliability: Experimental studies indicate there exists a

clear relationship between the McCabe’s metric and the number of errors latent
in the code after testing. This relationship exists possibly due to the correlation
of cyclomatic complexity with the structural complexity of code. Usually the
larger is the structural complexity, the more difficult it is to test and debug the
code.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 65

Data flow-based testing
•Data flow-based testing method selects test paths of a program according to the
definitions and uses of different variables in a program.
•Consider a program P . For a statement numbered S of P , let
•DEF(S) = {X /statement S contains a definition of X } and
•USES(S)= {X /statement S contains a use of X }
•For the statement S: a=b+c;, DEF(S)={a}, USES(S)={b, c}.
• The definition of variable X at statement S is said to be live at statement S1 , if
there exists a path from statement S to statement S1 which does not contain any
definition of X .

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 66

Data flow-based testing
•All definitions criterion is a test coverage criterion that requires that an
adequate test set should cover all definition occurrences i.e. for each definition
occurrence, the testing paths should cover a path through which the definition
reaches a use of the definition.
•All use criterion requires that all uses of a definition should be covered.
•Clearly, all-uses criterion is stronger than all-definitions criterion. An even
stronger criterion is all definition-use-paths criterion, which requires the
coverage of all possible definition-use paths that either are cycle-free or have
only simple cycles.
•A simple cycle is a path in which only the end node and the start node are the
same.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 67

Data flow-based testing
•The definition-use chain (or DU chain) of a variable X is of the form [X, S, S1],
where S and S1 are statement numbers, such that X є DEF(S) and X є USES(S1),
and the definition of X in the statement S is live at statement S1 .
•One simple data flow testing strategy is to require that every DU chain be
covered at least once.
•Data flow testing strategies are especially useful for testing programs containing
nested if and loop statements.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 68

Data flow-based testing
1. Read a,b,c •generate the du chain for variable a
2. If(a>b) •Definition – Line 1
3. X=a+1 •Use – Line 2(p-use or predicate use) and Line 3(c-use or
computational use)
4. Print x
•DU Chains
5. Else
•1->2
6. X=b-1
•1->2->3
7. Print z

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 69

Data flow-based testing
1. Read a,b,c •generate the du chain for variable b
2. If(a>b) •Definition – Line 1
3. X=a+1 •Use – Line 2(p-use or predicate use) and Line 6(c-use or
computational use)
4. Print x
•DU Chains
5. Else
•1->2
6. X=b-1
•1->2->5->6
7. Print z

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 70

Data flow-based testing
1. Read a,b,c •generate the du chain for variable c
2. If(a>b) •Definition – Line 1
3. X=a+1 •Use – NA
4. Print x •DU Chains
5. Else •{ }
6. X=b-1
7. Print z

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 71

Data flow-based testing
1. Read a,b,c •generate the du chain for variable x
2. If(a>b) •Definition – Line 3,6
3. X=a+1 •Use – 4,
4. Print x •DU Chains
5. Else •1->2->3->4
6. X=b-1 •1->2->5->6 and then it becomes a dead code
7. Print z

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 72

Data flow-based testing
1. Read a,b,c •generate the du chain for variable z
2. If(a>b) •Definition – NA
3. X=a+1 •Use – 7
4. Print x •DU Chains
5. Else •Invalid
6. X=b-1 •Need to define z before it can be used.
7. Print z

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 73

Mutation Testing
•A fault-based testing technique in the sense that mutation test cases are
designed to help detect specific types of faults in a program.

•In mutation testing, a program is first tested by using an initial test suite
designed by using various white box testing strategies that we have discussed.

•After the initial testing is complete, mutation testing can be taken up.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 74

Mutation Testing
•The idea behind mutation testing is to make a few arbitrary changes to a
program at a time.
•Each time the program is changed, it is called a mutated program and the
change effected is called a mutant.
•An underlying assumption behind mutation testing is that all programming
errors can be expressed as a combination of simple errors.
•A mutation operator make specific changes to a program.
•For example, one mutation operator may randomly delete a program statement.
•A mutant may or may not cause an error in the program.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 75

Mutation Testing
•If a mutant does not introduce any error in the program, then the original
program and the mutated program are called equivalent programs.
•A mutated program is tested against the original test suite of the program.
•If there exists at least one test case in the test suite for which a mutated
program yields an incorrect result, then the mutant is said to be dead, since the
error introduced by the mutation operator has successfully been detected by
the test suite.
•If a mutant remains alive even after all the test cases have been exhausted, the
test suite is enhanced to kill the mutant.
•However, it is not this straightforward. Remember that there is a possibility of a
mutated program to be an equivalent program. When this is the case, it is futile
to try to design a test case that would identify the error.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 76

Mutation Testing
•An important advantage of mutation testing is that it can be automated to a
great extent.
•The process of generation of mutants can be automated by predefining a set of
primitive changes that can be applied to the program.
•These primitive changes can be simple program alterations such as—deleting a
statement, deleting a variable definition, changing the type of an arithmetic
operator (e.g., + to -), changing a logical operator ( and to or) changing the value
of a constant, changing the data type of a variable, etc.
•A major pitfall of the mutation-based testing approach is that it is
computationally very expensive, since a large number of possible mutants can
be generated.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 77

Mutation Testing
•Mutation testing involves generating a large number of mutants.
•Also, each mutant needs to be tested with the full test suite.
•Obviously therefore, mutation testing is not suitable for manual testing.
•Mutation testing is most suitable to be used in conjunction of some testing tool
that should automatically generate the mutants and run the test suite
automatically on each mutant.
•At present, several test tools are available that automatically generate mutants
for a given program.
•Mutation testing improves test suite quality by deliberately introducing faults
(mutations) into the code and then verifying whether the test suite can detect
those faults, ensuring that tests are robust and effective.
SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 78
Debugging
•After a failure has been detected, it is necessary to first identify the program
statement(s) that are in error and are responsible for the failure, the error can
then be fixed.
•Debugging approaches
•Brute force method
•Backtracking
•Cause elimination method
•Program slicing

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 79

Debugging
•Brute force method
•This is the most common method of debugging but is the least efficient method.
•In this approach, print statements are inserted throughout the program to print
the intermediate values with the hope that some of the printed values will help
to identify the statement in error.
•This approach becomes more systematic with the use of a symbolic debugger
(also called a source code debugger ), because values of different variables can
be easily checked and break points and watch points can be easily set to test the
values of variables effortlessly.
•Single stepping using a symbolic debugger is another form of this approach,
where the developer mentally computes the expected result after every source
instruction and checks whether the same is computed by single stepping
through the program.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 80

Debugging
•Backtracking
•This is also a fairly common approach.
•In this approach, starting from the statement at which an error symptom has
been observed, the source code is traced backwards until the error is
discovered.
•Unfortunately, as the number of source lines to be traced back increases, the
number of potential backward paths increases and may become unmanageably
large for complex programs, limiting the use of this approach.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 81

Debugging
•Cause elimination method
•In this approach, once a failure is observed, the symptoms of the failure (i.e.,
certain variable is having a negative value though it should be positive, etc.) are
noted.
•Based on the failure symptoms, the causes which could possibly have
contributed to the symptom is developed and tests are conducted to eliminate
each.
•A related technique of identification of the error from the error symptom is the
software fault tree analysis.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 82

Debugging
•Program slicing
•This technique is similar to back tracking. In the backtracking approach, one
often has to examine a large number of statements. However, the search space
is reduced by defining slices.
•A slice of a program for a particular variable and at a particular statement is the
set of source lines preceding this statement that can influence the value of that
variable.
•Program slicing makes use of the fact that an error in the value of a variable can
be caused by the statements on which it is data dependent.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 83

Debugging
•Debugging Guidelines
•Many a times debugging requires a thorough understanding of the program
design. Trying to debug based on a partial understanding of the program design
may require an inordinate amount of effort to be put into debugging even for
simple problems.
•Debugging may sometimes even require full redesign of the system. In such
cases, a common mistakes that novice programmers often make is attempting
not to fix the error but its symptoms.
•One must be beware of the possibility that an error correction may introduce
new errors. Therefore, after every round of error-fixing, regression testing must
be carried out.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 84

Integration Testing
•Integration testing is carried out after all (or at least some of ) the modules have
been unit tested.
•Successful completion of unit testing, to a large extent, ensures that the unit (or
module) as a whole works satisfactorily.
•In this context, the objective of integration testing is to detect the errors at the
module interfaces (call parameters). For example, it is checked that no
parameter mismatch occurs when one module invokes the functionality of
another module.
•The objective of integration testing is to check whether the different modules of
a program interface with each other properly.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 85

Integration Testing
•During integration testing, different modules of a system are integrated in a
planned manner using an integration plan.
•The integration plan specifies the steps and the order in which modules are
combined to realize the full system.
•After each integration step, the partially integrated system is tested.
•By examining the structure chart, the integration plan can be developed.
•Any one (or a mixture) of the following approaches can be used to develop the
test plan:
• Big-bang approach to integration testing
• Top-down approach to integration testing
• Bottom-up approach to integration testing
• Mixed (also called sandwiched ) approach to integration testing
SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 86
Integration Testing
•Big-bang approach to integration testing
•Big-bang testing is the most obvious approach to integration testing. In this
approach, all the modules making up a system are integrated in a single step. In
simple words, all the unit tested modules of the system are simply linked
together and tested.
•However, this technique can meaningfully be used only for very small systems.
•The main problem with this approach is that once a failure has been detected
during integration testing, it is very difficult to localise the error as the error may
potentially lie in any of the modules.
•Therefore, debugging errors reported during big-bang integration testing are
very expensive to fix.
•As a result, big-bang integration testing is almost never used for large programs.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 87

Integration Testing
•Bottom-up approach to integration testing
•Large software products are often made up of several subsystems.
•A subsystem might consist of many modules which communicate among each
other through well-defined interfaces.
•In bottom-up integration testing, first the modules for each of the subsystem are
integrated. Thus, the subsystems can be integrated separately and
independently.
•The primary purpose of carrying out the integration testing a subsystem is to
test whether the interfaces among various modules making up the subsystem
work satisfactorily.
•The test cases must be carefully chosen to exercise the interfaces in all possible
manners.
SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 88
Integration Testing
•Bottom-up approach to integration testing
•In a pure bottom-up testing no stubs are required, and only test-drivers are
required. Large software systems normally require several levels of subsystem
testing, lower-level subsystems are successively combined to form higher-level
subsystems.
•Advantages
•several disjoint subsystems can be tested simultaneously
•low-level modules get tested thoroughly, since they are exercised in each
integration step. Since the low-level modules do I/O and other critical functions,
testing the low-level modules thoroughly increases the reliability of the system.
•Disadvantages
•the complexity that occurs when the system is made up of a large number of
small subsystems that are at the same level. This extreme case corresponds to
the big-bang approach.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 89

Integration Testing
•Top-down approach to integration testing
•Top-down integration testing starts with the root module in the structure chart
and one or two subordinate modules of the root module.
•After the top-level ‘skeleton’ has been tested, the modules that are at the
immediately lower layer of the ‘skeleton’ are combined with it and tested.
•Top-down integration testing approach requires the use of program stubs to
simulate the effect of lower-level routines that are called by the routines under
test.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 90

Integration Testing
•Top-down approach to integration testing
•A pure top-down integration does not require any driver routines.
•Advantage
•it requires writing only stubs, and stubs are simpler to write compared to
drivers.
•disadvantage
•in the absence of lower-level routines, it becomes difficult to exercise the top-
level routines in the desired manner since the lower-level routines usually
perform input/output (I/O) operations.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 91

Integration Testing
•Mixed approach to integration testing
•The mixed (also called sandwiched ) integration testing follows a combination of
top-down and bottom-up testing approaches.
•In top-down approach, testing can start only after the top-level modules have
been coded and unit tested. Similarly, bottom-up testing can start only after the
bottom level modules are ready.
•The mixed approach overcomes this shortcoming of the top-down and bottom-
up approaches.
•In the mixed testing approach, testing can start as and when modules become
available after unit testing. Therefore, this is one of the most commonly used
integration testing approaches.
•In this approach, both stubs and drivers are required to be designed.
SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 92
Integration Testing
•Phased versus Incremental Integration Testing
•In incremental integration testing, only one new module is added to the partially
integrated system each time. In phased integration, a group of related modules
are added to the partial system each time.
•phased integration requires lesser number of integration steps compared to the
incremental integration approach.
•However, when failures are detected, it is easier to debug the system while
using the incremental testing approach since the errors can easily be traced to
the interface of the recently integrated module.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 93

System Testing
•After all the units of a program have been integrated together and tested,
system testing is taken up.
•System tests are designed to validate a fully developed system to assure that it
meets its requirements. The test cases are therefore designed solely based on
the SRS document.
•There are essentially three main kinds of system testing depending on who
carries out testing:
•Alpha Testing: Alpha testing refers to the system testing carried out by the test
team within the developing organisation.
•Beta Testing: Beta testing is the system testing performed by a select group of
friendly customers.
•Acceptance Testing: Acceptance testing is the system testing performed by the
customer to determine whether to accept the delivery of the system.
SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 94
System Testing
•In each of the above types of system tests, the test cases can be the same, but
the difference is with respect to who designs test cases and carries out testing.
•The system test cases can be classified into functionality and performance test
cases.
•Before a fully integrated system is accepted for system testing, smoke testing is
performed.
•Smoke testing is done to check whether at least the main functionalities of the
software are working properly.
•Unless the software is stable and at least the main functionalities are working
satisfactorily, system testing is not undertaken.
•The functionality tests are designed to check whether the software satisfies the
functional requirements as documented in the SRS document.
•The performance tests, on the other hand, test the conformance of the system
with the non-functional requirements of the system.
SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 95
System Testing
•Smoke Testing
•Smoke testing is carried out before initiating system testing to ensure that
system testing would be meaningful, or whether many parts of the software
would fail.
•The idea behind smoke testing is that if the integrated program cannot pass
even the basic tests, it is not ready for a vigorous testing.
•For smoke testing, a few test cases are designed to check whether the basic
functionalities are working.
•For example, for a library automation system, the smoke tests may check
whether books can be created and deleted, whether member records can be
created and deleted, and whether books can be loaned and returned.
SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 96
System Testing
•Performance Testing
•Performance testing is carried out to check whether the system meets the
nonfunctional requirements identified in the SRS document.
•There are several types of performance testing corresponding to various types
of non-functional requirements.
•For a specific system, the types of performance testing to be carried out on a
system depends on the different non-functional requirements of the system
documented in its SRS document.
•All performance tests can be considered as black-box tests.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 97

System Testing
•Reliability testing in software development is a type of software testing that
focuses on ensuring the software performs consistently and without failure
under specified conditions for a defined period of time.
•It's about making sure the software works as expected, consistently, even after
prolonged use or under stress.
Common Types of Reliability Testing:
[Link] Testing
[Link] Testing
[Link] Testing (Endurance Testing)
[Link] Testing
SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 98
System Testing

•Stress testing
•Stress testing is also known as endurance testing.
•Stress testing evaluates system performance when it is stressed for short
periods of time.
•Stress tests are black-box tests which are designed to impose a range of
abnormal and even illegal input conditions so as to stress the capabilities of the
software.
•Input data volume, input data rate, processing time, utilisation of memory, etc.,
are tested beyond the designed capacity. For example, suppose an operating
system is supposed to support fifteen concurrent transactions, then the system
is stressed by attempting to initiate fifteen or more transactions simultaneously.
SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 99
System Testing

•Stress testing
•A real-time system might be tested to determine the effect of simultaneous
arrival of several high-priority interrupts.
•Stress testing is especially important for systems that under normal
circumstances operate below their maximum capacity but may be severely
stressed at some peak demand hours.
•For example, if the corresponding nonfunctional requirement states that the
response time should not be more than twenty secs per transaction when sixty
concurrent users are working, then during stress testing the response time is
checked with exactly sixty users working simultaneously.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 100

System Testing
•Volume testing
•Volume testing checks whether the data structures (buffers, arrays, queues,
stacks, etc.) have been designed to successfully handle extraordinary situations.

•For example, the volume testing for a compiler might be to check whether the
symbol table overflows when a very large program is compiled.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 101

System Testing
•Configuration testing
•Configuration testing is used to test system behaviour in various hardware and
software configurations specified in the requirements.
•Sometimes systems are built to work in different configurations for different
users.
•For instance, a minimal system might be required to serve a single user, and
other extended configurations may be required to serve additional users during
configuration testing.
•The system is configured in each of the required configurations and depending
on the specific customer requirements, it is checked if the system behaves
correctly in all required configurations.
SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 102
System Testing
•Compatibility testing
•This type of testing is required when the system interfaces with external systems
(e.g., databases, servers, etc.).
•Compatibility aims to check whether the interfaces with the external systems
are performing as required.
•For instance, if the system needs to communicate with a large database system
to retrieve information, compatibility testing is required to test the speed and
accuracy of data retrieval.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 103

System Testing
•Regression testing
•This type of testing is required when a software is maintained to fix some bugs
or enhance functionality, performance, etc.
•Regression testing spans unit, integration, and system testing. Instead, it is a
separate dimension to these three forms of testing.
•Regression testing is the practice of running an old test suite after each change
to the system or after each bug fix to ensure that no new bug has been
introduced due to the change or the bug fix.
•However, if only a few statements are changed, then the entire test suite need
not be run — only those test cases that test the functions and are likely to be
affected by the change need to be run.
•Whenever a software is changed to either fix a bug, or enhance or remove a
feature, regression testing is carried out.
SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 104
System Testing
•Recovery testing
•Recovery testing tests the response of the system to the presence of faults, or
loss of power, devices, services, data, etc.
•The system is subjected to the loss of the mentioned resources (as discussed in
the SRS document) and it is checked if the system recovers satisfactorily.
•For example, the printer can be disconnected to check if the system hangs. Or,
the power may be shut down to check the extent of data loss and corruption.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 105

System Testing
•Maintenance testing
•This addresses testing the diagnostic programs, and other procedures that are
required to help maintenance of the system. It is verified that the artifacts exist
and they perform properly.
•Documentation testing
•It is checked whether the required user manual, maintenance manuals, and
technical manuals exist and are consistent. If the requirements specify the types
of audience for which a specific manual should be designed, then the manual is
checked for compliance of this requirement.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 106

System Testing
•Usability testing
•Usability testing concerns checking the user interface to see if it meets all user
requirements concerning the user interface.
•During usability testing, the display screens, messages, report formats, and
other aspects relating to the user interface requirements are tested.
•A GUI being just being functionally correct is not enough. Therefore, the GUI has
to be checked against the checklist

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 107

System Testing
•Security testing
•Security testing is essential for software that handle or process confidential data
that is to be gurarded against pilfering.
•It needs to be tested whether the system is fool-proof from security attacks such
as intrusion by hackers.
•Over the last few years, a large number of security testing techniques have been
proposed, and these include password cracking, penetration testing, and attacks
on specific ports, etc.

SUSHREE S. , SILICON UNIVERSITY, BHUBANESWAR 108