Software Testing, Module 2

Part-1: Functional Testing: Text Book-1, Chapter 5, 6 & 7

[Link] value analysis
[Link] classes
[Link] tables

Part-2: Fault Based Testing: Text Book-2, Chapter 16

Part-1: Functional Testing:

1. Boundary value analysis,

2. Robustness testing,
3. Worst-case testing,
4. Robust Worst testing for triangle problem,
Nextdate problem and
commission problem,

5. Equivalence classes,
6. Equivalence test cases for the triangle problem,
NextDate function, and
the commission problem,
7. Guidelines and observations,
8. Decision tables,
9. Test cases for the triangle problem,
NextDate function, and
the commission problem,
[Link] and observations.
From Text Book-1: Chapter 5, 6 & 7
Part-2: Fault Based Testing:
1. Overview
2. Assumptions in fault based testing,
3. Mutation analysis
4. Fault-based adequacy criteria
5. Variations on mutation analysis
From Text Book-2: Chapter 16

Page 1 of 14
Unit Testing:
The term “unit” needs explanation. There are several interpretations about
exactly what constitutes a unit.
In a procedural programming language,
a unit can be
 A single procedure
 A function
 A body of code that implements a single function
 Source code that fits on one page
 A body of code that represents work done in 4 to 40 hours
(as in a work breakdown structure)
 The smallest body of code that can be compiled and
executed by itself

In an object-oriented programming language, there is general agreement

that a class is a unit.

Chapter 5

1. Boundary Value Testing

In Chapter 3, we saw that a function maps values from one set (its domain) to
values in another set (its range) and that the domain and range can be cross
products of other sets.

Any program can be considered to be a function in the sense that program

inputs form its domain and program outputs form its range.

In this and the next two chapters, we examine how to use knowledge of the
functional nature of a program to identify test cases for the program.

Boundary Value Testing is also known as

Input domain testing or
Black box testing or
specification-based testing.
Historically, this form of testing has focused on the input domain;

Page 2 of 14
There are two independent considerations that apply to input domain
testing.
The first asks whether or not we are concerned with invalid values of
variables.
1. Normal boundary value testing
is concerned only
with valid values of the input variables.

2. Robust boundary value testing considers

invalid and valid variable values.

The second consideration is whether we make the “single fault”

assumption common to reliability theory.

This assumes that faults are due to incorrect values of a single variable.

If this is not warranted, meaning that we are concerned with interaction

among two or more variables, we need to take the cross product of the
individual variables.
Taken together, the two considerations yield

four variations of boundary value testing:

1. Normal boundary value testing

2. Robust boundary value testing
3. Worst-case boundary value testing
4. Robust worst-case boundary value testing

For the sake of comprehensible drawings, the discussion in this chapter

refers to a function, F, of two variables x1 and x2. When the function F is
implemented as a program, the input variables x1 and x2 will have some
(possibly unstated) boundaries:
a ≤ x1 ≤ b
c ≤ x2 ≤ d

Page 3 of 14
Unfortunately, the intervals [a, b] and [c, d] are referred to as the ranges of x1
and x2, so right away we have an overloaded term. The intended meaning will
always be clear from its context.

5.1 Normal Boundary Value Testing

All four forms of boundary value testing focus on the boundary of the
input space to identify test cases.

The rationale behind boundary value testing is that errors tend to occur near
the extreme values of an input variable.
The basic idea of boundary value analysis is to use
input variable values at their
minimum,
just above the minimum,
a nominal value,
just below their maximum,
and at their maximum.
(min, min+, nom, max–, and max)

Page 4 of 14
The normal boundary value analysis test cases for our function F of two
variables (illustrated in Figure 5.2) are
{<x1nom, x2min>, <x1nom, x2min+>, <x1nom, x2nom>, <x1nom, x2max–>,
<x1nom, x2max>, <x1min, x2nom>, <x1min+,x2nom>, <x1max–, x2nom>,
<x1max, x2nom>}
9 test cases for 2 variables. For 3 variables, how many test cases?

5.1.1 Generalizing Boundary Value Analysis

if we have a function of n variables,
we hold all but one at the nominal values and
let the remaining variable assume
the min, min+, nom, max–, and max values,
repeating this for each variable.
Thus, for a function of n variables,
boundary value analysis yields 4n + 1 unique test cases.

5.2 Robust Boundary Value Testing

(min-,min, min+, nom, max–, max, and max+)

For 3 variables … test cases?
Page 5 of 14
5.3 Worst-Case Boundary Value Testing
Rejecting single-fault assumption means that we are interested in what happens
when more than one variable has an extreme value.
For each variable, we start with
the five-element set that contains the
min, min+, nom, max–, and max values.
We then take the Cartesian product of these sets to generate test cases.
worst-case testing for a function of n variables generates 5n test cases.

The result of the two-variable version of this is shown in Figure 5.4.

Worst-case testing follows the generalization pattern we saw for boundary value
analysis. It also has the same limitations, particularly those related to
independence. Probably the best application for worst-case testing is where
physical variables have numerous interactions, and where failure of
the function is extremely costly. For really paranoid testing, we could go to
robust worst-case testing.

5.4 Robust worst-case boundary value testing

This involves the Cartesian product of the seven-element sets we used in

robustness testing resulting in 7n test cases. Figure 5.5 shows the robust worst-
case test cases for our two-variable function.

For each variable, we start with

the seven-element set that contains the
min-, min, min+, nom, max–, max and max+ values.
We then take the Cartesian product of these sets to generate test cases.
worst-case testing for a function of n variables generates 7n test cases.

Page 6 of 14
Chapter-6: Equivalence Class Testing

The use of equivalence classes as the basis for functional testing has two
motivations:
1. we would like to have a sense of complete testing and,
at the same time,
2. we would hope to avoid redundancy.

Equivalence class testing echoes the two deciding factors of

boundary value testing, robustness,
and
the single/multiple fault assumptions

Equivalence Classes

 Equivalence classes form a partition of a set,

where
partition refers to a collection of mutually disjoint subsets,
the union of which is the entire set.
This has two important implications for testing:
the fact that the entire set is represented provides a form of
completeness,
and
the disjointedness ensures a form of non-redundancy.
Because the subsets are determined by an equivalence relation,
the elements of a subset have something in common.
Page 7 of 14
 The idea of equivalence class testing is to identify test cases by
using one element from each equivalence class.
If the equivalence classes are chosen wisely, this greatly reduces the
potential redundancy among test cases.

 Ex: In the triangle problem, we would certainly have a test case for an
equilateral triangle, and we might pick the triple (5, 5, 5) as inputs for a
test case. If we did this, we would not expect to learn much from test
cases such as (6, 6, 6) and (100, 100, 100). Our intuition tells us that
these would be treated the same as the first test case; thus, they would be
redundant.

 The key (and the craft) of equivalence class testing is the choice of the
equivalence relation that determines the classes. Very often, we make this
choice by second- guessing the likely implementation and thinking about
the functional manipulations that must somehow be present in the
implementation.

 A function, F, of two variables, X1 and X2, when F is implemented as a

program, the input variables X1 and X2 will have the following
boundaries, and intervals within the boundaries:

a ≤ x1 ≤ d, with intervals [a, b), [b, c), [c, d]

e ≤ x2 ≤ g, with intervals [e, f ), [f, g]

Where square brackets and parentheses denote, respectively,

closed and open interval endpoints.

Page 8 of 14
 The intervals presumably correspond to some distinction in the program
being tested, for example,
the commission ranges in the commission problem.
Invalid values of X1 and X2 are X1 < a, X1 > d and X2 < e, X2 > g.

The equivalence class testing can be categorized into four different types,
which are integral part of testing and cater to different data set.

Four Types of Equivalence Class Testing:

1. Weak Normal Equivalence Class Testing,

2. Strong Normal Equivalence Class Testing,
3. Weak Robust Equivalence Class Testing, and
4. Strong Robust Equivalence Class testing.

[Link] Normal Equivalence Class Testing:

In this first type of equivalence class testing, one variable from each
equivalence class is tested by the team.
Moreover, the values are identified in a systematic manner.
Weak normal equivalence class
testing is also known as single fault assumption.

Page 9 of 14
2. Strong Normal Equivalence Class Testing:
Termed as multiple fault assumption, in strong normal equivalence
class testing the team selects test cases from each element
of the Cartesian product of the equivalence. This ensures
the notion of completeness in testing, as it covers all
equivalence classes and offers the team one of each possible
combinations of inputs.

[Link] Robust Equivalence Class Testing:

The name for this form is admittedly counterintuitive

and oxymoronic (a figure of speech in which apparently contradictory terms
appear in conjunction (e.g. faith unfaithful. less is more)
Like weak normal equivalence,
weak robust testing too tests one variable from each equivalence class.

Page 10 of 14
 However, unlike the former method, it is also focused on testing test cases for
invalid values.

4. Strong Robust Equivalence Class Testing:

Another type of equivalence class testing, strong robust testing produces test
cases for all valid and invalid elements of the product of the equivalence class.
At least the name for this form is neither counterintuitive nor oxymoronic, just
redundant.
As before, the robust part comes from consideration of invalid values, and the
strong part refers to the multiple fault assumption. We obtain test cases
from each element of the Cartesian product of all the equivalence classes, both
valid and invalid, as shown in Figure 6.6.
However, it is incapable of reducing the redundancy in testing.

Page 11 of 14
Advantages:
1. Helps reduce the number of test cases,
without compromising the test coverage.

2. Reduces the overall test execution time as

it minimizes the set of test data.

3. Enables the testers to focus on smaller data sets,

which increases the probability of uncovering
more defects in the software product.

4. Used in cases where perfoming exhaustive

testing is difficult.

Disadvantages:

1. It does not consider the conditions for

boundary value.

2. The identification of equivalence classes

relies heavily on the expertise of tester.

3. Testers might assume that the output for all input

data set are correct, which can become
a great hurdle in testing

Page 12 of 14
 weak single fault assumption
strong multiple fault assumption
normal valid data
robust invalid data

The single versus multiple fault assumption yields the

weak/strong distinction
and the focus on invalid data yields a second distinction:
normal versus robust.

Taken together, these two assumptions result in

Weak Normal,
Strong Normal,
Weak Robust, and
Strong Robust Equivalence Class testing.

Page 13 of 14
Page 14 of 14