Domain testing
Tor Stlhane
Domain testing revisited
We have earlier looked at domain testing as
a simple strategy for selecting test cases.
We will now take this a step further.
Testers have frequently observed that
domain boundaries are particularly fault
prone and should therefore be carefully
checked.
Predicates
We will assume that all predicates are
simple. This implies that they contain only
one relational operator. Allowable
operators are =, ><, >=, <=, < and >.
Predicates that are not simple can be split
into two or more simple predicates.
Path conditions
Each path thorough a system is defined by a
path condition the conjunction of all
predicates encountered along this path.
An input traverses a path if and only if the
path condition is fulfilled.
The path condition defines a path domain,
which is the set of all inputs that cause
the path to be executed.
Path domains 1
A path domain is surrounded by a boundary,
consisting of one or more borders where
each border corresponds to a predicate.
A border is
Closed if defined by a predicate containing
=, =< or >=. Closed borders belong to the
path domain.
Open if defined by a predicate containing
><, > or >. Open borders do not belong to
the path domain.
Path domains 2
Note the difference between open and
closed domains.
Closed: what is
on the line belongs
to the domain
Open: what is off the line
below or above
belongs to the domain
X
X
X
Domain error
A domain error occurs if an input traverses
the wrong path through the program.
We have no way to know the correct borders
and there is no unique correct version of
the program.
When domain error occurs along a path, it
may be thought of as being caused by one
of the given borders being different from
the correct one.
Path domains
P1
P2
S1
S5
S4
S3
S2
P1 and P2: {S1, S3,S4}
P1 and not P2: {S1, S3, S5}
not P1 and P2: {S2, S3, S4}
not P1 and not P2: {S2, S3, S5}
ON and OFF points 1
The test strategy is a strategy for selecting
ON and OFF points, defined as follows:
ON point for a
Closed border lies on the border
Open border lies close to the border and
satisfies the inequality relation
OFF point lies close to the border and on
the open side or alternatively does not
satisfy the path condition associated with
this border
ON and OFF points 2
The ON and OFF points are used as follows:
For testing a closed border we use
Two ON points to identify the border
One OFF point, to test that the correct border
does not lie on the open side of the border
For testing an open border the role of the
ON and OFF points are reversed.
The strategy can be extended to N-
dimensional space by using N ON points
ON and OFF points 3
If the border line has V vertices, we will need
One ON point close to each vertex.
One OFF point per vertex at a uniform
distance from the border.
In all cases, it is important that the OFF
points are as close as possible to the ON
points
Example two-dimensional space
Correct border
Given border
OFF
ON
ON
Example for an open border: ><, < or >. Border outside the line
Example two-dimensional space
Given border
Correct border
OFF
ON
ON
Example for an closed border: = predicate. Border on the line
OFF
The problem of size
The main problem with this strategy is the
cost. Let us assume we have 20 input
variables. Then the suggested strategy
would need:
For each > or <: 20 ON points and one
OFF point
For each = or ><: 20 ON points plus two or
three OFF points
Ten predicates would require 210 230 test
cases.
The problem of precision
The strategy described require the ON
points to lie exactly on the border.
For any pair of real numbers, there is always
a third real number that lies between them.
For a computer, however, this is not the
case, due to limited precision.
Thus, there exist borders for which no ON
point can be represented in the computer.
A simplified strategy
We will drop the requirement that the border
can be exactly identified. Then we can
also drop the requirement that the ON
point lies exactly on the border. This
remove the precision problem.
In addition we can reduce the number of
points by one per border. The only error
that will not be detected is if the real
border passes between an ON and an
OFF point. Thus, these two points need to
be close.
Simplified use of On and OFF points
>, <
=, ><
ON ON
OFF
OFF
OFF
Effectiveness
>, <
ON
OFF
L
d
Assume that
m1 is the smallest real value
m2 is the largest real value
The length of L, called M, is
M = m2 m1 +1
P(not detect) = d / M
Code containing array references
Code segment. The array b has 10 elements.
i = x + 3;
IF b[i] <=5 THEN ,,,
ELSE
We need three predicates:
b[x + 3] <= 5 the original predicate
x + 3 >= 1 not below lower bound
x + 3 >= 10 not above upper bound
Non-linear borders 1
Everything discussed till now has been
based on the assumption that the real
border is linear. If this is not true, the
domain strategy might fail.
In the example on the next slide, the ON
points are on the border, OFF point is off
the border but nevertheless the given
border is wrong.
Non-linear borders 2
Given border
OFF
ON
ON
Correct border
A simple algorithm
We can apply domain testing as follows:
1. Select a test case and run it. This will
cause one path to be traversed.
2. Identify the borders of this path and test
them with ON and OFF points
3. If the OFF point belongs to a new path
then this path is selected for testing
otherwise check another OFF point.
4. Terminate when no new path can be
found
Simple algorithm - example
IF x in A THEN S1
ELSE
IF x in B THEN S2
ELSE
S3
A
B
C
1
2
3
When to use domain testing
Domain testing, as it is described here,
requires that we know how the input
partition the input space into input
domains.
Thus, it is only possible to use it for small
chunks of code.
Acknowledgement
This set of slides is based on the paper
A Simplified Domain-Testing Strategy
by B. Jeng and E.J. Weyuker.