Module 3 1
Module 3 1
DOMAIN TESTING
3. DOMAIN TESTING
3.1. Overview
3.2. Domain Bugs and How to Test for Them
3.3. Procedure
1
DOMAIN TESTING
o Domain:In mathematics, domain is a set of possible values of an independant variable or the variables of a
function.
o Programs as input data classifiers: domain testing attempts to determine whether the classification is or is
not correct.
o Domain testing can be based on specifications or equivalent implementation information.
o If domain testing is based on specifications, it is a functional test technique.
o If domain testing is based implementation details, it is a structural test technique.
o For example, you're doing domain testing when you check extreme values of an input variable.
o All inputs to a program can be considered as if they are numbers. For example, a character string can be
treated as a number by concatenating bits and looking at them as if they were a binary integer.
o This is the view in domain testing, which is why this strategy has a mathematical flavor.
o Before doing whatever it does, a routine must classify the input and set it moving on the right path.
o An invalid input (e.g., value too big) is just a special processing case called 'reject'.
o The input then passses to a hypothetical subroutine rather than on calculations.
o In domain testing, we focus on the classification aspect of the routine rather than on the calculations.
o Structural knowledge is not needed for this model - only a consistent, complete specification of input
values for each case.
o We can infer that for each case there must be atleast one path to process that case.
2
o If the source language supports set definitions (E.g. PASCAL set types and C enumerated types) less testing
is needed because the compiler does much of it for us.
o Domain testing does not work well with arbitrary discrete sets of data objects.
o Domain for a loop-free program corresponds to a set of numbers defined over the input vector.
DOMAINS
IF x>0
THEN ALPHA
ELSE BETA
o we know that numbers greater than zero belong to ALPHA processing domain(s) while zero and smaller
numbers belong to BETA domain(s).
o A domain may have one or more boundaries - no matter how many variables define it.
o For example, if the predicate is x2 + y2 < 16, the domain is the inside of a circle of radius 4 about the origin.
Similarly, we could define a spherical domain with one boundary but in three variables.
o Domains are usually defined by many boundary segments and therefore by many predicates.
PREDICATES:
PATHS
o For every domain, there is at least one path through the routine.
o There may be more than one path if the domain consists of disconnected parts .
o If the domain is defined by the union of two or more domains..
o A domain boundary is closed with respect to a domain if the points on the boundary belong to the domain.
o If the boundary points belong to some other domain, the boundary is said to be open.
o Figure 4.2 shows three situations for a one-dimensional domain - i.e., a domain defined over one input
variable; call it x
o The importance of domain closure is that incorrect closure bugs are frequent domain bugs. For example, x
>= 0 when x > 0 was intended.
3
Figure 4.2: Open and Closed Domains.
o The bug assumption for the domain testing is that processing is okay but the domain definition is wrong.
o An incorrectly implemented domain means that boundaries are wrong, which may in turn mean that
control flow predicates are wrong.
o Many different bugs can result in domain errors. Some of them are:
Domain Errors:
o In computers or Languages that have a distinct positive and negative zero, boundary errors for negative
zero are common.
o A floating point number can equal zero only if the previous definition of that number set it to zero or if it is
subtracted from it self or multiplied by zero.
o So the floating point zero check to be done against a epsilon value.
4
3. Contradictory domains:
o An implemented domain can never be ambiguous or contradictory, but a specified domain can. A
contradictory domain specification means that at least two supposedly distinct domains overlap.
4. Ambiguous domains:
o Ambiguous domains means that union of the domains is incomplete. That is there are missing domains or
holes in the specified domains. Not specifying what happens to points on the domain boundary is a
common ambiguity.
5. Overspecified Domains:
o The domain can be overloaded with so many conditions that the result is a null domain. Another way to
put it is to say that the domain's path is unachievable.
6. Boundary Errors:
o Errors caused in and around the boundary of a domain. Example, boundary closure bug, shifted, tilted,
missing, extra boundary.
7. Closure Reversal:
o A common bug. The predicate is defined in terms of >=. The programmer chooses to implement the logical
complement and incorrectly uses <= for the new predicate; i.e., x >= 0 is incorrectly negated as x <= 0,
thereby shifting boundary values to adjacent domains.
8. Faulty Logic:
o Compound predicates (especially) are subject to faulty logic transformations and improper simplification. If
the predicates define domain boundaries, all kinds of domain bugs can result from faulty logic
manipulations.
o Domain testing has restrictions, as do other testing techniques. Some of them include:
1. Co-incidental Correctness:
o Domain testing isn't good at finding bugs for which the outcome is correct for the wrong reasons.
o If we're plagued by coincidental correctness we may misjudge an incorrect boundary.
o Note that this implies weakness for domain testing when dealing with routines that have binary outcomes
(i.e., TRUE/FALSE)
2. Representative Outcome:
5
o If the selected input is shown to be correct by a test, then processing is presumed correct, and therefore
all inputs within that domain are expected (perhaps unjustifiably) to be correct.
o Most test techniques, functional or structural, fall under partition testing and therefore make this
representative outcome assumption. For example, x2 and 2x are equal for x = 2, but the functions are
different.
o The functional differences between adjacent domains are usually simple, such as x + 7 versus x + 9, rather
than x2 versus 2x.
o Compound predicates in which each part of the predicate specifies a different boundary are not a
problem:
o for example, x >= 0 AND x < 17, just specifies two domain boundaries by one compound predicate.
o As an example of a compound predicate that specifies one boundary, consider: x = 0 AND y >= 7 AND y <=
14.
o This predicate specifies one boundary equation (x = 0) but alternates closure, putting it in one or the other
domain depending on whether y < 7 or y > 14.
o Treat compound predicates with respect because they’re more complicated than they seem.
o Whatever the bug is, it will not change the functional form of the boundary predicate.
o For example, if the predicate is ax >= b, the bug will be in the value of a or b but it will not change the
predicate to ax >= b, say.
o Most papers on domain testing, assume linear boundaries - not a bad assumption because in practice most
boundary predicates are linear.
o Domains are and will be defined by an imperfect iterative process aimed at achieving (user, buyer, voter)
satisfaction.
o Many requirement bugs can be viewed as ill-defined domains—meaning ill-defined domain boundaries.
o The first step in applying domain testing should be to analyze domain definitions in order to get (at least)
consistent and complete domain specifications.
o As designers and testers, we have been far too timid about challenging specified domains.
6
2.2. Specified Versus Implemented Domains
1. Linear,
2. Complete,
3. Systematic,
4. Orthogonal,
5. Consistently closed,
6. Convex and
7. Simply connected
o To the extent that domains have these properties domain testing is easy as testing gets.
7
o The bug frequency is lesser for nice domain than for ugly domains.
[Link] BOUNDARIES:
o Nice domain boundaries are complete in that they span the number space from plus to minus infinity in all
dimensions.
o Figure 4.4 shows some incomplete boundaries.
o Boundaries A and E have gaps.
o Such boundaries can come about because the path that hypothetically corresponds to them is
unachievable, because inputs are constrained in such a way that such values can't exist, because of
compound predicates that define a single boundary, or because redundant predicates convert such
boundary values into a null set.
o The advantage of complete boundaries is that one set of tests is needed to confirm the boundary no
matter how many domains it bounds.
o If the boundary is chopped up and has holes in it, then every segment of that boundary must be tested for
every domain it bounds.
8
Figure 4.4: Incomplete Domain Boundaries.
[Link] BOUNDARIES:
o Systematic boundary means that boundary inequalities related by a simple function such as a constant.
o In Figure 4.3 for example, the domain boundaries for u and v differ only by a constant. We want relations
such as
where fi is an arbitrary linear function, X is the input vector, ki and c are constants, and g(i,c) is a decent
function over i and c that yields a constant, such as k + ic.
o The first example is a set of parallel lines, and the second example is a set of systematically (e.g., equally)
spaced parallel lines (such as the spokes of a wheel, if equally spaced in angles, systematic).
o If the boundaries are systematic and if you have one tied down and generate tests for it, the tests for the
rest of the boundaries in that set can be automatically generated.
[Link] BOUNDARIES:
o Two boundary sets U and V (See Figure 4.3) are said to be orthogonal if every inequality in V is
perpendicular to every inequality in U.
o If two boundary sets are orthogonal, then they can be tested independently
o In Figure 4.3 we have six boundaries in U and four in V.
o We can confirm the boundary properties in a number of tests proportional to 6 + 4 = 10 (O(n)). If we tilt
the boundaries to get Figure 4.5, we must now test the intersections.
o We've gone from a linear number of cases to a quadratic: from O(n) to O(n2).
9
Figure 4.5: Tilted Boundaries.
o Actually, there are two different but related orthogonality conditions. Sets of boundaries can be
orthogonal to one another but not orthogonal to the coordinate axes (condition 1), or boundaries can
be orthogonal to the coordinate axes (condition 2).
[Link] CONSISTENCY:
o Figure 4.6 shows another desirable domain property: boundary closures are consistent and systematic.
o The shaded areas on the boundary denote that the boundary belongs to the domain in which the shading
lies - e.g., the boundary lines belong to the domains on the right.
o Consistent closure means that there is a simple pattern to the closures - for example, using the same
relational operator for all boundaries of a set of parallel boundaries.
[Link]:
o A geometric figure (in any number of dimensions) is convex if you can take two arbitrary points on any two
different boundaries, join them by a line and all points on that line lie within the figure.
o Nice domains are convex; dirty domains aren't.
o You can smell a suspected concavity when you see phrases such as: ". . . except if . . .," "However . . .," ". . .
but not. . . ." In programming, it's often the buts in the specification that kill you.
[Link] CONNECTED:
10
o Nice domains are simply connected; that is, they are in one piece rather than pieces all over the place
interspersed with other domains.
o Simple connectivity is a weaker requirement than convexity; if a domain is convex it is simply connected,
but not vice versa.
o Consider domain boundaries defined by a compound predicate of the (boolean) form ABC. Say that the
input space is divided into two domains, one defined by ABC and, therefore, the other defined by its
negation .
o For example, suppose we define valid numbers as those lying between 10 and 17 inclusive. The invalid
numbers are the disconnected domain consisting of numbers less than 10 and greater than 17.
o Simple connectivity, especially for default cases, may be impossible.
o Some domains are born ugly and some are uglified by bad specifications.
o Every simplification of ugly domains by programmers can be either good or bad.
o Programmers in search of nice solutions will "simplify" essential complexity out of existence. Testers in
search of brilliant insights will be blind to essential complexity and therefore miss important cases.
o If the ugliness results from bad specifications and the programmer's simplification is harmless, then the
programmer has made ugly good.
o But if the domain's complexity is essential (e.g., the income tax code), such "simplifications" constitute
bugs.
o Nonlinear boundaries are so rare in ordinary programming that there's no information on how
programmers might "correct" such boundaries if they're essential.
o Two kinds of contradictions are possible: overlapped domain specifications and overlapped closure
specifications
o Figure 4.7c shows overlapped domains and Figure 4.7d shows dual closure assignment.
11
[Link] THE TOPOLOGY:
o The programmer's and tester's reaction to complex domains is the same - simplify
o There are three generic cases: concavities, holes and disconnected pieces.
o Programmers introduce bugs and testers misdesign test cases by: smoothing out concavities (Figure 4.8a),
filling in holes (Figure 4.8b), and joining disconnected pieces (Figure 4.8c).
o If domain boundaries are parallel but have closures that go every which way (left, right, left, . . .) the
natural reaction is to make closures go the same way (see Figure 4.9).
3. DOMAIN TESTING:
12
3.1 OVERVIEW:
o An interior point (Figure 4.10) is a point in the domain such that all points within an arbitrarily small
distance (called an epsilon neighborhood) are also in the domain.
o A boundary point is one such that within an epsilon neighborhood there are points both in the domain and
not in the domain.
o An extreme point is a point that does not lie between any two other arbitrary but distinct points of a
(convex) domain.
13
o Figure 4.12 shows generic domain bugs: closure bug, shifted boundaries, tilted boundaries, extra
boundary, missing boundary.
o Figure 4.13 shows possible domain bugs for a one-dimensional open domain boundary.
o The closure can be wrong (i.e., assigned to the wrong domain) or the boundary (a point in this case) can be
shifted one way or the other, we can be missing a boundary, or we can have an extra boundary.
14
Figure 4.13: One Dimensional Domain Bugs, Open Boundaries.
o In Figure 4.13a we assumed that the boundary was to be open for A. The bug we're looking for is a closure
error, which converts > to >= or < to <= (Figure 4.13b). One test (marked x) on the boundary point
detects this bug because processing for that point will go to domain A rather than B.
o In Figure 4.13c we've suffered a boundary shift to the left. The test point we used for closure detects this
bug because the bug forces the point from the B domain, where it should be, to A processing. Note
that we can't distinguish between a shift and a closure error, but we do know that we have a bug.
o Figure 4.13d shows a shift the other way. The on point doesn't tell us anything because the boundary shift
doesn't change the fact that the test point will be processed in B. To detect this shift we need a point
close to the boundary but within A. The boundary is open, therefore by definition, the off point is in A
(Open Off Inside).
o The same open off point also suffices to detect a missing boundary because what should have been
processed in A is now processed in B.
o To detect an extra boundary we have to look at two domain boundaries. In this context an extra boundary
means that A has been split in two. The two off points that we selected before (one for each boundary)
does the job. If point C had been a closed boundary, the on test point at C would do it.
o For closed domains look at Figure 4.14. As for the open boundary, a test point on the boundary detects the
closure bug.
o The rest of the cases are similar to the open boundary, except now the strategy requires off points just
outside the domain.
o Figure 4.15 shows possible domain boundary bugs for a two-dimensional domain.
o A and B are adjacent domains and the boundary is closed with respect to A, which means that it is open
with respect to B.
15
For Closed Boundaries:
Closure Bug:
o Figure 4.15a shows a faulty closure, such as might be caused by using a wrong operator (for example, x >=
k when x > k was intended, or vice versa). The two on points detect this bug because those values will
get B rather than A processing.
Shifted Boundary:
o In Figure 4.15b the bug is a shift up, which converts part of domain B into A processing, denoted by A'. This
result is caused by an incorrect constant in a predicate, such as x + y >= 17 when x + y >= 7 was
intended. The off point (closed off outside) catches this bug. Figure 4.15c shows a shift down that is
caught by the two on points.
Tilted Boundary:
o A tilted boundary occurs when coefficients in the boundary inequality are wrong. For example, 3x + 7y > 17
when 7x + 3y > 17 was intended. Figure 4.15d has a tilted boundary, which creates erroneous domain
segments A' and B'. In this example the bug is caught by the left on point.
Extra Boundary:
o An extra boundary is created by an extra predicate. An extra boundary will slice through many different
domains and will therefore cause many test failures for the same bug. The extra boundary in Figure
4.15e is caught by two on points, and depending on which way the extra boundary goes, possibly by
the off point also.
Missing Boundary:
o A missing boundary is created by leaving a boundary predicate out. A missing boundary will merge
different domains and will cause many test failures although there is only one bug. A missing
boundary, shown in Figure 4.15f, is caught by the two on points because the processing for A and B is
the same - either A or B processing.
16
Figure 4.15: Two Dimensional Domain Bugs.
o The procedure is conceptually is straight forward. It can be done by hand for two dimensions and for a few
domains and practically impossible for more than two variables.
o Identify input variables.
o Identify variable which appear in domain defining predicates, such as control flow predicates.
o Interpret all domain predicates in terms of input variables.
o For p binary predicates, there are at most 2p combinations of TRUE-FALSE values and therefore, at most 2p
domains.
o Find the set of all non null domains. The result is a boolean expression in the predicates consisting a set of
AND terms joined by OR's. For example ABC+DEF+GHI ...... Where the capital letters denote predicates.
17
o Each product term is a set of linear inequality that defines a domain or a part of a multiply connected
domains.
o Solve these inequalities to find all the extreme points of each domain using any of the linear programming
methods.
o Recall that we defined integration testing as testing the correctness of the interface between two
otherwise correct components.
o Components A and B have been demonstrated to satisfy their component tests, and as part of the act of
integrating them we want to investigate possible inconsistencies across their interface.
o Interface between any two components is considered as a subroutine call.
o We're looking for bugs in that "call" when we do interface testing.
o Let's assume that the call sequence is correct and that there are no type incompatibilities.
o For a single variable, the domain span is the set of numbers between (and including) the smallest value
and the largest value.
o For every input variable we want (at least): compatible domain spans and compatible closures (Compatible
but need not be Equal).
o The set of output values produced by a function is called the range of the function, in contrast with the
domain, which is the set of input values over which the function is defined.
o For most testing, our aim has been to specify input values and to predict and/or confirm output values that
result from those inputs.
o Interface testing requires that we select the output values of the calling routine i.e. caller's range must be
compatible with the called routine's domain.
o An interface test consists of exploring the correctness of the following mappings:
o Assume that the caller's range and the called domain spans the same numbers - for example, 0 to 17.
o Figure 4.16 shows the four ways in which the caller's range closure and the called's domain closure can
agree.
o The thick line means closed and the thin line means open. Figure 4.16 shows the four cases consisting of
domains that are closed both on top (17) and bottom (0), open top and closed bottom, closed top and
open bottom, and open top and bottom.
18
Figure 4.16: Range / Domain Closure Compatibility.
o Figure 4.17 shows the twelve different ways the caller and the called can disagree about closure. Not all of
them are necessarily bugs. The four cases in which a caller boundary is open and the called is closed
(marked with a "?") are probably not buggy. It means that the caller will not supply such values but the
called can accept them.
o Figure 4.18: Harmless Range / Domain Span incompatibility bug (Caller Span is smaller than Called).
o In all cases, the caller's range is a subset of the called's domain. That's not necessarily a bug.
o The routine is used by many callers; some require values inside a range and some don't.
o This kind of span incompatibility is a bug only if the caller expects the called routine to validate the called
number for the caller.
o Figure 4.19a shows the opposite situation, in which the called routine's domain has a smaller span than
the caller expects. All of these examples are buggy.
19
Figure 4.19: Buggy Range / Domain Mismatches
o In Figure 4.19b the ranges and domains don't line up; hence good values are rejected, bad values are
accepted, and if the called routine isn't robust enough, we have crashes.
o Figure 4.19c combines these notions to show various ways we can have holes in the domain: these are all
probably buggy.
o For interface testing, bugs are more likely to concern single variables rather than peculiar combinations of
two or more variables.
o Test every input variable independently of other input variables to confirm compatibility of the caller's
range and the called routine's domain span and closure of every domain defined for that variable.
o There are two boundaries to test and it's a one-dimensional domain; therefore, it requires one on and one
off point per boundary or a total of two on points and two off points for the domain.
o Start with the called routine's domains and generate test points in accordance to the domain-testing
strategy used for that routine in component testing.
o Unless you're a mathematical whiz you won't be able to do this without tools for more than one variable
at a time.
o Start with the called routine’s domains and generate test points in accordance to the domain-testing
strategy used for that routine in component testing.
o A good component test should have included all the interesting domain-testing cases, and as a
consequence there’s no need to repeat the work.
o Those test cases are the values for which you must find the input values of the caller. If the caller’s
domains are linear, then this is another exercise in solving inequalities.
o Some things to consider:
o 1. The solution may not be unique.
o 2. There may be no solution for the specific points you need.
o 3. In general, you must find and evaluate an inverse function for the caller. Note “an inverse” rather than
“the inverse.”
20
o 4. Unless you’re a mathematical whiz you won’t be able to do this without tools for more than one
variable at a time.
o To convert non linear boundaries to equivalent liner boundaries we apply linearizig transformation.
o The methods used here are centuries old, but they are simple.
1. POLYNOMIAL
2. LOGARITHMIC Transforms:
o Other linearizable forms include x/(ax+b) and ax^b. we can also linearize by using the Taylor series
expansion of non linear functions, which yields an infinite polynomial.
21