Outline Structural coverage over design Data flow coverage over design
Design Integration Testing and Graph Coverage
Meenakshi D’Souza
International Institute of Information Technology Bangalore.
Outline Structural coverage over design Data flow coverage over design
Goals
Applying graph coverage criteria (structural and data flow) to
code involving method/procedure calls.
Phase of testing where this will apply— Integration testing.
Note: We will consider integration testing in the presence of
object-oriented features separately.
Outline Structural coverage over design Data flow coverage over design
Design integration testing and graph coverage criteria
Graph models for integration testing: Call graphs
Nodes are modules/test stubs/test drivers.
Edges are call interfaces.
Structural coverage criteria: Deals with calls over interfaces.
Data flow coverage criteria: Deals with exchange of data over
interfaces.
Outline Structural coverage over design Data flow coverage over design
Call Graph: Example
x=5 B(y)
x=4
z=y t=y
x=3 print(y)
Call interface
B(x)
Outline Structural coverage over design Data flow coverage over design
Structural coverage criteria on call graphs
Nodes are modules. Node coverage will be to call every
module at least once.
Edges are calls to modules. Edge coverage is to execute every
call at least once.
(Specified) path coverage can be used to test a sequence of
method calls.
Outline Structural coverage over design Data flow coverage over design
Data flow at the design level
Structural coverage criteria might not reveal interesting faults.
Data flow interfaces amongst modules are more complicated
than control flow interfaces.
When values are passed, they change names, get assigned to
different variables.
There are many kinds of interfaces.
Need to traces uses for defs across modules, there could be
several uses in different modules.
We now focus on data flow interface testing using graphs.
Outline Structural coverage over design Data flow coverage over design
Definitions: Integration Testing
Caller: A module that
invokes/calls another
A Caller
module.
Callee: The module that
B(x) Actual parameter is called.
Call site: The statement
end A or node where the call
Interface Callee
appears in the code.
B(y) Formal parameter
Actual parameter(s):
end B Variable(s) in the caller.
Formal parameter(s):
Variables(s) in the callee.
Outline Structural coverage over design Data flow coverage over design
More terminologies: Integration Testing
Coupling variables are variables that are defined in one unit
and used in the other.
There are different kinds of couplings based on the interfaces:
Parameter coupling: Parameters are passed in calls.
Shared data coupling: Two units access the same data through
global or shared variables.
External device coupling: Two units access an external object
like a file.
Message-passing interfaces: Two units communicate by
sending and/or receiving messages over buffers/channels.
Outline Structural coverage over design Data flow coverage over design
Inter-procedural DU pairs
Since focus is on testing interfaces, we consider the last
definitions of variables before calls to and returns from the
called units and the first uses inside modules and after calls.
Last-def: The set of nodes that define a variable x and has a
def-clear path from the node through a call site to a use in
the other module.
Can be from caller to callee (parameter or shared variable) or
from callee to caller (return value).
First-use: The set of nodes that have uses of a variable y and
for which there is a def-clear and use-clear path from the call
site to the nodes.
Outline Structural coverage over design Data flow coverage over design
Coupling du-pairs example
Caller
F x=14 last−def
DU pair
y=G(x) call site
print(y) first−use
Callee
G(a) print(a) first−use
DU pair
b=42 last−def
return(b)
Outline Structural coverage over design Data flow coverage over design
Last defs and first uses: Example
1 x=5 10 B(y)
2 x=4 11 12
z=y t=y
3 x=3 13 print(y)
4 B(x) Last defs: 2,3
First uses: 11,12
Outline Structural coverage over design Data flow coverage over design
Coupling data flow coverage criteria
A coupling du-path is from a last-def to a first-use.
Data flow coverage criteria can now be extended to coupling
variables:
All-coupling-def coverage: A path is to be executed from every
last-def to at least one first-use.
All-coupling-use coverage: A path is to be executed from every
last-def to every first-use.
All-coupling-du-paths coverage: Every simple path from every
last-def to every first-use needs to be executed.
The above criteria can be met with side trips.
Outline Structural coverage over design Data flow coverage over design
Example: Quadratic Root
1 // Program to compute the quadratic root for two numbers
2 import [Link];
3 class Quadratic {
4 private static float Root1, Root2;
5 public static void main (String[] argv)
6 { int X, Y, Z;
7 boolean ok;
8 int controlFlag = [Link] (argv[0]);
9 if (controlFlag == 1)
10 { X = [Link] (argv[1]);
11 Y = [Link] (argv[2]);
12 Z = [Link] (argv[3]); }
13 else
14 { X = 10;
15 Y = 9;
16 Z = 12; }
17 ok = Root(X, Y, Z);
18 if (ok)
19 [Link](‘‘Quadratic roots:’’ + Root1, + Root2);
20 else
21 [Link] (‘‘No Solution’’);
22 }
Outline Structural coverage over design Data flow coverage over design
Example: Quadratic Root, contd.
23 // Three positive integers, finds quadratic root
24 private static boolean Root (int A, int B, int C)
25 {
26 double D;
27 boolean Result;
28 D = (double)(B*B)-(double)(4.0*A*C);
29 if (D < 0.0)
30 {
31 Result = false;
32 return (Result);
33 }
34 Root1 = (double) ((-B + [Link](D))/(2.0*A));
35 Root2 = (double) ((-B - [Link](D))/(2.0*A));
36 Result = true;
37 return (Result);
38 } // End method Root
39 } // End class Quadratic
Full code available here: https:
//[Link]/~offutt/softwaretest/edition1/programs/ch02/[Link]
Outline Structural coverage over design Data flow coverage over design
Quadratic Root Example: Defs and Uses
Shared variables: Root1, Root2.
Last defs:
Lines 10, 11, 12 and 14, 15, 16: Declaration and initialization
of variables X,Y and Z.
Lines 31 or 36: Result.
Lines 34, 35: Computing Root1, Root2.
First uses:
Line 17: ok.
Line 28: A, B, C (X, Y, Z).
Outline Structural coverage over design Data flow coverage over design
Quadratic Root Example: Coupling du-pairs
Legend: Pairs of locations as (method name, variable name,
statement number)
(main(),X,10) — (Root(),A,28)
(main(),Y,11) — (Root(),B,28)
(main(),Z,12) — (Root(),C,28)
(main(),X,14) — (Root(),A,28)
(main(),Y,15) — (Root(),B,28)
(main(),Z,16) — (Root(),C,28)
(Root(),Root1,34) — (main(),Root1,19)
(Root(),Root2,35) — (main(),Root2,19)
(Root(),Result,31) — (main(),ok,18)
(Root(),Result,36) — (main(),ok,18)
Outline Structural coverage over design Data flow coverage over design
Coupling data flow criteria
Only variables that are used or defined in the callee are
considered for du-pairs and criteria.
We need to consider implicit initialization of class and global
variables. Such default initial values that are given need to be
considered as definitions.
Transitive du-pairs (A calls B, B calls C and there is a variable
defined in A and used in C) is not supported in this analysis.
For arrays, a reference to one element is considered as a
reference to the entire array.
Outline Structural coverage over design Data flow coverage over design
Coupling data flow criteria and certification standards
Several certification standards for certifying embedded
software insist on testing using coupling criteria.
Example: DO-178C used by Federal Aviation Authority (FAA)
in the USA states that “Analysis should confirm the data
coupling and control coupling between the code components”
(page 33, section [Link]).
Outline Structural coverage over design Data flow coverage over design
Coupling data flow criteria: Additional references
M. J. Harrold and G. Rothermel, Performing data flow testing
on classes, in ACM SIGSOFT Symposium on Foundations of
Software Engineering, 154-163, 1994.
Z. Jin and J. Offutt, Coupling-based criteria for integration
testing, Software Testing, Verification and Reliability, 8(3),
133-154, 1994.
Outline Structural coverage over design Data flow coverage over design
Credits
Part of the material used in these slides are derived from the
presentations of the book Introduction to Software Testing, by
Paul Ammann and Jeff Offutt.