SOFTWARE TESTING –COMPONENT LEVEL

Software Testing

Testing is the process of exercising a program with the specific intent of

finding errors prior to delivery to the end user.

Software component testing incorporates a strategy that describes

the steps to be conducted as part of testing, when these steps are
planned and then undertaken, and how much effort, time, and resources
will be required. Within the testing strategy, software component testing
implements a collection of component testing tactics that address test
planning, test-case design, test execution, and resultant data collection
and evaluation.

What is it?

• Software is tested to uncover errors that were made

inadvertently as it was designed and constructed. A software
component testing strategy considers testing of individual
components and integrating them into a working system.

Who does it?

• A software component testing strategy is developed by the project

manager, software engineers, and testing specialists.

What are the steps?

• Testing begins “in the small” and progresses “to the large.”
• Early testing focuses on a single component or on a small group of
related components and applies tests to
• uncover errors in the data and processing
• Logic that has been encapsulated by the component(s).
• After components are tested, they must be integrated until the
complete system is constructed.
1. A Strategic Approach to Software Testing

• Testing is a set of activities that can be planned in advance and

conducted systematically.
• For this reason, a template for software testing—a set of steps into
which we can place specific test-case design techniques and testing
methods—should be defined for the software process.
Following generic characteristics:
i. To perform effective testing, you should conduct technical
reviews. By doing this, many errors will be eliminated before
testing commences.
ii. Testing begins at the component level and works “outward”
toward the integration of the entire computer-based system.
 iii. Different testing techniques are appropriate for different
software engineering approaches and at different points in time.
iv. Testing is conducted by the developer of the software and (for
large projects) an independent test group.
v. Testing and debugging are different activities, but debugging
must be accommodated in any testing strategy.

 A strategy for software testing incorporates a set of tactics

that accommodate the low-level tests necessary to verify that
a small source code segment has been correctly implemented as
well as high-level tests that validate major system functions
against customer requirements. A strategy should provide
guidance for the practitioner and a set of milestones for the
manager. Because the steps of the test strategy occur at a time
when deadline pressure begins to rise, progress must be
measurable and problems should surface as early as possible.
a) Verification and Validation
 Software testing is one element of a broader topic that is often
referred to as verification and validation (V&V).
 Verification refers to the set of tasks that ensure that
software correctly implements a specific function.
 Validation refers to a different set of tasks that ensure that
the software that has been built is traceable to customer
requirements.
o Verification: “Are we building the product, right?”
o Validation: “Are we building the right product?”
 Verification and validation include a wide array of SQA
activities:
technical reviews,
quality and configuration audits,
performance monitoring,
simulation,
feasibility study,
documentation review,
database review,
algorithm analysis,
development testing,
usability testing,
qualification testing,
acceptance testing, and installation testing.
 Verification and validation include a wide array of SQA activities:
technical reviews, quality and configuration audits, performance
monitoring, simulation, feasibility study, documentation review,
database review, algorithm analysis, development testing,
usability testing, qualification testing, acceptance testing, and
installation testing. Although testing plays an extremely
important role in V&V, many other activities are also necessary.
b) Organizing for Software Testing
  For every software project, there is an inherent conflict of
interest that occurs as testing begins. The people who have
built the software are now asked to test the software.
 The software developer is always responsible for testing
the individual units (components) of the program,
ensuring that each performs the function or exhibits the
behavior for which it was designed.
 In many cases, the developer also conducts integration testing
—a testing step that leads to the construction (and test) of the
complete software architecture.
 Only after the software architecture is complete does an
independent test group become involved.
 The role of an independent test group (ITG) is to remove
the inherent problems associated with letting the builder test
the thing that has been built. Independent testing removes the
conflict of interest that may otherwise be present. After all, ITG
personnel are paid to find errors.
c) The Big Picture
 The software process may be viewed as the spiral illustrated in
diagram. Initially, system engineering defines the role of
software and leads to software requirements analysis, where
the information domain, function, behavior,
performance, constraints, and validation criteria for
software are established.
 Moving inward along the spiral, you come to design and finally
to coding.
 To develop computer software, you spiral inward along
streamlines that decrease the level of abstraction on each
turn.

Testing strategy

 A strategy for software testing may also be viewed in the context of

the spiral.
 Unit testing begins at the vortex of the spiral and concentrates on
each unit (e.g., component, class, or WebApp content object) of the
software as implemented in source code.
 Testing progresses by moving outward along the spiral to
integration testing, where the focus is on design and the
construction of the software architecture.
 Taking another turn outward on the spiral, you encounter validation
testing, where requirements established as part of requirements
modeling are validated against the software that has been
constructed.
 Finally, you arrive at system testing, where the software and other
system elements are tested as a whole.
 To test computer software, you spiral out along streamlines that
broaden the scope of testing with each turn.

Software testing steps

 Initially, tests focus on each component individually, ensuring that it

functions properly as a unit. Hence, the name unit testing.
 Unit testing makes heavy use of testing techniques that exercise
specific paths in a component’s control structure to ensure
complete coverage and maximum error detection.
 Next, components must be assembled or integrated to form the
complete software package.
 Integration testing addresses the issues associated with the dual
problems of verification and program construction.
 Test-case design techniques that focus on inputs and outputs are
more prevalent during integration, although techniques that
 exercise specific program paths may be used to ensure coverage of
major control paths.
 After the software has been integrated (constructed), a set of high-
order tests is conducted. Validation criteria (established during
requirements analysis) must be evaluated. Validation testing
provides final assurance that software meets all functional,
behavioral, and performance requirements.
 The last high-order testing step falls outside the boundary of
software engineering and into the broader context of computer
system engineering. Software, once validated, must be combined
with other system elements (e.g., hardware, people, databases).
System testing verifies that all elements mesh properly and that
overall system function and performance is achieved.

d) Criteria for “Done”

 A classic question arises every time software testing is
discussed: “When are we done testing—how do we know that
we’ve tested enough?” Sadly, there is no definitive answer to
this question, but there are a few pragmatic responses and
early attempts at empirical guidance.

2. Planning and Recordkeeping

 Many strategies can be used to test software. At one extreme,

you can wait until the system is fully constructed and then
conduct tests on the overall system in the hope of finding
errors. This approach, although appealing, simply does not
work. It will result in buggy software that disappoints all
stakeholders. At the other extreme, you could conduct tests on
a daily basis, whenever any part of the system is constructed.

 A testing strategy that is chosen by many software teams (and

the one we recommend) falls between the two extremes. It
takes an incremental view of testing, beginning with the
testing of individual program units, moving to tests designed to
facilitate the integration of the units (sometimes on a daily
basis), and culminating with tests that exercise the
constructed system as it evolves. The remainder of this
chapter will focus on component-level testing and test-case
design.

 Unit testing focuses verification effort on the smallest unit of

software design—the software component or module.

 The best strategy will fail if a series of overriding issues is not

addressed.
 Tom Gilb argues that a software testing strategy will
succeed only when software testers:

(1)specify product requirements in a quantifiable manner long

before testing commences,
(2)state testing objectives explicitly,
(3)understand the users of the software and develop a profile for
each user category,
(4)develop a testing plan that emphasizes “rapid cycle testing,”
(5)build “robust” software that is designed to test Itself
(6)use effective technical reviews as a filter prior to testing,
(7)conduct technical reviews to assess the test strategy and test
cases themselves, and
(8) Develop a continuous improvement approach for the testing
process.

i) Role of Scaffolding
• Component is not a stand-alone program; some type of
scaffolding is required to create a testing framework.
• As part of this framework, driver and/or stub software must
often be developed for each unit test.
• The unit-test environment is illustrated in Figure.
• In most applications a driver is nothing more than a “main
program” that accepts test-case data, passes such data to
the component (to be tested), and prints relevant results.

Note: Scaffolding, in the software development sense, is the technique

of generating a generic application with features common to most
modern applications already built-in.

• Stubs serve to replace modules that are subordinate (invoked by)

the component to be tested.
• A stub or “dummy subprogram” uses the subordinate module’s
interface, may do minimal
data manipulation,
prints verification of entry, and
returns control to the module undergoing testing.

Unit-test environment
 ii) Cost-Effective Testing
• Exhaustive testing requires every possible combination
of input values and test-case orderings be processed by the
component being tested (e.g., consider the move generator in
a computer chess game).
3. Test-Case Design
• It is a good idea to design unit test cases before you develop code
for a component. This ensures that you’ll develop code that will pass
the tests or at least the tests you thought of already.
• All independent paths through the control structure are exercised
to ensure that all statements in a module have been executed at
least once.
• Boundary conditions are tested to ensure that the module
operates properly at boundaries established to limit or restrict
processing. And finally, all error-handling paths are tested.
• Selective testing of execution paths is an essential task during the
unit test. Test cases should be designed to uncover errors due to
erroneous computations, incorrect comparisons, or improper control
flow.
Unit test

i) Requirements and Use Cases

  In requirements engineering starting the requirements
gathering process by working with the customers to
generate user stories that developers can refine into
formal use cases and analysis models.
 These use cases and models can be used to guide the
systematic creation of test cases that do a good job of
testing the functional requirements of each software
component and provide good test coverage overall.
ii) Traceability
To ensure that the testing process is auditable, each test
case needs to be traceable back to specific functional or
nonfunctional requirements or anti-requirements.

4. White Box Testing

 White box testing sometimes called Glass box testing or
structural testing.
 Using white box testing methods, the software engineer can
derive test cases:
1. Guarantee that all independent paths within the module have been
exercised at least once
2. Exercise all logical decisions on their true and false sides
3. Execute all loops at their boundaries and within their operational
bounds
4. Exercise internal data structures to ensure their validity.
i) Basis Path Testing
Basis path testing is a white-box testing technique first proposed
by Tom McCabe. The basis path method enables the test-case
designer to derive a logical complexity measure of a procedural
design and use this measure as a guide for defining a basis set of
execution paths. Test cases derived to exercise the basis set are
guaranteed to execute every statement in the program at least
one time during testing.
Flow Graph Notation
 Start with simple notation for the representation of control flow
(called flow graph). It represents logical control flow.

(a) Flowchart
(b) flow graph

a) flowchart is used to depict program control structure.

b) maps the flowchart into a corresponding flow graph (assuming that
no
compound conditions are contained in the decision diamonds of the
flowchart). Referring to Figure-b, each circle, called a flow graph
node, represents one or more procedural statements. A sequence
of process boxes and a decision diamond can map into a single
node. The arrows on the flow graph, called edges or links,
represent flow of control and are analogous to flowchart arrows.
An edge must terminate at a node, even if the node does not
represent any procedural statements (e.g., see the flow graph
symbol for the if-then-else construct). Areas bounded by edges and
nodes are called regions. When counting regions, we include the
area outside the graph as a region.
 An independent path is any path through the program that
introduces at least one new set of processing statements or a new
condition. When stated in terms of a flow graph, an independent
path must move along at least one edge that has not been
traversed before the path is defined. For example, a set of
independent paths for the flow graph illustrated in Figure-b is
Path 1: 1-11
Path 2: 1-2-3-4-5-10-1-11
Path 3: 1-2-3-6-8-9-10-1-11
Path 4: 1-2-3-6-7-9-10-1-11
Note that each new path introduces a new edge. The path
1-2-3-4-5-10-1-2-3-6-8-9-10-1-11
is not considered to be an independent path because it is simply a
combination of already specified paths and does not traverse any
new edges.
How do you know how many paths to look for?
 It is nothing but Cyclomatic Complexity
 Cyclomatic Complexity is a software metric that provides a
quantitative measure of the logical complexity of a program.
 When used in the context of basis path testing method, the
value computed for Cyclomatic Complexity defines the number
of independent paths in the basis set of a program and
provides us with an upper bound for the number of tests that
must be conducted to ensure that all statements have been
executed at least once.

How do you compute Cyclomatic Complexity?

1. The number of regions corresponds to the Cyclomatic
Complexity.
2. Cyclomatic Complexity, V(G), for a flow graph, G, is defined as
V(G)=E-N+2 Where E is the number of flow graph edges, and
N is the number of flow graph nodes.
3. Cyclomatic Complexity, V(G), for a flow graph, G, is defined as
V(G)=P+1 Where P is the number of predicate nodes
contained in the flow graph G
 The flow graph has four regions
 V(G)= 11 edges- 9 nodes+2=4
 V(G)= 3 predicate nodes+1=4

ii) Control Structure Testing

  Condition testing is a test-case design method that
exercises the logical conditions contained in a program
module.
 Data flow testing selects test paths of a program according
to the locations of definitions and uses of variables in the
program.

Classes of loops

Loop testing is a white-box testing technique that focuses exclusively on

the validity of loop constructs. Two different classes of loops [Bei90] can
be defined: simple loops and nested loops.
Simple Loops.
The following set of tests can be applied to simple loops, where n is the
maximum number of allowable passes through the loop.
a. Skip the loop entirely.
b. Only one pass through the loop.
c. Two passes through the loop.
d. m passes through the loop where m < n.
e. n − 1, n, n + 1 passes through the loop.

Nested Loops.
If we were to extend the test approach for simple loops to nested
loops, the number of possible tests would grow geometrically as the
level of nesting increases. This would result in an impractical number
of tests. Beizer suggests an approach that will help to reduce the
number of tests:
a. Start at the innermost loop. Set all other loops to minimum values.
b. Conduct simple loop tests for the innermost loop while holding the
outer loops at their minimum iteration parameter (e.g., loop
counter) values. Add other tests for out-of-range or excluded
values.
 c. Work outward, conducting tests for the next loop, but keeping all
other outer loops at minimum values and other nested loops to
“typical” values.
d. Continue until all loops have been tested.

5. Black-Box Testing

Black-box testing, also called behavioral testing or functional

testing, focuses on the functional requirements of the software. That
is, black-box testing techniques enable you to derive sets of input
conditions that will fully exercise all functional requirements for a
program. Black-box testing is not an alternative to white-box
techniques. Rather, it is a complementary approach that is likely to
uncover a different class of errors than white-box methods.
Black-box testing attempts to find errors in the following categories:
(1) incorrect or missing functions,
(2) interface errors,
(3) errors in data structures or external
database access,
(4) behavior or performance errors, and
(5) initialization and termination errors.
i) Interface Testing
Interface testing is used to check that the program component
accepts information passed to it in the proper order and data types
and returns information in proper order and data format Interface
testing is often considered part of integration testing. Because most
components are not stand-alone programs, it is important to make
sure that when the component is integrated into the evolving
program it will not break the build.

ii) Equivalence Partitioning

Equivalence partitioning is a black-box testing method that
divides the input domain of a program into classes of data from
which test cases can be derived. An ideal test case single-handedly
uncovers a class of errors (e.g., incorrect processing of all character
data) that might otherwise require many test cases to be executed
before the general error is observed.
Equivalence classes may be defined according to the following
guidelines:
a. If an input condition specifies a range, one valid and two
invalid equivalence classes are defined.
b. If an input condition requires a specific value, one valid and
two invalid equivalence classes are defined.
c. If an input condition specifies a member of a set, one valid
and one invalid equivalence class are defined.
d. If an input condition is Boolean, one valid and one invalid class
are defined.
Example
iii) Boundary Value Analysis
A greater number of errors occurs at the boundaries of the input
domain rather than in the “center.” It is for this reason that
boundary value analysis (BVA) has been developed as a testing
technique. Boundary value analysis leads to a selection of test cases
that exercise bounding values.
Guidelines for BVA are similar in many respects to those provided
for equivalence partitioning:
a. If an input condition specifies a range bounded by values a
and b, test cases should be designed with values a and b and
just above and just below a and b.
b. If an input condition specifies a number of values, test cases
should be developed that exercise the minimum and
maximum numbers. Values just above and below minimum
and maximum are also tested.
c. Apply guidelines 1 and 2 to output conditions. For example,
assume that a temperature versus pressure table is required
as output from an engineering analysis program. Test cases
should be designed to create an output report that produces
the maximum (and minimum) allowable number of table
entries.
a. If internal program data structures have prescribed
boundaries (e.g., a table has a defined limit of 100 entries), be
certain to design a test case to exercise the data structure at
its boundary.

Example If we want to test a field where values from 1 to 100

should be accepted, then we choose the boundary values: 1-1, 1,
1+1, 100-1, 100, and 100+1. Instead of using all the values from
1 to 100, we just use 0, 1, 2, 99, 100, and 101.