Chapter 5 : Software Testing

Strategic approach to software

testing
■ Generic characteristics of strategic software
testing:
◻ To perform effective testing, a software team should
conduct effective formal technical reviews. By doing
this, many errors will be eliminated before testing start.
◻ Testing begins at the component level and works
"outward" toward the integration of the entire
computer-based system.
◻ Different testing techniques are appropriate at
different points in time.
◻ Testing is conducted by the developer of the software
and (for large projects) an independent test group.
◻ Testing and debugging are different activities, but
debugging must be accommodated in any testing
strategy.
Verification and Validation
■ Testing is one element of a broader topic that is often
referred to as verification and validation (V&V).
■ Verification refers to the set of activities that ensure that
software correctly implements a specific function.
■ Validation refers to a different set of activities that ensure
that the software that has been built is traceable to
customer requirements.
■ State another way:
◻ Verification: "Are we building the product right?"
◻ Validation: "Are we building the right product?“
■ The definition of V&V encompasses many of the
activities that are similar to software quality assurance
(SQA).
Software Testing Strategy for
conventional software architecture
■ A Software process & strategy for software testing may
also be viewed in the context of the spiral.
■ Unit testing begins at the vortex of the spiral and
concentrates on each unit (i.e., component) of the
software.
■ Testing progresses by moving outward along the spiral
to integration testing, where the focus is on design and
the construction.
■ Another turn outward on the spiral, we encounter
validation testing, where requirements established as
part of software requirements analysis are validated
against the software.
■ Finally, we arrive at system testing, where the software
and other system elements are tested as a whole.
■ Software process from a procedural point
of view; a series of four steps that are
implemented sequentially.
■ Initially, tests focus on each component individually,
ensuring that it functions properly as a unit.
■ Unit testing makes heavy use of white-box testing
techniques, exercising specific paths in a module's
control structure.
■ Integration testing addresses the issues associated
with the dual problems of verification and program
construction.
■ Black-box test case design techniques are the most
prevalent during integration.
■ later, validation testing provides final assurance that
software meets all functional, behavioral, and
performance requirements.
■ Black-box testing techniques are used exclusively during
validation.
■ .
Criteria for Completion of Testing
■ There is no definitive answer to state that “we have done
with testing”.
■ One response to the question is: "You're never done
testing, the burden simply shifts from you (the software
engineer) to your customer." Every time the customer/
user executes a computer program, the program is being
tested.
■ Another response is: "You're done testing when you run
out of time (deadline to deliver product to customer) or
you run out of money (spend so much money on testing).
Criteria for Completion of
Testing
■ But few practitioners would argue with these responses,
a software engineer needs more rigorous criteria for
determining when sufficient testing has been conducted.
■ Response that is based on statistical criteria: "No, we
cannot be absolutely predict that the software will never
fail, but relative to a theoretically sound and
experimentally validated statistical model, we have done
sufficient testing to say with 95 percent confidence that
program will not fail.
Unit testing strategies for conventional
software
■ Focuses verification effort on the smallest unit of
software design – component or module.
■ Using the component-level design description as a guide
◻ important control paths are tested to uncover errors
within the boundary of the module.
■ Unit test is white-box oriented, and the step can be
conducted in parallel for multiple components.
■ Unit test consists of
◻ Unit Test Considerations
◻ Unit Test Procedures
Unit Test Considerations
Contd.
■ Module interface - information properly flows into
and out of the program unit under test.
■ local data structure - data stored temporarily
maintains its integrity.
■ Boundary conditions -module operates properly
at boundaries established to limit or restrict
processing
■ Independent paths - all statements in a module
have been executed at least once.
■ And finally, all error handling paths are tested.
Unit Test Procedures
■ Perform before coding or after source code has been
generated.
■ A review of design information provides guidance for
establishing test cases. Each test case should be coupled
with a set of expected results.
■ Because a component is not a stand-alone program, driver
and/or stub software must be developed for each unit test.
■ In most applications a driver is nothing more than a "main
program" that accepts test case data, passes such data to the
component (to be tested), and prints relevant results.
■ A stub or "dummy subprogram" uses the subordinate
module's interface, may do minimal data manipulation, prints
verification of entry, and returns control to the module
undergoing testing.
■ Stubs serve to replace modules that are subordinate the
component to be tested.
Unit Test Procedures

Unit Test Environment

Integration testing
■ Integration testing is a systematic technique for constructing the
program structure while at the same time conducting tests to
uncover errors associated with interfacing.
■ The objective is to take unit tested components and build a
program structure that has been dictated by design.
■ There is often a tendency to attempt non-incremental
integration; that is, to construct the program using a "big bang"
approach.
■ A set of errors is encountered. Correction is difficult because
isolation of causes is complicated by the vast expanse of the
entire program.
■ Once these errors are corrected, new ones appear and the
process continues in a seemingly endless loop.
■ Incremental integration is the exact opposite of the big bang
approach. The program is constructed and tested in small
increments, where errors are easier to isolate and correct;
Top-down Integration
■ Top-down integration testing is an incremental approach
to construction of program structure.
■ Modules subordinate to the main control module are
incorporated into the structure in either a depth-first or
breadth-first manner.
■ Depth-first integration would integrate all components on
a major control path of the structure.
■ Selection of a major path is somewhat arbitrary and
depends on application-specific characteristics.
■ For example, selecting the left hand path,
◻ Components M1, M2 , M5 would be integrated first.
◻ Next, M8 or M6 would be integrated
◻ The central and right hand control paths are built.
Top down integration
■ Breadth-first integration incorporates all
components directly subordinate at each
level, moving across the structure
horizontally.
■ Step would be:
◻ components M2, M3, and M4 would be
integrated first
◻ next control level, M5, M6, and so on follows.
Bottom-up Integration
■ Bottom-up integration testing, as its name
implies, begins construction and testing with
atomic modules (i.e., components at the lowest
levels in the program structure)
■ Because components are integrated from the
bottom up, processing required for components
subordinate to a given level is always available
and the need for stubs is eliminated.
Bottom up integration process steps

■ Low-level components are combined into

clusters (sometimes called builds) that perform a
specific software sub function.
■ A driver (a control program for testing) is written
to coordinate test case input and output.
■ The cluster is tested.
■ Drivers are removed and clusters are combined
moving upward in the program structure.
Bottom up integration
Example
■ Components are combined to form clusters 1, 2,
and 3. Each of the clusters is tested using a
driver.
■ Components in clusters 1 and 2 are subordinate
to Ma.
■ Drivers D1 and D2 are removed and the clusters
are interfaced directly to Ma. Similarly, driver D3
for cluster 3 is removed prior to integration with
module Mb.
■ Both Ma and Mb will ultimately be integrated with
component Mc, and so forth.
Regression Testing
■ Each time a new module is added as part of integration
testing
◻ New data flow paths are established
◻ New I/O may occur
◻ New control logic is invoked
■ Due to these changes, may cause problems with
functions that previously worked flawlessly.
■ Regression testing is the re-execution of some subset of
tests that have already been conducted to ensure that
changes have not propagated unintended side effects.
■ Whenever software is corrected, some aspect of the
software configuration (the program, its documentation,
or the data that support it) is changed.
Contd.
■ Regression testing is the activity that helps to ensure that
changes do not introduce unintended behavior or additional
errors.
■ Regression testing may be conducted manually, by
re-executing a subset of all test cases or using automated
capture/playback tools.
■ Capture/playback tools enable the software engineer to
capture test cases and results for subsequent playback and
comparison.
■ Regression testing contains 3 diff. classes of test cases:
◻ A representative sample of tests that will exercise all software functions
◻ Additional tests that focus on software functions that are likely to be
affected by the change.
◻ Tests that focus on the software components that have been changed.
Contd.
■ As integration testing proceeds, the number of
regression tests can grow quite large.
■ Regression test suite should be designed to
include only those tests that address one or
more classes of errors in each of the major
program functions.
■ It is impractical and inefficient to re-execute
every test for every program function once a
change has occurred.
Alpha and Beta Testing
■ When custom software is built for one customer,
a series of acceptance tests are conducted to
enable the customer to validate all requirements.
■ Conducted by the end-user rather than software
engineers, an acceptance test can range from
an informal "test drive" to a planned and
systematically executed series of tests.
■ Most software product builders use a process
called alpha and beta testing to uncover errors
that only the end-user seems able to find.
Alpha testing
■ The alpha test is conducted at the
developer's site by a customer.
■ The software is used in a natural setting
with the developer "looking over the
shoulder" of the user and recording errors
and usage problems.
■ Alpha tests are conducted in a controlled
environment.
Beta testing
■ The beta test is conducted at one or more customer sites
by the end-user of the software.
■ beta test is a "live" application of the software in an
environment that cannot be controlled by the developer.
■ The customer records all problems (real or imagined)
that are encountered during beta testing and reports
these to the developer at regular intervals.
■ As a result of problems reported during beta tests,
software engineers make modifications and then prepare
for release of the software product to the entire customer
base.
System Testing
■ System testing is actually a series of different
tests whose primary purpose is to fully exercise
the computer-based system.
■ Although each test has a different purpose, all
work to verify that system elements have been
properly integrated and perform allocated
functions.
■ Types of system tests are:
◻ Recovery Testing
◻ Security Testing
◻ Stress Testing
◻ Performance Testing
Recovery Testing
■ Recovery testing is a system test that forces the
software to fail in a variety of ways and verifies
that recovery is properly performed.
■ If recovery is automatic (performed by the
system itself), reinitialization, checkpointing
mechanisms, data recovery, and restart are
evaluated for correctness.
■ If recovery requires human intervention, that is
mean-time-to-repair (MTTR) is evaluated to
determine whether it is within acceptable limits.
Security Testing
■ Security testing attempts to verify that protection mechanisms
built into a system will, in fact, protect it from improper break
through .
■ During security testing, the tester plays the role(s) of the
individual who desires to break through the system.
■ Given enough time and resources, good security testing will
ultimately penetrate a system.
■ The tester may attempt to acquire passwords through
externally, may attack the system with custom software
designed to breakdown any defenses that have been
constructed; may browse through insecure data; may
purposely cause system errors.
 Stress Testing
■ Stress testing executes a system in a manner that demands
resources in abnormal quantity, frequency, or volume.
For example,
1. special tests may be designed that generate ten interrupts per
second
2. Input data rates may be increased by an order of magnitude to
determine how input functions will respond
3. test cases that require maximum memory or other resources are
executed

■ A variation of stress testing is a technique called sensitivity testing

Performance Testing
■ Performance testing occurs throughout all steps in the
testing process.
■ Even at the unit level, the performance of an individual
module may be assessed as white-box tests are
conducted.
■ Performance tests are often coupled with stress testing
and usually require both hardware and software
instrumentation
■ It is often necessary to measure resource utilization
(e.g., processor cycles).
Software Maintenance
■ Software Maintenance is the process of modifying a software
product after it has been delivered to the customer.

Need for Maintenance –

Software Maintenance must be performed in order to:
■ Correct faults.
■ Improve the design.
■ Implement enhancements.
■ Interface with other systems.
■ Accommodate programs so that different hardware, software, system
features, and telecommunications facilities can be used.
■ Migrate legacy software.
■ Retire software.
Categories of Software
Maintenance
■ Corrective maintenance

■ Adaptive maintenance

■ Perfective maintenance

■ Preventive maintenance:
Software Re-Engineering
■ Software Re-Engineering is the examination and alteration of a system to
reconstitute it in a new form.
■ It involves analysing the current software system, identifying areas for
improvement, and implementing changes to the system to address these
issue

The software reengineering is necessary for having

■ a) Boost up productivity: Software reengineering increase productivity by
optimizing the code and database so that processing gets faster.

■ b) Improvement opportunity: Meanwhile the process of software

reengineering, not only software qualities, features and functionality but also
your skills are refined, new ideas hit in your mind. This makes the
developers mind accustomed to capturing new opportunities so that more
and more new features can be developed.
Software Re-Engineering
■ c) Reduction in risks: Instead of developing the software product
from scratch or from the beginning stage here developers develop
the product from its existing stage to enhance some specific
features that are brought in concern by stakeholders or its users.
Such kind of practice reduces the chances of fault fallibility.

■ d) Saves time: As we stated above here that the product is

developed from the existing stage rather than the beginning stage
so the time consumes in software engineering is lesser.

■ e) Optimization: This process refines the system features,

functionalities and reduces the complexity of the product by
consistent optimization as maximum as possible.
Reverse Engineering
■ Reverse engineering is an activity called extract
abstractions.
■ The engineer must evaluate the old program and from
the (often undocumented) source code, extract a
meaningful Specification of the processing that is
performed, the user interface that is applied, and the
program data structures or database that is used.
■ In most cases, however, the program to be reverse
engineered is not a competitor’s. Rather, it is the
company’s own work (often done many years earlier).
Reverse Engineering
■ Code restructuring
■ To accomplish code reconstructing, the source code is analyzed
using a reconstructing tool. Violations of structured programming
construct are noted and code is then reconstructed.
■ The resultant restructured code is reviewed and tested to ensure
that no anomalies have been introduced.

■ Data restructuring
■ Data restructuring begins with a reverse engineering activity.
■ Current data architecture is dissected, and the necessary data
models are defined.
■ Data objects and attributes are identified, and existing data structure
is reviewed for quality.
Forward Engineering?
■ SDLC
Black-Box Testing
■ Black-box testing, also called behavioral testing, focuses
on the functional requirements of the software.
■ Black-box testing is not an alternative to white-box
techniques. Rather, it is a complementary approach that
is likely to uncover a different class of errors than
white-box methods.
◻ Graph-Based Testing Methods
◻ Equivalence Partitioning
◻ Boundary Value Analysis
Graph-Based Testing Methods

■ The first step in black-box testing is to understand the

objects that are modeled in software and the
relationships that connect these objects.
■ software testing begins by creating a graph of important
objects and their relationships and then devising a series
of tests that will cover the graph so that each object and
relationship is exercised and errors are uncovered.
Graph-Based Testing Methods
Equivalence Partitioning
■ Equivalence partitioning is a black-box testing method
that divides the input domain of a program into classes of
data from which test cases can be derived.
■ Test-case design for equivalence partitioning is based on
an evaluation of equivalence classes for an input
condition.
Equivalence Partitioning
1. If an input condition specifies a range, one valid and two invalid equivalence classes
are defined.
Range 21 to 25
21-25 valid class
<21 invalid class
>25 invalid class

2. If an input condition requires a specific value, one valid and two invalid equivalence
classes are defined.

Value 25
25 valid class
<25 invalid
>25 invalid
Equivalence Partitioning

3. If an input condition specifies a member of a set, one valid and one invalid equivalence
class are defined.

Value 4 set {1,2,3,4,5}

4 valid class
6 invalid
Boundary Value Analysis
■ A greater number of errors occurs at the boundaries of the input
domain rather than in the “center.” It is for this reason that boundary
value analysis (BVA) has been developed as a testing technique.
■ Boundary value analysis is a test-case design technique that
complements equivalence partitioning.
■ Rather than selecting any element of an equivalence class, BVA
leads to the selection of test cases at the “edges” of the class.
Boundary Value Analysis
■ 1. If an input condition specifies a range bounded by values a and b, test cases
should be designed with values a and b and just above and just below a and b.

1-100

0-100
1-101
0-101

■ 2. If an input condition specifies a number of values, test cases should be developed

that exercise the minimum and maximum numbers. Values just above and below
minimum and maximum are also tested.

{20, 18, 21, 24}

Min-18
Max-24
WHITE-BOX TESTING

■ White-box testing, sometimes called glass-box testing or

structural testing

■ Basis path testing

■ Control Structure Testing

 Basis path testing

■ The basis path method enables the test-case designer to derive a

logical complexity for defining a basis set of execution paths.
■ Test cases derived to exercise the basis set are guaranteed to
execute every statement in the program at least one time during
testing.
■ A. Flow Graph Notation
Flow Graph Notation
Flow Graph Notation
■ Cyclomatic Complexity:Cyclomatic complexity is software metric that
provides a quantitative measure of the logical complexity of a program.
■ When used in the context of the basis path testing method, the value
computed for Cyclomatic complexity defines the number of independent
paths in the basis set of a program and provides us with an upper bound for
the number of tests that must be conducted to ensure that all statements
have been executed at least once.
■ Independent Paths in the Above Fig:
■ Path 1: 1-11
■ Path 2: 1-2-3-4-5-10-1-11
■ Path 3: 1-2-3-6-8-9-10-1-11
■ Path 4: 1-2-3-6-7-9-10-1-11
Flow Graph Notation
■ Complexity is computed using following ways
[Link] number of regions of the flow graph correspond to the
Cyclomatic complexity which is 4.

[Link] complexity, V(G), for a flow graph, G, is defined as

V (G) = (E –N) + 2
=(11-9)+2
=2+2
=4
■ where E is the number of flow graph edges, N is the number of flow
graph nodes.
Control Structure Testing

■ Control structure testing checks the various aspects like condition

testing and loop testing.
■ Condition Testing : Condition testing is a test case design method
that exercises the logical conditions contained in a program module.

■ Loop Testing: Loop testing is a white-box testing technique that

focuses exclusively on the validity of loop constructs.
Four different classes of loops can be defined: simple loops,
concatenated loops, nested loops, and unstructured loops
Control Structure Testing
Thank you.