VERIFICATION, VALIDATION
Verification: Demonstration of consistency, completeness,
and correctness of the software artifacts at each stage of
and between each stage of the software life-cycle.
Different types of verification: manual inspection, testing,
formal methods
Verification answers the question: Are we building the product
right?
The software should conform to its specification.
Validation: The process of evaluating software at the end
of the software development to ensure compliance with
CHAPTER SIX respect to the customer needs and requirements.
Validation can be accomplished by verifying the artifacts
produced at each stage of the software development life cycle
Validation answers the question: Are we building the right
product?
The software should do what the user really requires.
SOFTWARE TESTING DEFINATION
Goal of testing Software testing is the process of executing a
finding faults in the software
demonstrating that there are no faults in the software (for
software system to determine whether it matches
the test cases that has been used during testing) its specification and executes in its intended
environment.
It is not possible to prove that there are no faults in
the software using testing
Testing should help locate errors, not just detect their
presence
a “yes/no” answer to the question “is the program correct?”
is not very helpful
Testing should be repeatable
could be difficult for distributed or concurrent software
effect of the environment, uninitialized variables
1
WHO TESTS THE SOFTWARE?
[Pressman]
TYPES OF TESTING
Black box testing: are named so as the source
code is not used during software testing. Instead
the executable binary machine code is used.
Focus: I/O behavior. If for any given input, we
can predict the output, then the module passes
developer independent tester the test.
Almost always impossible to generate all possible
Understands the system Must learn about the system, inputs ("test cases")
but, will test "gently" but, will attempt to break it
and, is driven by "delivery" and, is driven by quality
BLACK BOX TESTS TYPES OF TESTING
Selection of equivalence classes (No rules, only
input
guidelines):
output
Input is valid across range of values. Select test cases
interface from 3 equivalence classes:
Below the range
Within the range
Above the range
1. Does it perform the specified functions?
[Link] it handle obvious errors in input?
2
WHITE BOX TESTING WHITE BOX TESTING
are named so as the source code is used during Focus: Thoroughness (Coverage). Every statement in the
component is executed at least once.
software testing. White box tests are also known Four types of white-box testing
as structural tests as they test small parts Statement Testing
(structures) of the software product Loop Testing
Path Testing
Branch Testing
test 1
test 2
COMPARISON OF WHITE & BLACK-BOX TESTING LVELS OF TESTING
White-box Testing: Unit (Module) testing
Potentially infinite number of paths have to be tested testing of a single module in an isolated environment
White-box testing often tests what is done, instead of what
should be done
Cannot detect missing use cases
Integration testing
testing parts of the system by combining the modules
Black-box Testing:
Potential combinatorical explosion of test cases (valid &
invalid data) System testing
Often not clear whether the selected test cases uncover a testing of the system as a whole after the integration
particular error phase
Does not discover extraneous use cases ("features")
Acceptance testing
testing the system as a whole to find out if it satisfies
the requirements specifications
3
UNIT TESTING INTEGRATION TESTING
Involves testing a single isolated module Integration testing: Integrated collection of
modules tested as a group or partial system
Note that unit testing allows us to isolate the errors
to a single module
we know that if we find an error during unit testing it is in Integration plan specifies the order in which to
the module we are testing
combine modules into partial systems
Modules in a program are not isolated, they interact
with each other. Possible interactions:
calling procedures in other modules Different approaches to integration testing
receiving procedure calls from other modules Bottom-up
sharing variables Top-down
For unit testing we need to isolate the module we
want to test, we do this using two thing
MODULE STRUCTURE
A B
• We assume that
BOTTOM-UP INTEGRATION
the uses hierarchy is
a directed acyclic graph. Only terminal modules (i.e., the modules that do not
• If there are cycles merge call other modules) are tested in isolation
D them to a single module
Modules at lower levels are tested using the
level 1 C H previously tested higher level modules
Non-terminal modules are not tested in isolation
level 0 E F G I
Requires a module driver for each module to feed the
• A uses C and D; B uses D; C uses E and F; D uses F, G, H and I; H uses I test case input to the interface of the module being
• Modules A and B are at level 3; Module D is at level 2 tested
Modules C and H are at level 1; Modules E, F, G, I are at level 0
However, stubs are not needed since we are starting with
• level 0 components do not use any other components
the terminal modules and use already tested modules
• level i components use at least one component on level i-1 and no
when testing modules in the lower levels
component at a level higher than i-1
4
BOTTOM-UP INTEGRATION TOP-DOWN INTEGRATION
Only modules tested in isolation are the modules
A B
which are at the highest level
After a module is tested, the modules directly called
D
by that module are merged with the already tested
module and the combination is tested
C H
Requires stub modules to simulate the functions of
the missing modules that may be called
E F G I However, drivers are not needed since we are starting with
the modules which is not used by any other module and use
already tested modules when testing modules in the higher
levels
TOP-DOWN INTEGRATION SYSTEM TESTING, ACCEPTANCE TESTING
A B System and Acceptance testing follows the
integration phase
testing the system as a whole
D
Test cases can be constructed based on the
C
H
the requirements specifications
main purpose is to assure that the system
E G I meets its requirements
F
5
SYSTEM TESTING, ACCEPTANCE TESTING REGRESSION TESTING
Alpha testing is performed within the You should preserve all the test cases for a program
development organization
During the maintenance phase, when a change is
made to the program, the test cases that have been
Beta testing is performed by a select group of saved are used to do regression testing
figuring out if a change made to the program introduced
friendly customers any faults
Regression testing is crucial during maintenance
Stress testing It is a good idea to automate regression testing so that all
push system to extreme situations and see if it fails test cases are run after each modification to the software
large number of data, high input rate, low input rate,
etc. When you find a bug in your program you should
write a test case that exhibits the bug
Then using regression testing you can make sure that the
old bugs do not reappear
BLACK BOX TESTING WHITE BOX TESTING
Checks that the product conforms to specifications Allows tester to be sure every statement has been
Cannot determine how much code has been tested tested.
Difficult to discover missing functionality.
Omissions detected by
black-box tests Commissions detected by
Program Program White-box tests
6
END…