Object-Oriented Software Engineering
Using UML, Patterns, and Java
Chapter 11, Testing
Outline
¨ Terminology ¨ System testing
¨ Types of errors Function testing
¨ Structure Testing
Dealing with errors
Performance testing
¨ Quality assurance vs Testing Acceptance testing
¨ Component Testing Installation testing
Unit testing
Integration testing
¨ Testing Strategy
¨ Design Patterns & Testing
nd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 2
Some Observations
¨ It is impossible to completely test any nontrivial module or any
system
Theoretical limitations: Halting problem ??
Practial limitations: Prohibitive in time and cost
¨ Testing can only show the presence of bugs, not their absence
(Dijkstra)
total number of execution paths?
loop 200 times
nd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 3
Testing Activities
Subsystem Requirements
Unit System
Code Test Analysis
Design Document
Tested Document User
Subsystem
Subsystem Manual
Unit
Code Test
Tested Integration Functional
Subsystem
Test Test
Integrated Functioning
Subsystems System
Tested Subsystem
Subsystem Unit
Code Test
All
Alltests
testsby
bydeveloper
developer
Cf. levels of testing
nd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 4
Testing Activities continued
Client’s
Global Understanding User
Requirements of Requirements Environment
Functioning Validated Accepted
System PerformanceSystem System
Acceptance Installation
Test Test Test
Usable
Tests
Testsby
byclient
client System
Tests
Testsby
bydeveloper
developer
User’s understanding
System in
Use
Tests
Tests(?)
(?) by
byuser
user
nd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 5
Level of abstraction
Levels of Testing in V Model
system system
requireme integration
nts
software
requireme acceptance
nts test
preliminar software
te
y integratio
an
t s
design n
aly
and
componen
z
detailed
ea
i
t
nt
design
nd
test
eg
r
de
at
code & unit
sig
e
debug test
n
Time
nd Bruegge & Allen H. Dutoit
N.B.: component test vs. unit test; acceptance test vs. system integration
Object-Oriented Software Engineering: Using UML, Patterns, and Java 6
Test Planning [Pressman]
¨ A Test Plan: ¨ A test plan includes:
covers all types and phases of test objectives
testing schedule and logistics
guides the entire testing process test strategies
who, why, when, what test cases
developed as requirements, procedure
functional specification, and high- data
level design are developed expected result
should be done before procedures for handling
implementation starts problems
nd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 7
Fault Handling Techniques
Fault Handling
Fault Tolerance
Fault Avoidance Fault Detection
Design Atomic Modular
Reviews
Methodology Transactions Redundancy
Configuration
Verification
Management
Testing Debugging
Unit Integration System Correctness Performance
Testing Testing Testing Debugging Debugging
nd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 8
Quality Assurance encompasses Testing
Quality Assurance
Usability Testing
Scenario Prototype Product
Testing Testing Testing
Fault Avoidance Fault Tolerance
Atomic Modular
Configuration Transactions Redundancy
Verification
Management
Fault Detection
Reviews
Debugging
Walkthrough Inspection
Testing
Correctness Performance
Unit Integration System Debugging Debugging
Testing Testing Testing
nd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 9
Types of Testing
¨ Unit Testing:
Individual subsystem
Carried out by developers
Goal: Confirm that subsystems is correctly coded and carries out
the intended functionality
¨ Integration Testing:
Groups of subsystems (collection of classes) and eventually the
entire system
Carried out by developers
Goal: Test the interface among the subsystem
nd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 10
System Testing
Terminology:
¨ System Testing: system testing here = validation testing
The entire system
Carried out by developers
Goal: Determine if the system meets the requirements (functional
and global)
¨ Acceptance Testing: 2 kinds of Acceptance testing
Evaluates the system delivered by developers
Carried out by the client. May involve executing typical
transactions on site on a trial basis
Goal: Demonstrate that the system meets customer requirements
and is ready to use
¨ Implementation (Coding) and testing go hand in hand
nd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 11
Unit Testing
¨ Informal:
Incremental coding Write a little, test a little
¨ Static Analysis:
Hand execution: Reading the source code
Walk-Through (informal presentation to others)
Code Inspection (formal presentation to others)
Automated Tools checking for
syntactic and semantic errors
departure from coding standards
¨ Dynamic Analysis:
Black-box testing (Test the input/output behavior)
White-box testing (Test the internal logic of the subsystem or object)
Data-structure based testing (Data types determine test cases)
Which is more effective, static or dynamic analysis?
nd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 12
Black-box Testing
¨ Focus: I/O behavior. If for any given input, we can predict the
output, then the module passes the test.
Almost always impossible to generate all possible inputs ("test
cases") why?
¨ Goal: Reduce number of test cases by equivalence partitioning:
Divide input conditions into equivalence classes
Choose test cases for each equivalence class. (Example: If an object
is supposed to accept a negative number, testing one negative
number is enough)
If x = 3 then …
If x > -5 and x < 5 then …
What would be the equivalence classes?
nd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 13
Black-box Testing (Continued)
¨ Selection of equivalence classes (No rules, only guidelines):
Input is valid across range of values. Select test cases from 3
equivalence classes:
Below the range
Within the range Are these complete?
Above the range
Input is valid if it is from a discrete set. Select test cases from 2
equivalence classes:
Valid discrete value
Invalid discrete value
¨ Another solution to select only a limited amount of test cases:
Get knowledge about the inner workings of the unit being tested =>
white-box testing
nd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 14
White-box Testing
¨ Focus: Thoroughness (Coverage). Every statement in the component is
executed at least once.
¨ Four types of white-box testing
Statement Testing
Loop Testing
Path Testing
Branch Testing
nd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 15
White-box Testing (Continued)
¨ Statement Testing (Algebraic Testing): Test single statements
¨ Loop Testing:
Cause execution of the loop to be skipped completely. (Exception:
Repeat loops)
Loop to be executed exactly once
Loop to be executed more than once
¨ Path testing:
Make sure all paths in the program are executed
¨ Branch Testing (Conditional Testing): Make sure that each
possible outcome from a condition is tested at least once
if ( i = TRUE) printf("YES\n");else printf("NO\n");
Test cases: 1) i = TRUE; 2) i = FALSE
nd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 16
White-Box Testing
Loop Testing
[Pressman]
Simple
loop
Nested
Loops
Concatenated
Loops Unstructured
Loops
Why is loop testing important?
nd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 17
Self reading
Comparison of White & Black-box Testing 25.1.2002
¨ White-box Testing: ¨ Both types of testing are needed
Potentially infinite number of ¨ White-box testing and black box
paths have to be tested testing are the extreme ends of a
White-box testing often tests testing continuum.
what is done, instead of what ¨
should be done
Any choice of test case lies in
between and depends on the
Cannot detect missing use cases
following:
¨ Black-box Testing: Number of possible logical paths
Potential combinatorical Nature of input data
explosion of test cases (valid &
invalid data) Amount of computation
Often not clear whether the Complexity of algorithms and
selected test cases uncover a data structures
particular error
Does not discover extraneous
use cases ("features")
nd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 18
The 4 Testing Steps
1.
[Link]
Selectwhat
whathas
hasto
tobe
be 3.
[Link]
Developtest
testcases
cases
measured
measured AAtest
testcase
caseisisaaset
setof
oftest
test
Analysis: data
dataororsituations
situationsthat
thatwill
Analysis:Completeness
Completenessof of will
requirements be
beused
usedtotoexercise
exercisethetheunit
unit
requirements
Design: (code,
(code,module,
module,system)
system)being
being
Design:tested
testedfor
forcohesion
cohesion tested
Implementation: testedor
orabout
aboutthetheattribute
attribute
Implementation:Code
Codetests
tests being
beingmeasured
measured
2.
[Link]
Decidehow
howthe
thetesting
testingisis 4.
[Link]
Createthe
thetest
testoracle
oracle
done
done An
Anoracle
oraclecontains
containsofofthe
the
Code
Codeinspection
inspection predicted
predictedresults
resultsfor
foraaset
setof
of
Proofs test
testcases
Proofs(Design
(DesignbybyContract)
Contract) cases
Black-box, The
Thetest
testoracle
oraclehas
hasto
tobe
Black-box,white
whitebox,
box, be
Select written
writtendown
downbefore
beforethe
the
Selectintegration
integrationtesting
testing actual
strategy actualtesting
testingtakes
takesplace
place
strategy(big
(bigbang,
bang,bottom
bottom
up,
up,top
topdown,
down,sandwich)
sandwich)
Next module
nd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 19
Who Tests the Software?
[Pressman]
developer independent tester
Understands the system Must learn about the system,
but, will test "gently" but, will attempt to break it
and, is driven by "delivery" and, is driven by quality
nd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 20