Introduction to Software Testing

Module 4

Input Space Partition Testing

2
Input Domains
• The input domain for a program contains all the possible inputs to that
program
• For even small programs, the input domain is so large that it might as well be
infinite
• Testing is fundamentally about choosing finite sets of values from the input
domain
• Input parameters define the scope of the input domain
• Parameters to a method
• Data read from a file
• Global variables
• User level inputs
• Domain for each input parameter is partitioned into regions
• At least one value is chosen from each region

3
Benefits of ISP

• Can be equally applied at several levels of testing

• Unit
• Integration
• System

• Relatively easy to apply with no automation

• Easy to adjust the procedure to get more or fewer tests

• No implementation knowledge is needed

• just the input space

4
ISP
• Input space partitioning describes the input domain of the software
• Domain (D) are partitioned into blocks (b1, b2, .., bn)

5
 Partitioning Domains
• Domain D
• Partition scheme q of D
• The partition q defines a set of blocks, Bq = b1 , b2 , … bQ
• The partition must satisfy two properties :
1. blocks must be pairwise disjoint (no overlap)
2. together the blocks cover the domain D (complete)

b1 b2 bi  bj = ,  i  j, bi, bj  Bq

b3  b=D
b  Bq

6
Using Partitions – Assumptions
• Choose a value from each block
• Each value is assumed to be equally useful for testing
• Application to testing
• Find characteristics in the inputs : parameters, semantic
descriptions, …
• Partition each characteristic
• Choose tests by combining values from characteristics
• Example Characteristics
• Input X is null
• Order of the input file F (sorted, inverse sorted, arbitrary, …)
• Min separation of two aircraft
• Input device (DVD, CD, VCR, computer, …)

7
 Choosing Partitions
• Choosing (or defining) partitions seems easy, but is easy to get wrong
• Consider the “order of file F”
Solution:
Each characteristic should address just
one property
b1 = sorted in ascending order
b2 = sorted in descending order File F sorted ascending
b3 = arbitrary order - b1 = true
- b2 = false
but … something’s fishy … File F sorted descending
- b1 = true
- b2 = false
What if the file is of length 1?

The file will be in all three blocks …

That is, disjointness is not satisfied
8
Properties of Partitions

• If the partitions are not complete or disjoint, that means the

partitions have not been considered carefully enough

• They should be reviewed carefully, like any design attempt

• Different alternatives should be considered

• We model the input domain in five steps …

9
 Modeling the Input Domain
• Step 1 : Identify testable functions
• Individual methods have one testable function
• In a class, each method often has the same characteristics
• Programs have more complicated characteristics—modeling documents such
as UML use cases can be used to design characteristics
• Systems of integrated hardware and software components can use devices,
operating systems, hardware platforms, browsers, etc

• Step 2 : Find all the parameters

– Often fairly straightforward, even mechanical
– Important to be complete
– Methods : Parameters and state (non-local) variables used
– Components : Parameters to methods and state variables
– System : All inputs, including files and databases

10
Modeling the Input Domain (cont)
• Step 3 : Model the input domain
• The domain is scoped by the parameters
• The structure is defined in terms of characteristics
• Each characteristic is partitioned into sets of blocks
• Each block represents a set of values
• This is the most creative design step in using ISP
• Step 4 : Apply a test criterion to choose combinations of
values
– A test input has a value for each parameter
– One block for each characteristic
– Choosing all combinations is usually infeasible
– Coverage criteria allow subsets to be chosen

• Step 5 : Refine combinations of blocks into test inputs

– Choose appropriate values from each block

11
Two Approaches to Input Domain Modeling
1. Interface-based approach
• Develops characteristics directly from individual input parameters
• Simplest application
• Can be partially automated in some situations

2. Functionality-based approach
• Develops characteristics from a behavioral view of the program under test
• Harder to develop—requires more design effort
• May result in better tests, or fewer tests that are as effective

Input Domain Model (IDM)

12
1. Interface-Based Approach
• Mechanically consider each parameter in isolation
• This is an easy modeling technique and relies mostly on syntax
• Some domain and semantic information won’t be used
• Could lead to an incomplete IDM
• Ignores relationships among parameters

13
 2. Functionality-Based Approach
• Identify characteristics that correspond to the intended functionality
• Requires more design effort from tester
• Can incorporate domain and semantic knowledge
• Can use relationships among parameters
• Modeling can be based on requirements, not implementation
• The same parameter may appear in multiple characteristics, so it’s harder to
translate values to test cases

14
15
Steps 1 & 2 – Identifying Functionalities, Parameters and
Characteristics
• A creative engineering step
• More characteristics means more tests
• Interface-based : Translate parameters to characteristics
• Candidates for characteristics :
• Preconditions and postconditions
• Relationships among variables
• Relationship of variables with special values (zero, null, blank, …)
• Should not use program source – characteristics should be based on the input
domain
• Program source should be used with graph or logic criteria
• Better to have more characteristics with few blocks
• Fewer mistakes and fewer tests

16
Steps 1 & 2 : Interface vs Functionality-Based
public boolean findElement (List list, Object element)
// Effects: if list or element is null throw NullPointerException
// else return true if element is in the list, false otherwise
Interface-Based Approach
Two parameters : list, element
Characteristics :
list is null (block1 = true, block2 = false)
list is empty (block1 = true, block2 = false)

Functionality-Based Approach
Two parameters : list, element
Characteristics :
number of occurrences of element in list
(0, 1, >1)
element occurs first in list
(true, false)
element occurs last in list
(true, false)
17
Step 3 : Modeling the Input Domain
• Partitioning characteristics into blocks and values is a very creative
engineering step
• More blocks means more tests
• The partitioning often flows directly from the definition of
characteristics and both steps are sometimes done together
• Should evaluate them separately – sometimes fewer
characteristics can be used with more blocks and vice versa
• Strategies for identifying values :
• Include valid, invalid and special values
• Sub-partition some blocks
• Explore boundaries of domains
• Include values that represent “normal use”
• Try to balance the number of blocks in each characteristic
• Check for completeness and disjointness
18
 Interface-Based IDM – TriTyp
• TriTyp, from Chapter 3, had one testable function and three
integer inputs

First Characterization of TriTyp’s Inputs

Characteristic b1 b2 b3
q1 = “Relation of Side 1 to 0” greater than 0 equal to 0 less than 0

q2 = “Relation of Side 2 to 0” greater than 0 equal to 0 less than 0

q3 = “Relation of Side 3 to 0” greater than 0 equal to 0 less than 0

• A maximum of 3*3*3 = 27 tests

• Some triangles are valid, some are invalid
• Refining the characterization can lead to more tests …
19
Interface-Based IDM – TriTyp (cont)
Characteristic b1 b2 b3 b4
q1 = “Refinement
Secondof qCharacterization
1” greater than 1 equal to 1 equal
of TriTyp’s to 0 less than 0
Inputs
q2 = “Refinement of q2” greater than 1 equal to 1 equal to 0 less than 0

q3 = “Refinement of q3” greater than 1 equal to 1 equal to 0 less than 0

• A maximum of 4*4*4 = 64 tests

• This is only complete because the inputs are integers (0 . . 1)

Possible values for partition q1

Characteristic b1 b2 b3 b4
Side1 52 1 0 -1
-5

Test boundary conditions 20

Functionality-Based IDM – TriTyp
• First two characterizations are based on syntax–parameters and their type
• A semantic level characterization could use the fact that the three integers
represent a triangle
Geometric Characterization of TriTyp’s Inputs
Characteristic b1 b2 b3 b4
q1 = “Geometric Classification” scalene isosceles equilateral invalid

• Oops … something’s fishy … equilateral is also isosceles !

• We need to refine the example to make characteristics valid

Correct Geometric Characterization of TriTyp’s Inputs

Characteristic b1 b2 b3 b4
q1 = “Geometric Classification” scalene isosceles, not equilateral invalid
equilateral
21
Functionality-Based IDM – TriTyp (cont)
• Values for this partitioning can be chosen as

Possible values for geometric partition q1

Characteristic b1 b2 b3 b4
Triangle (4, 5, 6) (3, 3, 4) (3, 3, 3) (3, 4, 8)

22
Functionality-Based IDM – TriTyp (cont)
• A different approach would be to break the geometric characterization into
four separate characteristics

Four Characteristics for TriTyp

Characteristic b1 b2
q1 = “Scalene” True False

q2 = “Isosceles” True False

q3 = “Equilateral” True False

q4 = “Valid” True False

• Use constraints to ensure that

– Equilateral = True implies Isosceles = True
– Valid = False implies Scalene = Isosceles = Equilateral = False
23
Using More than One IDM
• Some programs may have dozens or even hundreds of
parameters
• Create several small IDMs
• A divide-and-conquer approach
• Different parts of the software can be tested with
different amounts of rigor
• For example, some IDMs may include a lot of invalid
values
• It is okay if the different IDMs overlap
• The same variable may appear in more than one
IDM
24
 Step 4 – Choosing Combinations of Values
• Once characteristics and partitions are defined, the next step is to choose test
values
• We use criteria – to choose effective subsets
• The most obvious criterion is to choose all combinations …

All Combinations (ACoC) : All combinations of blocks from all

characteristics must be used.

• Number of tests is the product of the number of blocks in each

characteristic : Q
 (B )
i=1 i
• The second characterization of TriTyp results in 4*4*4 = 64 tests
– too many ?

25
 ISP Criteria – Each Choice
• 64 tests for TriTyp is almost certainly way too many
• One criterion comes from the idea that we should try at least one value from
each block

Each Choice (EC) : One value from each block for each
characteristic must be used in at least one test case.
• Number of tests is the number of blocks in the largest
characteristic Q
Max (B )
i=1 i
For TriTyp: 2, 2, 2
1, 1, 1
0, 0, 0
-1, -1, -1
26
ISP Criteria – Pair-Wise
• Each choice yields few tests – cheap but perhaps ineffective
• Another approach asks values to be combined with other values

Pair-Wise (PW) : A value from each block for each characteristic

must be combined with a value from every block for each other
characteristic.
• Number of tests is at least the product of two largest
characteristics
Q Q
(Max (B )
i=1 i ) * (Max (B )
j=1, j!=i j )
For TriTyp: 2, 2, 2 2, 1, 1 2, 0, 0 2, -1, -1
1, 2, 1 1, 1, 0 1, 0, -1 1, -1, 2
0, 2, 0 0, 1, -1 0, 0, 2 0, -1, 1
-1, 2, -1 -1, 1, 2 -1, 0, 1 -1, -1, 0
27
ISP Criteria –T-Wise
• A natural extension is to require combinations of t values instead of 2

t-Wise (TW) : A value from each block for each group of t

characteristics must be combined.

• Number of tests is at least the product of t largest

characteristics
• If all characteristics are the same size, the formula is
Q
(Max i=1 i ) (B ) t

• If t is the number of characteristics Q, then all combinations

• That is … Q-wise = AC
• t-wise is expensive and benefits are not clear

28
ISP Criteria – Base Choice
• Testers sometimes recognize that certain values are important
• This uses domain knowledge of the program

Base Choice (BC) : A base choice block is chosen for each

characteristic, and a base test is formed by using the base choice for
each characteristic. Subsequent tests are chosen by holding all but
one base choice constant and using each non-base choice in each
other characteristic.
• Number of tests is one base test + one test for each other block


Q
1 + i=1 (Bi -1 )
For TriTyp: Base 2, 2, 2 2, 2, 1 2, 1, 2 1, 2, 2
2, 2, 0 2, 0, 2 0, 2, 2
2, 2, -1 2, -1, 2 -1, 2, 2
29
Base Choice Notes

• The base test must be feasible

• That is, all base choices must be compatible
• Base choices can be
• Most likely from an end-use point of view
• Simplest
• Smallest
• First in some ordering
• The base choice is a crucial design decision
• Test designers should document why the choices were
made

30
ISP Criteria – Multiple Base Choice
• Testers sometimes have more than one logical base choice

Multiple Base Choice (MBC) : One or more base choice blocks are
chosen for each characteristic, and base tests are formed by using each
base choice for each characteristic. Subsequent tests are chosen by
holding all but one base choice constant for each base test and using
each non-base choices in each other characteristic.
• If there are M base tests and mi base choices for each
characteristic:
M +  i=1 (M * (Bi - mi ))
Q

For TriTyp: Base

2, 2, 2 2, 2, 0 2, 0, 2 0, 2, 2
2, 2, -1 2, -1, 2 -1, 2, 2
1, 1, 1 1, 1, 0 1, 0, 1 0, 1, 1
1, 1, -1 1, -1, 1 -1, 1, 1 31
ISP Coverage Criteria Subsumption
All Combinations
Coverage
AC

T-Wise Multiple Base

Coverage Choice Coverage
TW MBC

Pair-Wise Base Choice

Coverage Coverage
PW BC

Each Choice
Coverage
EC

32
Constraints Among Characteristics

• Some combinations of blocks are infeasible

• “less than zero” and “scalene” … not possible at the same time
• These are represented as constraints among blocks
• Two general types of constraints
• A block from one characteristic cannot be combined with a specific block
from another
• A block from one characteristic can ONLY BE combined with a specific block
form another characteristic
• Handling constraints depends on the criterion used
• AC, PW, TW : Drop the infeasible pairs
• BC, MBC : Change a value to another non-base choice to find a feasible
combination

33
 Example Handling Constraints
• Sorting an array
Blocks from other
• Input : variable length array of arbitrary type
characteristics are
• Outputs : sorted array, largest value, smallest value
irrelevant

Characteristics: Partitions: Blocks must be

• Length of array• Len { 0, 1, 2..100, 101..MAXINT combined
}
• Type of elements
• Type { int, char, string, other }
• Max value • Max { 0, 1, >1, ‘a’, ‘Z’, ‘b’, …, ‘Y’ }
• Min value • Min { … } Blocks must be
• Position of max value
• Max Pos { 1, 2 .. Len-1, Len } combined
• Position of min value
• Min Pos { 1, 2 .. Len-1, Len }

34
 Input Space Partitioning Summary
• Fairly easy to apply, even with no automation

• Convenient ways to add more or less testing

• Applicable to all levels of testing – unit, class, integration, system, etc.

• Based only on the input space of the program, not the implementation

Simple, straightforward, effective, and widely

used in practice

35
Problem
• Consider a simple program to classify a triangle. Its
input consists of three positive integers (say x, y, z)
and the data types for input parameters ensures
that these will be integers greater than zero and
less than or equal to 100. The three values are
interpreted as representing the lengths of the sides
of a triangle. The program then prints a message to
the standard output that states whether the
triangle, if it can be formed, is scalene, isosceles,
equilateral, or right-angled

36
• Solution: Following possible boundary conditions are
formed:
• 1. Given sides (A; B; C) for a scalene triangle, the sum
of any two sides is greater than the third and so, we
have boundary conditions A + B > C, B + C > A and A+
C > B.
• 2. Given sides (A; B; C) for an isosceles triangle two
sides must be equal and so we have boundary
conditions A = B, B = C or A = C.
• 3. Continuing in the same way for an equilateral
triangle the sides must all be of equal length and we
have only one boundary where A = B = C.
• 4. For right-angled triangles, we must have A 2+B2= C 2

37
• On the basis of the above boundary conditions, test cases are
designed as follows

38
 Functional Testing
• FUNCTIONAL TESTING is a type of software testing that validates
the software system against the functional
requirements/specifications.
• The purpose of Functional tests is to test each function of the
software application, by providing appropriate input, verifying
the output against the Functional requirements.
• Functional testing mainly involves black box testing and it is not
concerned about the source code of the application.
• This testing checks User Interface, APIs, Database, Security,
Client/Server communication and other functionality of the
Application Under Test.
• The testing can be done either manually or using automation.

39
What do you test in Functional Testing?

• The prime objective of Functional testing is checking the

functionalities of the software system. It mainly concentrates
on –
• Mainline functions: Testing the main functions of an
application
• Basic Usability: It involves basic usability testing of the
system. It checks whether a user can freely navigate through
the screens without any difficulties.
• Accessibility: Checks the accessibility of the system for the
user
• Error Conditions: Usage of testing techniques to check for
error conditions. It checks whether suitable error messages
are displayed.
40
How to do Functional Testing
• Following is a step by step process on How to do
Functional Testing :
➢Understand the Functional Requirements
➢Identify test input or test data based on
requirements
➢Compute the expected outcomes with selected test
input values
➢Execute test cases
➢Compare actual and computed expected results

41
42
43
44
Explain the types of functional testing.

45
Functional Testing Concepts of Howden
• The four key concepts in functional testing are:
➢Precisely identify the domain of each input and
each output variable
➢Select values from the data domain of each
variable having important properties
➢Consider combinations of special values from
different input domains to design test cases
➢Consider input values such that the program under
test produces special values from the domains of
the output variables

46
Different Types of Variables
• Numeric Variables
• A set of discrete values
• A few contiguous segments of values
• Arrays
• An array holds values of the same type, such as integer and real. Individual
elements of an array are accessed by using one or more indices.
• Substructures
• A structure means a data type that can hold multiple data elements. In the
field of numerical analysis, matrix structure is commonly used.
• Subroutine Arguments
• Some programs accept input variables whose values are the names of
functions. Such programs are found in numerical analysis and statistical
applications

47
Howden’s Functional Testing Summary

Let us summarize the main points in functional testing:

• Identify the input and the output variables of the program and their
data domains

• Compute the expected outcomes as illustrated in Figure 9.5(a), for

selected input values

• Determine the input values that will cause the program to produce
selected outputs as illustrated in Figure 9.5(b).

48
Howden’s Functional Testing Summary

Figure 9.5: Obtaining output values from an input vector (a), and obtaining an input vector from
an output value (b) in functional testing
49