Functional

and
Non-Functional Testing

Vikas Bajpai
Topics:

✓ Introduction
➢ Types of Testing
➢ Test Management and Automation
➢ Test Generation
➢ Test Metrics
➢ Software Test Effort Estimation Techniques
➢Types of Testing
• Black Box Testing
• White Box Testing
• Integration Testing
• System Testing
• Performance Testing
• Regression Testing
• Ad hoc Testing
Different Types of Testing Requires
Different Skill Sets

• White Box Testing --> Programming Language

• Black Box Testing --> Module Functionality
• Integration Testing --> Module Inter-relationship
• System Testing --> Real-life deployment
• Performance Testing --> Application & System Tuning
• Regression Testing --> Functionality & Change
Management
• Ad hoc testing --> Domain Knowledge
Black Box Testing
Black Box Testing:
• Is done without the knowledge of internals of
the system.
• Involves looking at the specification.
• No examining of code is required.
• Is done from Customer’s viewpoint.
• Independent test laboratories can easily
administer black box test.
 Principles of Black Box Testing [1]
• It is not possible to exhaustively test a product
• Choose tests so that we can
– Test “as much of the external functionality as
possible”
– Uncover “as many defects as possible”
– Use “as short a time as possible”

Have methodologies that choose tests that have a

higher likelihood of uncovering new defects.
Daily Black Box Test:
Black Box Test:
Reasons for Black Box Testing:[1]

• It is done based on requirements.

• Addresses the stated requirements as well as

implied requirements.

• Encompasses the end user perspectives.

• Handles valid and invalid inputs.

When to do Black Box Testing:

• From the beginning of SDLC, regardless of the

SDLC model chosen.
Techniques of Black Box Testing:
• Requirements-based testing
• Positive and negative testing
• Boundary value analysis
• Equivalence partitioning
• Decision tables
• State-based testing
• Compatibility testing
• User documentation testing
• Domain testing (leads to ad hoc testing)
Requirement Based Testing

• Validating requirements given in SRS.

• Consistency, Correctness, Completeness and

testable.

• Test Requirements Specification.

Positive and Negative Testing:

• Positive Testing: A product does what it is

supposed to do.

• Negative Testing: A product does not fail when

an unexpected input is given.
Boundary Value Analysis:

• Test cases designed with boundary values as

inputs have a high chance to find errors.

• Using boundary values, probability of finding

errors gets maximized.

* There are experiences from the past which indicate that

errors tend occur at the extreme points.
Boundary Value Analysis:
• A is any integer between 50 to 100. A program
P finds whether A is divisible by 4 or not.
Test ID Input Condition Expected Output Actual Output

1. 49 Not Divisible Display: “Invalid Input”

2. 50 Not Divisible Display: “Not Divisible by 4”

3. 51 Not Divisible Display: “Not Divisible by 4”

4. 99 Not Divisible Display: “Not Divisible by 4”

5. 100 Divisible Display: “Divisible by 4”

6. 101 Not Divisible Display: “Invalid Input”

Equivalence Class Testing:

• Identify the Equivalence Classes*

• Generate the Test Case(s)

* No. of Equivalence Classes = No. of Test Cases

Goals of Equivalence Class Partitioning:

• Completeness

• Non- Redundancy
Equivalence Class Testing:

aa bb cc

aa aa bb bb cc cc
aa bb cc

A B C
Two Mandatory Classes:

a. Valid Equivalence Class

b. Invalid Equivalence Class

Valid Input Class

Input Software Output

System

Invalid Input Class

Advantages of Equivalence Class
Testing:
• Reduces number of Test Cases.

• Good hit rate to find maximum errors with

smallest number of test cases.
Example: Equivalence Class Partitioning

• A is any integer between 50 to 100. A program

P finds whether A is divisible by 4 or not.

– Step 1: Identify Classes

» C1 = { <A>: A < 50}
» C2 = { <A>: 50 <= A <= 100}
» C3 = { <A>: A> 100 }
– Step 2: Generate Test Case(s)

………
Example: Equivalence Class Partitioning

Test ID Input Condition Expected Output Classes Covered

1. 25 Not Divisible Display: “Invalid Input” C1

2. 55 Not Divisible Display: “Not Divisible by 4” C2

3. 60 Divisible Display: “Divisible by 4” C2

4. 101 Not Divisible Display: “Invalid Input” C3

Decision Tables
• A precise yet compact way to model
complicated logic.

• Associate conditions with actions to perform.

• Can associate many independent conditions

with several actions in an elegant way.
Decision Tables
• A program’s behavior is characterized by several
decision variables.
• Each decision variable specifies a Boolean condition.
• Combinations of these decision variables lead to
different scenarios.
• Each scenario occupies a row in the decision table,
and the row also has expected results.
• One representative data point from each scenario
needs to be tested.
Decision Tables

Rule 1 Rule 2 Rule 3 Rule 4 …… Rule

N
C1 T T T F
Conditions C2 F T T F
C3 T F F T
O1 X
Outcome O2 X X
O3 X
Application Areas of Decision Tables:
• Business Analysis
• Programming
• Hardware Design
• Testing
Decision Tables
• Identify the decision variables.
• Identify the possible values of each of the decision
variables.
• Enumerate the combinations of the allowed values of
each of the variables.
• Identify the cases.
• List out the action or expected result.
• Form a table.
Example 1 on Decision Table:

……
Decision Table:
Rule 1 Rule 2

Checkbox T F
Conditions
Continue T F
Outcome Button
Example 2 on Decision Table:

……
Decision Table:
Rule 1 Rule 2

I accept the agreement F T

Conditions I do not accept the T F

agreement
Back T T
Next F T
Outcome
Cancel T T
Example 3 on Decision Table:
• Decision table on Triangle, with side A, B and C
Possibilities:
– Not a Triangle
– Scalene
– Isosceles
– Equilateral
 Decision Table:
1 2 3 4 5 6 7 8 9 10 11

C1: A < B+C ? F T T T T T T T T T T

C2: B < C+A ? -- F T T T T T T T T T

C3: C < A+B ? -- -- F T T T T T T T T

C4: A = B ? -- -- -- T T T T F F F F

C5: B = C ? -- -- -- T T F F T T F F

C6: C = A ? -- -- -- T F T F T F T F

O1: Not a Triangle X X X

O2: Scalene X

O3: Isosceles X X X

O4: Equilateral X

O5: Impossible X X X
Test Case Design using Decision Table:

Test ID A B C Expected Output

1. 4 1 2 Not a Triangle
2. 1 4 2 Not a Triangle
3. 1 2 4 Not a Triangle
4. 5 5 5 Equilateral
5. -- -- -- Impossible
6. -- -- -- Impossible
7. 2 2 3 Isosceles
8. Impossible
9. 2 3 2 Isosceles
10. 3 2 2 Isosceles
11. 3 4 5 Scalene
When to use Decision Table:

• When there is lot of decision making.

• There are important logical relationships

among input variable.

• There are cause and effect relationship

between input and output.
Compatibility Testing:
• To test the effect on product due to change in
the Infrastructure.

• Insures the correct working of product on

different infrastructure components.
Infrastructure includes:

1. Processor
2. Characteristics of Machine ( 32 bit / 64 bit)
3. Memory available
4. Operating System
5. Web Server, Application Server, Network Server
6. Database Servers
7. Compilers / linkers / loaders
 User Documentation Testing
• To check if what is stated in the document is available in
the product
• To check if what is there in the product is available in
the document
• Documents include user guides, installation guides, set-
up guides, read-me files, software release notes and
on-line help.

“UDT ensures the documentation matches the

product”
Benefits of User Documentation Testing
• User documentation testing aids in highlighting problems that
have been overlooked during reviews.
• High-quality user documentation ensures the consistency of
documentation and product, thus minimizing possible defects
reported by customers.
• Results in less difficult support calls.
• Contributes to better customer satisfaction and better morale of
support staff.
• New programmers and testers who join a project group can use
the documentation to learn the external functionality of the
product.
• Customers need less training and can proceed more quickly to
advanced training and product usage.
• Reduced training costs for customers.
Domain Testing:
• Testing the product on the basis of Domain
Knowledge.
• Performed by the test-engineers with the
knowledge of Business Domain.

“Domain Testing is the ability to design and

execute test cases that relate to the people who
will buy and use the software”
*Error Guessing

• Used when all methods fail.

• Intuition based testing.

• Uses Bug History.

*Error Guessing
ax2 + bx + c = 0

System for calculating roots of a quadratic

equation:
• When a =0
• When all inputs are negative
• When input list is empty
Pros and Cons of BBT:
• Ease of Use
• Quicker Test Case development
• Simplicity

• Script Maintenance
• Lacks Thoroughness
References:

[1]. Srinivasan Desikan et. al. “ Software Testing:

Principles and Practices”, Pearson Education,
2006.