MULUNGUSHI UNIVERSITY

DEPARTMENT OF COMPUTER SCIENCE & IT

ICT382 – Software Engineering

SOFTWARE TESTING

Sinyinda Muwanei, PhD

 OUTLINE
 Introduction
 Black Box Testing
 White Box Testing
 Documentation of the Test
Activity
 INTRODUCTION
 The testing phase has now evolved and become a field of its own
 In the classical waterfall model, the testing activity is deferred to
the end of the development process which should always be
avoided
 As you begin the software development process, the activity of
testing should begin as well, and continue throughout the process
 Who is responsible for testing?
 The responsibility of testing software lies squarely on the software
developer. He/she plays a key role when it comes to unit testing
 With the rise of testing as a field, however, not only developers test
software, but there are also what is known as software testers who
test not only the logic of the system, but the user experience as
well
 What is software testing?
 Software testing has to do with assessing the quality of the software
with respect to the extent to which the software meets the
requirements of the users.
 It’s a means of ensuring that quality products are delivered to the
target market
 Software that is not tested is software that can not be depended on
and that users have no confidence in.
 There are several levels of testing and these are:
• unit testing
• integration testing
• system testing
• acceptance testing
 (cont’d)
 Unit testing: This is testing at an extremely detailed level. Various units
of the system, such as classes, packages etc., are tested individually for
proper functionality.
 Integration testing: This level of testing looks at the communication
between the different units or classes, and whether the communication
is successful or not. It looks at how classes interface with other classes.
 System testing: System testing checks the requirements outlined in the
requirements specification document, maps them to the functioning of
the software, and looks at whether the system meets each
requirement.
 Acceptance testing: Under this level of testing, the user/client tests the
system, checks if it meets his/her requrements and provides feedback.
 Types of
testing
 Alpha testing: Under alpha testing, the client/user is invited to a
testing exercise performed at the organization developing the
software and the software is tested on the organization’s computers.
 Beta testing: Under beta testing, a copy of the software under
development is sent to the client’s organization and the testing is
done from there, on the client’s computers.
 Functional testing: This type of testing has to do with testing what the
system does. It is usually answered with either yes or no for each
functionality.
 Non-functional testing: This is testing how the system performs.
These tests are answered using a range. For example, in a range of 1-
5, how is the system performing in terms of speed?
 (cont’d)
 Black box testing: This is testing without knowing the internal
structure of the system. It is only interested in testing and checking if
the requirements have been met (that is, is the expected output for
a given input produced when passed through a given function?).
 As a software tester, 80% of your time is spent on black box testing
 Black box testing can be done without knowing the programming
language used for the software.

 Note: Exhaustive testing is not a possibilty. If it were, testing would be a very

expensive undertaking which would take a lot of time.
 (cont’d)
 White box testing: This is testing while monitoring the internal
structure of the system. The interest is how the inputs are being
converted into outputs.
 There are certain faults in the software that will not be found during
black box testing. These are addressed during white box testing
when the code is inspected.
 White box testing is 20% of a software testers efforts.
 (cont’d)
 Dynamic testing: This is any type of testing that includes executing the
software, such as black box and white box testing.
 Static testing: This is any type of testing that does not include
executing software, such as testing an algorithm with values.
 Conformation testing: This is testing done after a bug has been fixed
to ascertain whether it has actually been fixed or not.
 Smoke testing: This is testing done when a new release of the system
is created. The new features are tested before an overall testing of
the software with the new features is done.
 Regression testing: This is testing done after bug fixes have been done
where the unchanged areas are tested to check whether they were
affected by the bug fixes or not.
 Objectives of testing
 The following are the reasons why testing is done
[Link] find defects in the software
[Link] determine whether the software meets the intended
requirements
[Link] provide information to stakeholders
4. To reduce the risk of developing software, including the risk on the
lives of people such as with real time systems
[Link] order to make sure that the deployed software is in compliance
with laws
[Link] build confidence with respect to users and user organizations
 Test process
 There are two groups of thought that can be followed when it
comes to the testing process.
[Link] phases in the first group of thought are:
• Planning phase
• Design phase
• Execution of tests
 (cont’d)
2. The phases in the second group of thought are:
• Test planning
• Test monitoring/monitoring and control
• Test analysis
• Test design
• Test implementation
• Test execution
• Test completion phase
 TEST SCENARIOS
 A test scenario is any functionality that can be tested.
 They are sometimes referred to as test possibilities.
 When looking at test scenarios, one should put themselves in the end
user’s shoes with respect to the functionality and use of the system.
 One should figure out real use case scenarios and look at the various
use cases that are critical for the application or system being tested.
 How do you create test scenarios?
1. Carefully study the requirements document
2. Isolate every requirement and identify possible user actions that
need to be tested. Remember to analyse and frame any possible
form of abuse of the system.
3. Generate the test scenarios that will cover the business flow
involved in the operation of the software and list these scenarios
down.
4. Using the listed scenarios from step 3, create a traceability matrix
which maps the scenarios to the requirements. The traceability
matrix is important to ensure that all the requirements are
catered for in the scenarios.
5. Show the final test scenarios to your immediate supervisor.
 (cont’d)
 The first test scenario you’ll have is one that is going to have valid
data used everywhere (what will happen if valid data is used?).
 For the next set of scenarios, isolate each of the controls. That is,
put valid data in all the controls except one which will have invalid
data.
 Next, look at the rules within every control. For each rule in each
control, generate separate test scenarios.
 Try out test scenarios where you do not put any data in one control
while the others have valid data
 How to write test
cases
 The following table shows how to write test cases using a login
page as an example.
Test case Pre- Test Expected Test Test Actual Test Status
title conditions steps result suite environme result
nt
1 Verify login User should [Link] User is (module/ (where the (filled Pass/fail/bloc
with valid already be valid logged in package testing is when ked (used
username registered usernamei and where being carried testing is when the
and using valid [Link] directed to the out, e.g. carried test cannot
password credentials valid xyz page function Windows 10, out as be executed
password would be Chrome, either until another
[Link] on e.g. etc.) successfu part of the
sign in Login) l or system is
button unsucces completed)
sful)
2 … … … … … … … …
 (cont’d)
 The above table can be modified by adding a column ‘inputs’ after
test steps where the inputs used during the testing are
highlighted.
 Another column for comments can also be added at the end of the
table.
 BLACK BOX TESTING
 In black box testing, we want to be able to test the system without
knowing the internal structure of that system.
 All we want to know is what the inputs are for the functionalities and
what their corresponding expected outputs are.
 The internal functioning of the system does not need to be known.
 The following are the various black box testing techniques:
[Link] Partitioning Technique (EPT)
[Link] Value Analysis (BVA)
[Link] Table Testing (DTT)
[Link] Transition Testing (STT)
[Link] Testing (PT)
 Equivalence Partitioning
Technique
 In this black box testing technique, look at the function and see
whether it can be partitioned.
 If it can, input values for testing between each partition. These values
should be any values inside the range and not the boundary values.
 Every partition must be catered for and tested.
 Look at the partitions and choose input values inside the partitions.
 Apart from values within the range, also test values far from the very
least and very most boundary.
 (cont’d)
 For example, consider a function to print a student’s grade
according to the following rules:
0 <= grade < 45 : D
45 <= grade < 55 : D+ 55 <= grade < 60 : C 60 <= grade < 65 : C+ 65 <=
grade < 70 : B 70 <= grade < 75 : B+ 75 <= grade < 85 : A
85 <= grade <= 100 : A+
 (cont’d)
 Using equivalence partitioning on this example, the function will be divided
into 8 partitions for each of the grades.
 Test cases will be generated for each of these partitions using values within
the given ranges and also test cases using values far from 0 (the least most
boundary) and 100 (the very most boundary), as shown below:
TC1: input = 35 TC2: input = 47 TC3: input = 58
...
TC9: input = -5.3 TC10: input = 110
 Boundary Value Analysis
 In this technique, test what happens at the boundaries of each
partition.
 Input values for testing that are at the boundaries of each partition
and not within the range.
 Apart from the boundary values, also test values that are very close
to the very least and very most boundary.
 Considering the previous example, test cases will be generated for
each partition using the boundary values and also test cases using
values near 0 (the least most boundary) and 100 (the very most
boundary), as shown below:
 (cont’d)
TC1: input = 0
TC2: input = 45
TC3: input = 55
...
TC9: input = 100
TC10: input = -0.2
TC11: input = 101
 State Transition Testing
 This type of testing is used when a system has states and
transitions.
 For example, an ATM can have various states. These states and their
transitions are shown in the state transition diagram below:
 (cont’d)
 To test this system using state transition testing, first you must
construct a state transition diagram and number every state like in
the diagram above.
 Next, identify the test cases using the following steps:
[Link] the number of dead states (states with no transistions leading
out of them)
[Link] at the paths transitioning from the initial state to a dead state.
These will be your test cases.
 In our partial representation of the system, there are two dead states
which are states 4 and 5. Therefore, our test cases will be:
 (cont’d)
TC1: 1 → 5 (execute this test case by providing the correct pin)
TC2: 1 → 2 → 3 → 4 (execute this test case by entering the wrong pin
three times)
TC3: 1 → 2 → 5 (excecute this test case by entering the wrong pin
followed by the correct pin)
TC4: 1 → 2 → 3 → 5 (execute this test case by entering the wrong pin
twice followed by the correct pin)
 Decision Table
Testing
 This technique is used when ou have a scenario where different
combinations of conditions result in different outcomes.
 Example:
• for you to get a discount in a store, you must buy items more than K1000
and you must have a gold subscription
• this example has two conditions; buying items more than K1000 and having
a gold subscription
• test what happens when different combinations of these conditions are true
or false. This can be done using a decision table like the one shown below:
 (cont’d)
Conditions
Buy items more than K1000 T T F F
Gold subscription T F T F
Action:
Apply discount x(TC1)
Don’t apply discount x(TC2) x(TC3) x(TC4)

•from creating the table, a number of test cases have been generated:
TC1: both conditions are true
TC2: condition 1 is true and condition 2 is false
TC3: condition 1 is false and condition 2 is true TC4: both conditions
are false
 WHITE BOX TESTING
 The following are the white box testing
techniques:
[Link] Coverage
[Link] Coverage
[Link] Coverage
[Link] Coverage
 Statement Coverage
 A statement is a single line of code
 The goal of statement coverage testing is to cover every statement
in the program
 If every statement is covered (tested), then we say we have
achieved a 100% statement coverage
 The formula to compute the extent to which the coverage has been
done is given by:
𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑠𝑡𝑎𝑡𝑒𝑚𝑒𝑛𝑡𝑠 𝑡𝑒𝑠𝑡𝑒𝑑
Statement coverage 𝑡𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑠𝑡𝑎𝑡𝑒𝑚𝑒𝑛𝑡𝑠
x 100
=
 (cont’d)
 Example:
IF day = “Monday”:
Statement a
ELSE IF day = “Tuesday”
Statement b
ELSE IF day = “Friday”
Statement c
ELSE
Statement d
 The above example has four statements to be tested under
statement coverage.
 Decision Coverage
 This technique is also referred to as branch
coverage.
 The goal is to test all decision outcomes
 Example: consider the flowchart below
 (cont’d)
 From the above flowchart there are six decision outcomes. These
are the decision outcomes to be tested and not the conditions
 The formula for decision coverage is as follows:
𝑁𝑜.𝑑𝑒𝑐𝑖𝑠𝑖𝑜𝑛 𝑜𝑢𝑡𝑐𝑜𝑚𝑒𝑠 𝑡𝑒𝑠𝑡𝑒𝑑
DC = 𝑇𝑜𝑡𝑎𝑙 𝑁𝑜.𝑑𝑒𝑐𝑖𝑠𝑖𝑜𝑛 𝑜𝑢𝑡𝑜𝑚𝑒𝑠
x 100
 Condition Coverage
 In condition coverage, we are not only focused on the outcome of
the decision, but on each of the conditions in each decision.
 In the flowchart for the previous example, Decision 1 contained:
condition 1 && condition 2 && condition 3
 To perform condition coverage testing on Decision 1, test each of the
conditions in the decision in true and false cases until all of the
conditions have been thoroughly tested.
 (cont’d)
 Suppose you have the following pseudocode:
read x read y read z
IF (x > y) AND (x > z)
print “x is greatest”
ELSE IF (y > z)
print “...”
 For each of the three conditions above, check their true and false
cases giving various test cases as shown in the table below
 (cont’d)
Inputs
x y z
1 x > y = false x > z = true 10 12 9
2 x > y = false x > z = false … … …
3 x > y = true x > z = true … … …
4 x > y = true x > z = false … … …
 Path Coverage
 The goal of this technique is to test all possible paths
 Consider the flowchart below
 (cont’d)
 Check all possible routes from beginning to end
 For the above flowchart, paths to be tested are:
path 1: 1 → 2
path 2: 1 → 3 → 4
path 3: 1 → 3 → 5
 Each path will be its own test case as highlighted
below
Test case Path Inputs
1 1 →3 (values that will lead to Decision 1 being equal to false
2 … …
 Manual Testing
• Testing done without automation tools
• Whenever you test start with manual testing then proceed with
automation testing (Tools and scripts)
Steps in Testing
• Analyse requirements, if any gaps provide feedback to
development team
• Test planning phase.
• Test case creation : here you create test cases
• Execution of Test cases.
• If bugs identified, report to developers to fix

• Test Report Generation

 Objective of software testing
1. Find defects/ Prevent defects
2. Ensure that the product is of good quality (look and feel,
UI,interface , user experience is checked and reported)
3. To ensure the software meets requirements
4. Ensure that the software is fit for use
 Why is Software Testing Necessary
1. Failures in the past:
• Flipkart(October 2014)
• Yahoo (September 22,2016)
• TESCO (March 2012)
• RBS (June 2015)

2. Loss of money
3. Improves the quality of software
4. It reduces the risk
5. To ensure no reputational damage
 SEVEN PRINCIPLES OF TESTING
1. Begin Testing Early
2. Testing shows the presence of errors but never shows that the
product is defect free
3. Exhaustive testing is not possible
4. Defect clustering : means defects are clustered in modules and
where they are found, it is likely that they are more defects in
those modules.
5. PESTSIDE Paradox: e.g in agriculture when same pestiside is used,
the crop would be immune. Similarly, using the same test cases
will not give you new defects. To find new defects, it is necessary
to use different test cases.
 Software Development Life Cycle
• A series of phases involved in software development
1. Requirement Gathering Analysis and Specification
2. Design
3. Development
4. Testing
5. Deploy and Production
6. Maintenance
 Software Testing Life Cycle
• Fits with the software development life cycle
• It is the process of testing software
• Remains the same for all software development methodologies
• Phases:
1. Requirements analysis
• Role here is to understand requirements and check if there are any gaps
2. Test planning:
• You need to write the test plan
3. Test case design and development
4. Test Environment setup
5. Test Execution
6. Test reporting
7. Test closure