Understanding the Test

process
 Learning outcomes
• By the end of this unit, students should be able to:
• Explain various software testing phases
• Identify the various parameters required to successfully carry out software
testing
• Explain the Lifecycle of a test incident
• Explain various documents used in software testing.
 Testing phases
• The following are testing phases:
The Test planning and Control Phase
The Test Analysis and design Phase
The Test Implementation and Execution Phase
The Evaluating Exit Criteria and Reporting Phase
The Test Closure Phase
Cont’d
The Test planning and Control Phase
Test planning – process of creating or updating a test plan. It involves:
 Identifying software products and components to be tested
 Product and project risks (misunderstanding of requirements, government regulations, complexity….)
 Test objectives
 Scope (extent of testing)
 Teams involved
Test control – When a test plan is deviating from original plan, Test control task is carried out to
develop and apply corrective actions to get a test project on track. It includes:
 Measuring and analysing and test results such as number of review and tests performed.
 Passed and failed tests
 Number, type severity of defects
 Monitoring and documenting test progress
 Communicating progress, test results, extent of testing, risks identified and assumptions with the team
Cont’d
The Test Analysis and design Phase
This phase involves creating test designs and test procedures using test objectives
identified in test planning phase.
It includes:
 Reviewing documents that form basis of tests (requirements list or documentation,
design, interfaces specs)
 Identifying test conditions
 Designing tests for identifying potential risks possibly in s/w product
 Assessing testability
 Tools and infrastructure required
 Designing test environment
Cont’d
The Test Implementation and Execution Phase
Here, test conditions are converted into test cases, and the test environment is setup.
Test implementation involves:
 Developing and prioritising test cases, designing data for tests, and scripting instructions for executing
the tests
 Creating a schedule for executing tests
 Setup test environment

Test Execution includes:

 Executing individual and group test cases
 Record results
 Identify versions of software after Test execution
 Compare actual expected test results and report differences.
Cont’d
The Evaluating Exit Criteria and Reporting Phase
Activity where results of test execution are measured against defined objectives.
Why this stage?
 Determine whether enough testing has been done
 Declare testing as complete for a product.
 Compare actual expected test results and
 Analyse detected, fixed, rested, and confirmed or unresolved defects
 Determine if more tests are needed
 Determine if expected results need to be lowered
 Report tests performed, test outcomes (Reports should be summarised).
Cont’d
The Test Closure Phase
 Set of activities in which data from completed test activities is collected to consolidate the
testing experience.
 Typically done when product is:
 Delivered
 Cancelled
 Specific milestone is reached or
 Maintenance release has been completed
 This phase includes:
 Comparing actual and planned deliverables
 Ensuring all defects have been fully addressed
 Handover to maintenance team
 Evaluating testing process
 Analysing lessons learned
 The testing workbench process
• A process is a set of activities representing sequence of how a task is
performed
Cont’d
P – devise a Plan
D – Do a plan
C – Check the results
A - Act
Cont’d
• The following are steps involved in the tester’s workbench process:
• First inputs are given to a tester
• Once the inputs are received, a working procedure is followed and the work
performed
• If no problem had been encountered then the project is released
• If any problems are encountered, the project is sent back to rework to ensure
that the problems are addressed
• Finally the project or product is released with no major defects
Cont’d
 Test Matrices
 Representation used to determine the interrelationship between a functional event
and a test
 Every test matrix defines a set of conditions to be tested in order to verify the
proper functioning of the application system
 Cascading Test Matrices
 Test matrix that leads to another matrix, which in turn leads to yet another
 Advantages of Test Matrices
 Tests can be customised for specific functional events
 Tests on functional events can be the creation of new functional events showing a
relationship between events
 It is easy to lose sight of the interrelationships between functional events
 Cascading test matrices help reinforce interrelationships between functional events
and enable better testing
 The incident life cycle
Helps in finding and fixing various incidents that arise in different states of the life
cycle
 Incident
An event that occur during testing, that requires investigation (e.g. logical
errors, crashing programs..).
 Each incident has a life cycle
 An incident cannot be closed unless it has gone through its lifecycle (lifecycle next slide)
• Incident Log
Log and analyze incidents to gather details about the cause of the defect(s)
and report
 Root Cause
Source of a defect. When a root cause is fixed it removes or decreases
occurrence of the defect
Incident life cycle
 Incident reporting
Incident Report:
A document that records details of events that occur during testing and
that require investigation.
 Mechanism of recording incidents, defects, and their status
 Provides developers and other stakeholders with relevant feedback
 Provide means of tracking the quality of product being tested and
progress of the test process.
 Details in report include but not limited to: actual and expected
results, Date and time incidence occurred
Components of Incident Reports
 Identifier (identifies incident report)
 Summary of incident details; references to test case specification, test logs, test procedure that
revealed the problem.
 Description of the incident to enable reproduction and resolution of incident
 Impact of the incident on users and other stakeholders
 Severity of the impact on system
 Priority to fix incident
 Status of incident
 Names of people who found the problem
 Names of people responsible for incident resolution
 Change history – sequence of actions taken to isolate, repair, and confirm incident as fixed
 Other information: other areas that may be affected by the change, conclusions,
recommendations….
 Identify Test documents
 Enable planning, monitoring and assessing the test process
 Provides a basis for process improvement
 Documents at different test phases include:
Test Policies
Test Strategies
Test Plans
Test Cases
Test Scripts
Test Procedures
Test Oracles
Test Suites
Test Logs
Cont’d
Test policy
 A document describing principles, approach, objectives of an organization
regarding testing
 Testing goal must align with organizational goals and objectives: to help
achieve desired software quality
 Issued by IT management
 Provide guidance for testing
 Testing goals m
 Provides vision and framework for decision making
 Made available to all stakeholders
 Test Policy includes: Testing techniques to be used, guide on standards to be
followed while testing such as IEEE
Cont’d
Test strategies
 A document containing a description of test activities to perform
 Defines testing objectives and ways to achieve those objectives
 Test strategy determines testing effort and costs
 A suitable test strategy optimizes cost of testing v defects
 Components of Test strategy include:
 Test activities
 Test objectives
 Test techniques to implement
 Cost estimate for Testing
 Cost Required to solve defects

Where Master plan includes {Unit test plan , Integration test plan, systems test plan, performance test plan,
Security test plan, usability test plan, User acceptance test plan}
Cont’d
Test plans
 A document that describes scope, effort estimates, approach, resources, schedule
of testing activities
 You got the option to create with test plan for whole test process or separate plans
for each category of test plans.
 Test contains:
 Features t o be tested
 Tasks to be performed
 Testing team responsibilities
 Test environment
 Test design techniques
 Justification for selected test strategies and techniques
 Please note that a Test Plan DOES NOT define how testing will be performed but the
testing activities to perform
Cont’d
Test cases
 A set of input values, preconditions for execution, expected results,
post conditions for execution for a particular test objective or test
condition

 Developers, testers and quality assurance staff involved in designing

test cases
Cont’d
Test scripts
 A document that specifies actions to take for executing a set of tests
and the sequence of those actions (Put simply - A sequence of
instructions used to test functionality of a software application)

 Manual test script – instructions are executed manually

 Automated test script - instructions are executed using a test

execution tool
Cont’d
Test procedures
 A document describing the sequence of steps for the execution of a
test
 Used to prioritize execution order of tests
 Allow action steps to be combined for re-use for executing different
test cases
 Used for manual tests or used to describe instructions for a test
execution tool by creating an automation script
Cont’d
Test Oracles
 Entity used to determine expected result of a test

 Expected result is recorded (as part of a test case) then compared

with actual result

 A test oracle can be a program, user manual or an individual’s

judgement
Cont’d
Test suites
 Logical collection of test cases used to test applications

 Test suites share data and a common high-level set of objectives

 Tests cases in a suite are executed simultaneously

Cont’d
Test logs
 A document that records relevant details about execution of one or more test cases in
chronological.
 Details include:
 Test log identifier
 Test ID (specifies order of test case execution)
 Hardware and software
 Test description
 Execution description
 Name of tester
 Date and time of execution
 Status (Pass/fail/blocked)
 Provides an audit trail of test cases executed by different users
 Test case execution:
 Pass {}
 Fail {
 Blocked {Preferable conditions do not exist or unavailable)