Test Strategy – wizard to efficient testing

Test Strategy – Wizard to Efficient Testing

Author – Priyanka Prakash

InterGlobe Technologies
Gurgaon, India

International Software Testing Conference, 2009

International Software Testing Conference 2009 1

Test Strategy – wizard to efficient testing

Contents:
-------------------------------------------------------------------------------------------------
1. Abstract

2. Introduction
2.1 Stages of Testing Life Cycle

3. Test Strategy
2.1 Basic Contents of Test Strategy Document

4. Test Strategy in Testing Framework

4.1 Testing Artifacts

5. Test Plan Vs Test Strategy

5.1 Test Plan
5.2 Components of Test Plan
5.3 Test Strategy
5.4 Components of Test Strategy

6. Changing Trends

7. Why Test Strategy

8. Derivatives of Test Strategy

9. Role of Test Strategy in Test Management

[Link] Strategy Sample

11. References

[Link] the Author

International Software Testing Conference 2009 2

 REPORTING

Test Strategy – wizard to efficient testing

Test Strategy: Wizard to Efficient Testing

1.0 Abstract:

Test strategy is the road map for a Test Project. It is a medium of telling the rest of the
world how you are going to approach the Testing Project. It aims at defining a realistic path
to achieve Project Testing Goals. Test Strategy serves as a guideline that defines testing
methodology, test approach, test management, Testing techniques to be applied, testing
tools to be used and so on.

The paper is attempted to give a perspective on Test Strategy vs. Test Planning. Both these
terms are used interchangeably across organizations. This paper covers details about test
strategy, why test strategy, what are the differences between test strategy and test plan,
why it is important and advantages of test strategy. It will also summarize with a sample
test strategy template.

2.0 Introduction:
-------------------------------------------------------------------------------------------------
Today, success of every application/project has high dependency on Testing. Importance of
testing is well understood by its effectiveness, even though the Testing Team in any
organization is bounded by Strict Time Frames. Other parameters like cost effectiveness
also indirectly impacts the overall testing process.

Even in such environment with strict boundaries and water tight schedules, there is an ever
increasing pressure on testing teams to pave path of efficient and effective testing. It is
therefore important that every organization has a well defined testing process guided by a
framework. A framework should in turn have a well laid Test Strategy that is a guide to
Testing Team.

The basis of achieving efficient and effective testing is to mandatorily have well managed
testing team. Testing Team should be aware of the forthcoming assignment related to
scope, tasks, their role, defect handling, tools to be used well in advance. Since there are
strict time-frames and there is pressure to perform effective as well as efficient testing, the
testing team should be given enough time to understand the blueprint of the testing
strategy to be followed covering every corner of the testing life cycle.

2.1 Stages of Testing Life Cycle

ANALYSIS

PLANNING

DESIGNING

EXECUTION

REPORTING

International Software Testing Conference 2009 3

Test Strategy – wizard to efficient testing

Test Strategy incorporates guidelines for the testing team in all above stages of testing life
cycle. Basic pointers are highlighted below:

Analysis – test strategy specificies the incidents for analysis of scope and updation of
Testing Matrix
Planning – test strategy document helps to detail out plan in terms of Time and Resource
management for all activities of testing
Designing - test strategy guides through test case and scenario design for defined scope in
system testing, integration testing, regression testing and the like as defined in test
strategy. The coverage factor is well understood through the highlights in the test strategy
document.
Execution – test strategy helps testing team to understand how the defect will be reported –
in defect tracker, which testing tool to be used, what needs to be executed in various
rounds of testing.
Reporting – Test Strategy guides through the reporting structure, format and concerned
stakeholders for different testing execution cycles like UAT, System Testing along with
different rounds of testing.

3.0 Test Strategy:

-------------------------------------------------------------------------------------------------

Test Strategy is a document that acts as a guide to testing team in understanding the
testing methodology and approach employed in a particular project. Before commencement
of Testing in any project, it is important that the Testing Team has thorough knowledge of
Testing Scope, Approach, Time Frames, Role of testers and Testing Cycles. Apart from this,
all risks, dependencies, mitigation plans should be well highlighted. In order to have a
complete awareness of above factors, a Test Strategy document is designed.

The test strategy document is important to structurise the testing assignment. It is the
document that is sometimes presented even to the higher management and even client to
understand the Testing Perspective that would be undertaken by the Testing Team in the
forthcoming assignment.

3.1 Basic Contents of Test Strategy Document

Following are the basic contents covered in the test strategy document:
1) Finalized scope and detailed analysis of coverage in testing in broad perspective
2) What is out of scope of testing
3) Associated Risks
4) Dependencies of testing the items in scope
5) Testing Techniques
6) Testing Tools
7) Defect Management
8) Testing Cycles to be employed
9) Testing Methodologies to be employed
10)Resources and roles of each roles
11)Iterations/cycles of execution
12)Stakeholders in terms of higher management/client
13)Completion reporting

International Software Testing Conference 2009 4

Test Strategy – wizard to efficient testing

4.0 Test Strategy in Testing Framework

-------------------------------------------------------------------------------------------------

Every Organization today has a well defined Testing Framework. A framework can be
defined as a process – a well defined step by step guideline for entire lifecycle of Testing
Assignment. This usually describes roles and activities of Testing Team Members in the
Software Development Life Cycle for Testing Phases as well as for Pure Testing Projects. Al
processes are well documented, right from understanding scope of testing through Defect
Reporting to signing off any testing project.

Once Testing Scope is finalized, the Test Plan is created by the Project Manager/Test Team
Lead. The Test Plan identifies the high level scope of testing in the project, high level
estimates, resources required in the testing assignments.

After the Test Plan Document is well laid before the stakeholders, the Test Strategy
Document is designed. Test Plan acts as a guideline to a more detailed document on Testing
Assignment to be undertaken i.e. Test Strategy. A Test Strategy has detailed analysis on
the Scope as laid out in the Test Plan in terms of Risks and Dependencies.

In order to test the specified items under scope, the Lead needs to identify the basic
requirements to proceed with Testing in terms of the Environment, Database
Access/Availability – may be blank database or database with live data, Expert Resource
Availability may be Specific Testing Technique Expert or Functional Expert and so forth.

On having a clear understanding of the Scope, it is important to identify the Risks that
testing team can acknowledge at the time of testing. It may be in terms of criticality of the
scope or unusual access to a particular database or unavailability of particular environment.
These risks need to be explicitly stated to all stakeholders. There can be a mention of
mitigation plan associated with the risk if any.

The finalized scope may also have some basic dependencies – like the availability of signed
of specification documents or unavailability of resource due to planned/unplanned cause at
the critical test cycle of execution.

Based on the scope, it is important to identify the Resources to be involved in testing

phases and their role.

Egg: Team Lead – Only 1 team lead may be required to coordinate with the project
manager/ development team/ client s and when required.
2 Senior Test Experts could be required with at least one having thorough domain
knowledge.
Apart from the above, are 2 or 3 test executives can be required for testing.

Based on the estimates, Test Strategy identifies the Cycles of Testing that is required for
effective testing the scope.

Egg: 10 days of Testing Can be strategized as:

Testing Round 1 – 4 PD/Days

Testing Round 2 – 3 PD/Days with Defect Verifications from Round 1
Regression – 3 days with defect verification from Round 2
Usually, Round 2 is rounded off with 70% days/PD allocated to Testing Round 1.

International Software Testing Conference 2009 5

Test Strategy – wizard to efficient testing

It is important to explicitly state out the criteria of defects that would be required to be
fixed in Round 2 or Regression Round – e.g.: May be only Priority 1 and Priority 2 defects
would be fixed in Round 2 while rest can be scheduled to be fixed in Regression Round.

It is necessary that the Test Strategy document well defines actions/mitigation plans for
certain unconditional scenarios like when defects of only Priority 3 are identified in Round 1,
then Round 2 could be skipped and regression round could be continued to save time with
verification of fixes in round 1.

Test Strategy Document delves deep in identifying the Testing Methodology- the testing
techniques and testing approach for the Testing Assignments.
Testing Techniques like – Manual Testing: System, Regression, Security, Database,
Usability, UI, Specific Functional Verification and validations; apart from Automated Testing
Techniques.

4.1 Testing Artifacts

Phase 1 Phase 2 Phase 3 Phase 4 Phase 5

Analysis Planning Designing Execution Reporting

TESTING ARTIFACTS

SCOPE TEST TEST DEFECT DEFECT

DEFINITION PLAN SCENARIOS REPORTING UPDATE PER
BUILD

QUERY TEST TEST TEST SET FOR

LOG STRATEGY CASES DIFFERENT BUILD
ROUNDS COMPLETION
REPORT
TEST CASE
REVIEW REGRESSION
CHECKLIST TEST SUITE FINAL TEST
COMPLETION
REPORT
DEFECT
REVIEW
CHECKLIST

5.0 Test Plan Vs Test Strategy

-------------------------------------------------------------------------------------------------

5.1 Test Plan

A Test Plan document is a low level document and normally developed by project manager.
This document defines “Testing Approach” to achieve testing objectives. The Test Plan is
normally derived from the Business Requirement Specification document.

The Test Plan document is a static document meaning that it is not updated too often. It
sets the standards for testing processes and activities some companies include the “Test

International Software Testing Conference 2009 6

Test Strategy – wizard to efficient testing

Approach” or “Strategy” inside the Test Plan, which is fine and it is usually the case for
small projects. However, for larger projects, there is one Test Plan document and different
number of Test Plans for each phase or level of testing.

5.2 Components of Test Plan Document

• Test Plan id
• Introduction
• Test items
• Features to be tested
• Features not to be tested
• Test techniques
• Testing tasks
• Suspension criteria
• Features pass or fail criteria
• Test environment (Entry criteria, Exit criteria)
• Test deliverables
• Staff and training needs
• Responsibilities
• Schedule

5.3 Test Strategy

The Test Strategy document on the other hand, is derived from the Product Description,
Software Requirement Specification SRS, or Use Case Documents.
The Test Strategy document is usually prepared by the Test Lead or Test Manager and the
focus of the document is to describe what to test, how to test, when to test and who will do
what test.

It is not uncommon to have one Master Test Strategy which is a common document for the
test phases and each test phase has specific Test Plan documents.

There is much debate, as to whether the Test Strategy document should also be a static
document like the Test Plan document or should it be updated every often to reflect
changes according to the direction of the project and activities.

5.4 Components of the Test Strategy document

• Scope and Objectives

• Business issues
• Roles and responsibilities
• Communication and status reporting
• Test deliverability
• Industry standards to follow
• Test automation and tools
• Testing measurements and metrics
• Risks and mitigation
• Defect reporting and tracking
• Change and configuration management
• Training plan

International Software Testing Conference 2009 7

Test Strategy – wizard to efficient testing

This is a standard approach to prepare test plan and test strategy documents, but things
can vary company-to-company.

6.0 Changing Trends

-------------------------------------------------------------------------------------------------------
Earlier the Testing Team would only prepare Test Plan which would specifically describe all
scope, estimates, techniques, methodology, risks and dependencies for testing team along with Defect
-Tracker to be user and other tools to be used if any.

As time changed, there was need to identify Testing Approach explicitly. It became
necessary to view the finalized scope in low level in terms of Test Plan and high Level in
terms of Test Strategy.

So, many organizations have started using Test Plan as well as Test Strategy where Test
Plan is the mother document derived from the finalized scope while other document like
Test Strategy is derived from Test Plan.

Many other organizations use the term Test Plan and Test Strategy interchangeably. They
have also started creating only Test Strategy Document and do not use Test Plan
Document. There is only one document – Test Strategy which solves purpose of High as well
as Low Level rituality to the Testing Assignment for all stakeholders.

However, in many organizations, still Test Plan is used and they create a Test Approach
document separately to suffice high level Test Plan document.

In many organizations, Testing phase starts and the Test Manager is “controlling” the
activities, the test plan is updated to reflect any deviation from the original plan. Test
Strategy can be fixed/static document and requires to be changed very rarely based on the
change in Test Plan. After all, Planning and Control are continuous activities in the formal
test process.

7.0 Why Test Strategy

-------------------------------------------------------------------------------------------------

Test Strategy document is the proof of work for the testing team. The higher management
and client can always b presented with document at the initial stage for to understand the
effectiveness and efficiency of testing process to be implied in the upcoming assignment.
The document is the guideline to the testing team at all stages of testing in terms of
approach, time frames, process, tools to be employed, different techniques, methodologies
and so on.

Many a times the Development Team refers this document to identify dates allocated for
fixation of defects in between the testing cycle, priority of defects planned to be fixed and
so on.

8.0 Derivatives of Test Strategy

-------------------------------------------------------------------------------------------------
Once Test Strategy document is completed and presented to testing team, the testing
phases begin. The testing team starts preparing all relevant documents starting from
Traceability Matrix through Final System Test Completion report.

International Software Testing Conference 2009 8

Test Strategy – wizard to efficient testing

Test Strategy document is the guideline for the various artifacts prepared by the testing
team. Following artifacts are the derivatives of Test Strategy:
1) Traceability Matrix
2) Test Scenarios – both negative and positive
3) Test Cases for system testing, integration testing, and the like
4) Regression Test Suite
5) Build Completion Report
6) Final Test Completion Report

9.0 Role of Test Strategy in Test Management

-------------------------------------------------------------------------------------------------
Essentially test management is the basis of effective testing. Following are the challenges to
Test Management:
1) How the scope documents will be managed
2) What will be the Test Cases Repository
3) How will the defects be reported
4) How will the defect life cycle tracked
5) How to keep control over the Test Coverage in all cycles of testing

Test Strategy document thus plays an important role in test management. This document
lays down the answers to all above questions long before the testing activities begin. The
testing team thus has well formulated solutions to cover all composite challenges mentioned
above.

In absence of any of the answers, the testing team can have big risks. Even if they manage
to work towards testing the scope, their work might not be well presented or managed.

Following are the risks that can encircle testing team in absence of the solutions to above
mentioned challenges:
1) In scope and out of scope might not be clear
2) Risks and dependencies might not have been addressed
3) Allocated resources might not be clear with role and responsibility
4) How to address defects could be a dilemma
5) Tracking defects to closure might be missed
6) How to test and how much to test would always be a question mark
7) There would not be a standard approach to testing within one testing team

10.0 Sample Test Strategy Document

-------------------------------------------------------------------------------------------------
Here is the sample Test Strategy document.

Sample Test
[Link]

International Software Testing Conference 2009 9

Test Strategy – wizard to efficient testing

11.0 References
-------------------------------------------------------------------------------------------------
1. In house projects, frameworks and formats used
2. Forums in ITToolbox

About the Author:

The Author has 7years of experience in software testing. She is currently working as Senior
Software Engineer with InterGlobe Technologies, Gurgaon, India.
Priyanka Holds a [Link] Degree in Computer Engineering. She has several published articles
and papers in various Conferences, Online Journals and Seminars to her credit. She has
done Certificate in Software Estimation.

About the Organization:

InterGlobe Technologies is India’s leading pure-play Travel Technology organization,
providing software services and solutions to travel corporations worldwide in the areas of
Information Technology and Business Process Outsourcing.

IGT is a quality, process & metrics driven organization, with its IT practices assessed at
CMMI Level 5 and BPO processes ISO 9001:2000 certified with a roadmap for ISMS
27001:2005, PCMM & COPC certification.

Please Visit us at [Link]

International Software Testing Conference 2009 10