Performance Test Strategy

Chinnikrishna
12/14/2022

Page 1
Document Revision History
Revision Author Revision Date Description of Changes Date
Number Distributed

1.0 Chinnikrishna 12/14/2022 12/14/2022

Page 2
Table of Contents

1. Document Information.......................................................................................................5

1.1 Distribution List...............................................................................................................5

2 Performance Test Strategy Endorsement...............................................................................5

2.1 Stakeholder Endorsement..............................................................................................5

2.2 Recommendation............................................................................................................5

2.3 Comments.......................................................................................................................6

3 Introduction............................................................................................................................6
3.1 Application Architecture.................................................................................................6

3.1.1 Application Architecture Diagram:..........................................................................7

<Diagram>..................................................................................................................................7

3.2 Schematic Representation..............................................................................................7

3.3 Purpose of Performance Test Strategy...........................................................................7

3.4 Intended Audience of the Strategy.................................................................................7

3.5 Glossary..........................................................................................................................8

4 Performance Testing Process..................................................................................................8

4.1 Identify Key Scenarios.....................................................................................................9

4.2 Identify Work Load.........................................................................................................9

4.3 Identify Metrics...............................................................................................................9

4.4 Create Test Cases..........................................................................................................10

4.5 Simulate Load...............................................................................................................10

Page 3
 4.5.1 Performance Testing Types...................................................................................10

4.5.2 Performance Test Approach.................................................................................12

4.5.3 Performance Test Activates and deliverables.......................................................12

4.6 Analyze the Results.......................................................................................................14

5 Test Environments and Tools................................................................................................14

5.1 Test Environment..........................................................................................................14

5.2 Testing Tools.................................................................................................................14

6 Schedule...............................................................................................................................15
6.1 High-level test schedule................................................................................................15

6.2 Schedule dependencies................................................................................................15

7 Assumptions, dependencies.................................................................................................15
7.1 Assumptions.................................................................................................................15

7.2 Dependencies...............................................................................................................16

7.3 Out Of Scope :...............................................................................................................16

8 Upfront Issues and risks........................................................................................................16

8.1 Upfront Issues...............................................................................................................16

8.2 Risks..............................................................................................................................16

Page 4
1. Document Information

Title <Performance Test Strategy Document Name>

To communicate all the artifacts and elements required to
Document Purpose
execute performance tests against the <Project Name>
Version Number 0.34
Approval State Initial Draft/Ready for Review

1.1 Distribution List

Action (Info/
Name Area
Review/ Sign-off)
XXXXXX Project Manager Review
XXXXX QA Team Manager Review
Performance Testing Review
XXXXXX
team
Review
Project Manager
XXXXXX

2 Performance Test Strategy Endorsement

2.1 Stakeholder Endorsement

Your signature below indicates acceptance of the framework and focus of the
Performance Test Plan.

2.2 Recommendation

I have reviewed the Performance Test Plan in detail and recommend (tick the
appropriate box):

 The Performance Test Plan has been developed in accordance with the user’s criteria
and procedures. Testing may proceed as scheduled.

Page 5
  Approval is given for the Performance Test Plan to proceed, subject to minor alterations
as indicated. No further reference to me is required.
 The significant alterations, as indicated, are to be remedied and reviewed by me before
commencement of the performance testing.
 Rejection of the entire Performance Test Plan as given.

2.3 Comments

Please fill in with your particulars as appropriate:

Name Signature Date

3 Introduction

<Overview of the Project Name>

3.1 Application Architecture

This application is using the following technologies

< Define all the technologies>

< Description of technologies>

The below one is the architecture diagram

Page 6
3.1.1 Application Architecture Diagram: -

<Diagram>

3.2 Schematic Representation

<Diagram>

3.3 Purpose of Performance Test Strategy

This document describes the <Project Name> performance testing strategy, along with
the specific methodologies and techniques used by the Quality Assurance testing team to
plan, organize and manage the testing activities for the Performance testing.

Performance testing poses many challenges and varies a lot from web and client server
application testing. Performance Test requires different standards for different purposes
and so there is no single uniform testing methodology for Performance Test. The objective
here is to simplify the Performance Test testing and come up with a comprehensive and
efficient testing strategy.

3.4 Intended Audience of the Strategy

The Intended audiences of the Performance Test Strategy are

 Business Users
 Development Team
 QA Team

Page 7
3.5 Glossary

SLA Service Level Agreement

OLA Operational Level Agreement
Throttling Number Number of concurrent instances can handle by each web
server
Test Script The sequence of functions/code that is generated in
Controller for each User Path which is used to emulate the
real user actions
BRD Business Requirement Document

4 Performance Testing Process

The following steps are involved in performance testing process

Page 8
4.1 Identify Key Scenarios

It is important to identify application scenarios that are critical for performance rather
than testing all the scenarios that are defined by the functional test team. Here the
scenarios refer to the web methods that need to be Performance tested. BRD and
interaction with Business Analyst will help to identify test cases.

< Mention all Key scenarios>

NOTE: - Need to identify more scenarios based on the future requirements

4.2 Identify Work Load

It is important to identify the work loads for the various scenarios mentioned above. In
the real world several Performance Test / web methods which will be accessed
simultaneously and the load distribution among them are to be identified and simulated
in the test process.

The BRD and Functional specifications will be utilized to identify the work load. Any SLA
or OLA available for each of the UI Scenario will also drive identifying the work load.

This helps to design the realistic performance test scenario for execution.

< Mention the necessary work load>

Note: - Need to get more performance baselines from client

4.3 Identify Metrics

Identify the metrics that need to be collected during the performance testing. This will
help us to come up with the appropriate reports to analyze the test results. The primary
metrics that needs to be identified are:

Throughput It is the sum of the data rates that are

delivered to all terminals in a network
Response Time We need to check the web services and
UI response time [Link] TMS, VSP,
<Project Name>
Resource utilization While doing the Performance testing ,
we need to check the memory and CPU
utilizations of the servers

Page 9
 Maximum concurrent and simultaneous users We have to check how many
concurrent user can sustain in the
environment
System Level metrics We need to monitor each individual
system resources like Memory, CPU,
Disk read/write etc
Network Level metrics We need to monitor the network
bandwidth [Link] our SLA’s

4.4 Create User path

Create comprehensive user path where the scenarios will be simulated. The test scripts
need to be generated for all the user paths and also need to make sure all variants of
Performance Test are tested for all the scenarios. The variants which will be tested are:

 UI Scenarios with different security settings (X.509 security / User name &
password)
 UI Scenarios /Web service with logging on/off
 UI Scenarios /Web service which has shadow web service calls
 Synchronous / Asynchronous Web service invocation

Note: - Need to create the user path

4.5 Simulate Load

Concurrent user load will be simulated in various patterns using Controller component.
Following Load Testing types will be conducted to simulate the <Project Name>
application load with the production load.

4.5.1 Performance Testing Types

Testing Type Purpose of the testing

Load Testing By doing the load testing we need to
verify <Project Name> UI, TMS
services, VSP server’s behavior under
normal and peak load conditions
Objectives:- To Check the response
times, throughput rates, resource
utilization levels,
Stress Testing By doing the stress testing we need to
evaluate our <Project Name> UI, TMS

Page 10
 services, VSP server’s behavior when
it is pushed beyond the normal or
peak load conditions.
Objectives: - synchronization issues
and memory leaks. Stress testing
enables you to identify your
application's weak points, and how it
behaves under extreme load
conditions
Web Service Testing Need to test the each individual web-
service and need to find out the
system behavior, when we apply the
load
Objective: - To check the response
time and the server utilization

Page 11
4.5.2 Performance Test Approach

4.5.3 Performance Test Activates and deliverables

Activities Responsible Teams

Set-up of test environment Infra Team has to provide the stage

environment to do the performance
testing.

Set-up of test data Performance team needs to

coordinate with manual and dev

Page 12
 teams to get the required test data.

Performance Test Execution Performance Team is responsible

for doing this testing.

Monitoring and collating results Performance and Infra team need to

monitor the servers. Performance
team is responsible for gathering
the results.

Obtaining user acceptance Need to get customer approval for

the performance testing results.
Deliverables Responsible Teams
Test Strategy Project Manager, QA Manager and
Performance Team
Test Plan QA Manager and Performance
TEAM
Test Scripts Performance TEAM
Test Executions Performance TEAM
Results Performance TEAM, Infra Team

Page 13
4.6 Analyze the Results

Analyze the metric data captured during the test against the predefined expectations. If
the expectations are not met, necessary modifications are made and testing is repeated
until the desired results are achieved.

The project is in early stage of development and changes in requirements are expected
from the business as well as from within the team. Changes in requirements will force
changes in key scenarios and the work loads. Hence the iterative nature of the testing as
depicted in the above graphic.

Every test run fine tunes the testing web methods used. As additional metrics are
identified there will be additional test cases created.

5 Test Environments and Tools

5.1 Test Environment

 Database: We need to specify the database name and versions which we are going to
use in the stage environment.
 Web Servers: We need to specify how many web servers and what type of software and
hardware, we are going to use in the stage environment.
 App Servers: We need to specify how many app servers and what type of software and
hardware, we are going to use in the stage environment.

Server Type Host Name Description

Web Server
App Server
DB Server

Note: - Need to get these details from the infra Team regarding the servers hardware and
software details

5.2 Testing Tools

The following test tools will be used to achieve the objectives mentioned

Tools Purpose
NeoLoad For creating the Scripts, Test Execution, Analyzing the

Page 14
 Results
Perfmon Window based Monitoring Tool
Dynatrace We can configure monitor the entire environment in this
tool and it’s a agent less and URL based

NOTE: - Need to finalize about the testing tools

6 Schedule

6.1 High-level test schedule

Test Activity / Start Date End Date Duration

Milestone
Load Testing
Stress Testing
Web Service Testing
Capacity Testing

Note: Need to get the testing start and end date

6.2 Schedule dependencies

Document any external dependencies the milestone schedule relies upon,

 Other projects’ delivery schedule

 Previous systems’ release completion
 Third party solution delivery
 Any DB restore’s

7 Assumptions, dependencies

7.1 Assumptions

 The test organization should have completed system testing successfully and all high
priority errors should have been addressed.
 Email Notification is a static content
 Projects and Targets deletes will be hard deleted from <Project Name> UI database
 Data displayed in <Project Name> UI will be fetched from <Project Name> database for
all read operation

Page 15
7.2 Dependencies

 Components to be performance tested shall be completely functional.

 Components to be performance tested shall be housed in hardware/firmware
components that are representative or scalable to the intended production systems.
 Data repositories shall be representative or scalable to the intended production
systems.
 Performance objectives shall be agreed upon, including working assumptions and
testing scenarios.
 Performance testing tools and supporting technologies shall be installed and fully
licensed."

7.3 Out Of Scope :-

Below scenarios are out of scope for performance testing in phase -1 and it may covered
in phase -2

Scenarios Purpose

8 Upfront Issues and risks

8.1 Upfront Issues

The below issues may occur while doing the performance testing:
 Contention (data, file, memory, processor)
 Inappropriate distribution of workload across available resources
 Inappropriate locking strategy
 Inefficiencies in the application design
 Unexpected increase in transaction rate
 Inefficient use of memory

8.2 Risks

Risk Priority Avoidance

Servers not available early HIGH We need to use dev.
during the project environment to develop
single-user scripts
Difficulty with Controller MEDIUM Identify issues early by
licensing, capacity, etc. beginning to use

Page 16
 controller as soon as the
first small script (such as
login only) is coded
Not enough capacity in front- MEDIUM Quantify capacity of
end (portal/login) servers. front-end servers with
login only scripts.
Servers become unavailable LOW Use production staging
late during the project environment at night.
Instead of going through
load balancer, test
directly against one
server taken off its
cluster
Changes in server hardware LOW Conduct benchmark
tests on hardware as
part of project.
Save server
configuration files for
comparisons

Page 17