Load Testing Guidelines
1.0
Version Date Prepared/Modified Reviewed by Changes Done
By
Table of Contents
1 PROJECT PLANNING.......................................................................................................................3
2 REQUIREMENT PHASE...................................................................................................................4
3 PLANNING PHASE............................................................................................................................5
3.1 DATA...............................................................................................................................................5
4 SCRIPTING..........................................................................................................................................6
5 TEST EXECUTION.............................................................................................................................7
6 REPORTING........................................................................................................................................9
<Project Id> Page 2 of 10 <SCI id>/Last updated on<>>
1 PROJECT PLANNING
o Initial discussions get an idea about the application
o If other than web or even in web something different like applets running in browser,
call backs etc. request for a POC before committing on the actual project execution
o If env in offshore Cognizant’s LR license can be used if in onsite either client should
already have a tool which they should share with us or client should procure a tool or
last option is go for a open source tool
o Over the internet testing is not recommended
<Project Id> Page 3 of 10 <SCI id>/Last updated on<>>
2 REQUIREMENT PHASE
o Involve all the stakeholders of the application during requirements gathering
o If tool in place see it suits all the applications to be tested
o Feasibility study, poc, pilot
o Good understanding of the application architecture.’
<Project Id> Page 4 of 10 <SCI id>/Last updated on<>>
3 PLANNING PHASE
o What is in scope and out of scope needs to be mentioned clearly
o Scope – external interface, third party product performance testing
o Support available during load testing
3.1 Data
o Backup and restore procedures
Each round (iteration) of testing will cause changes to the data available in the database as a
result of the updates and deletions that happen during test execution. In the next iteration of
testing, these changes might in the worst case, result in serious variations in the performance
test results. Because of these variations, the comparison of the results from the two tests
becomes difficult and hence it is always advisable to have the same data in the database
during each round of testing.
To do this, the data in the database is backed up prior to the test execution. Once the tests
are executed, the database is restored to the original state using scripts or the restore feature
commonly available in common databases. The next iteration of testing can then be done
and the results from both these tests can be compared.
When is it done: This practice is followed before every round of testing.
o Data for scripting
o Destructive data
If any of the scripts have destructive data and data setup is in scope, then identify if a front-
end is available for creating that data. If so, then create scripts that will create data for the
other script with the destructive data.
o Identify realistic samples of data mix
o Do not use small sample set which will be used repeatedly
Using a small sample set repeatedly, during test execution would not give realistic results.
o Data creation script from performance testing team should be the last option
o Errors due to data should NOT be present during test execution
o Database to mimic production
<Project Id> Page 5 of 10 <SCI id>/Last updated on<>>
<Project Id> Page 6 of 10 <SCI id>/Last updated on<>>
4 SCRIPTING
o Verify business flow before recording
o Get multiple sets of data for scripting
Before scripting starts, it has to be verified if multiple sets of data have been provided for
each use case. Multiple sets of data are mandatory to start the scripting phase because it
will aid in finding out what values need to be correlated.
o Parameterize & correlate
o Strictly follow the scripting standards established within our COE as it has evolved
over years
o Execute single user multiple iterations to validate the scripts
o Peer review with checklist
Script level issues may be missed leading to errors during smoke run and actual test
executions.
When is it done: It is done as soon as a script is completed by one member and he/she
thinks it is ready for delivery.
o Project specific standards to be verified
o Maintain folder structure
Most projects are
o Error handling.
o Versioning of scripts
If proper versioning of the scripts is not done, it may lead to confusions
<Project Id> Page 7 of 10 <SCI id>/Last updated on<>>
5 TEST EXECUTION
o Application environment
o Dedicated Load generation environment
The environment provided for test must be dedicated for load testing. In other words, no
other team should access the environment at the time of test execution. Other teams
accessing the application during the load test will lead to results that are not realistic.
o Validate all scripts
All the scripts that are to be included in the load test have to be validated in the Controller
prior to the test execution. Doing this eliminates the script level or environment level errors
during the test
o Verify data
It is always a good practice to validate all the data that is to be used in the actual load test.
Doing this eliminates the possibility of data being the cause for the errors in the test.
When it is done: It is done prior to the actual test execution
o Monitors in all layers
o Workload mix
The proper workload mix has to be identified during the requirement gathering phase by
having discussions with the business team and the development team. If workload analysis
is not done and the system is tested at conditions (load) different from the ones seen in
production, then the results that we see in our tests may not be a good indication of how the
application will perform in production.
When it has to be done: Workload Analysis has to be during the requirement gathering
phase itself.
o Ramp up and steady state
o Smoke test
o Results folder naming convention
o Restart all servers (if possible)
Prior to the test being executed, it is advisable to restart all the servers. This will shut down
any connections that are left hanging and thus give realistic results.
o Production log settings
<Project Id> Page 8 of 10 <SCI id>/Last updated on<>>
o If load high use load generators, monitor the health of load generators for heavy load
o Production configuration preset load env
o If errors during test run access application manually and check for errors
<Project Id> Page 9 of 10 <SCI id>/Last updated on<>>
6 REPORTING
o No extrapolation of results
o If test env is not equal to prod, specify the clearly in the report what was out of scope
If the details regarding the test environment are not mentioned in the test report then the
person reading the report may assume that the results
o Mention application build that was tested
<Project Id> Page 10 of 10 <SCI id>/Last updated on<>>