Software Testing

What is testing?

Software testing is process to identify correctness, completeness and quality of development

software.

SQA (Software Quality Assurance)

Why testing is required?

Testing is a quality of software should be assured when software quality maintenance for good
application

To monitor & measure the strength of development process organization follows SQA concept.

Technical:

1) Meet customer requirement ( On what purpose)

2) Meet customer expectation ( Privacy and Performance)

Non-Technical:

3) Time to market ( Timely Deliverable)

4) Cost of Software ( About Money)

5) Risk Management ( If application crack while working risk management team should be
there)

Software Development Life Cycle (SDLC):

What is SDLC process?

To develop software we come across two stages:

SDLC

Life Cycle Development Life Cycle Testing

LCD= Upper Angle

LCT= Lower Angle

Developer : Tester = 3:1

Different Stages of SDLC process:

Information Gathering

Analysis

Design

Coding

Testing

Maintenance

There are three generic phases:

Definition: Customer requirement & analysis (Information Gathering & Analysis)

Development: (Design, Coding, Testing)

Maintenance: (Re-engineering, Adaptability, Correction)

What is difference between verification & validation?

Verification defines static testing. It starts from BRS, Analysis and analysis will be reviewed,
design will be reviewed and coding will be reviewed (WBT).

A validation means dynamic testing. In that only maintenance will be reviewed.

Verification is also known as “In Process Testing”.

Validation is also known as “End Process Testing”.

Validation starts after the completion of verification.

BRS (Business Requirement Specification):

Business people act as a bridge between customer & technical people. This document defines
customer requirement to be developed as software. This document developed by Business
Analyst (BA people).

SRS (Software Requirement Specification):

This document defines with respect to BRS. This document is also known as FS (Functional
Specification). This document defines functional requirement to be developed & system
requirement to be used. This document is also developed by Business Analyst category people.

Review:

It is a static testing technique used to check completeness & correctness of a document.

HLD (High Level Design):

It is also known as External Design. This document defines structure/ hierarchy of all possible
functionality to be developed as main modules. This is developed by project architect & software
designer.

LLD (Low Level Design):

It is also known as Internal Design. This document defines static logic of every sub-module.

E.g. Entity Relationship (ER) Diagram, Class Diagram, DFD (Data Flow Diagram).

Prototype:

A sample module of an application without any functionality is called prototype.

E.g. PowerPoint slideshow.

White Box Testing:

It is a coding level testing techniques used to check completeness & correctness of the program.
It is done by development team.

There are three possible WBT testing techniques:

1. Execution Testing Technique:

Basic Path Coverage: Execution of all possible block (Debugging).

Loop Coverage: Termination of loop.

Program Technique Coverage: Take less no. of memory & CPU cycle.
2. Operation Testing:

Runs on customer expected platforms, customer expected platform means operating system,
compiler browser &other systems.

3. Mutation Testing:

Mutation means change. Developer perform the change to estimate test coverage in the program.

Black Box Testing:

It is a build level testing technique. During this testing we people I mean to say Test Eng.
Validates internal functionality depends on external interface.

Integration Testing:

After completion of coding and their review developer combine all independent module to form
as system, during integration they apply integration testing on that coupled module with respect
to HLD & LLD.

There are three approaches conducts the integration testing such as:

a. Top Down Approach: In this approach test engineer conduct testing on main module without
going to sub-module, using STUB.

STUB is a temporary program used to check main module instead of under constructive sub-
module. This is also known as ‘Called Program’.

b. Bottom-Up Approach: In this approach test engineer conducts testing on the main module
instead of coming from the main module, using DRIVER.

DRIVER is a temporary program which is used instead of main module. This is also known as
‘Calling Program’.

c. Sandwich/Bi-Directional/Hybrid Approach: Combination of top down and bottom up

approach is called hybrid approach.

“After completion of Integration Testing Build is created”.

Who does the integration testing?

Developer and testing people both does the integration testing.

A development person writes Database Connectivity.

Testing people w.r. to combination of sub-module & w.r. to functionality.

System & Functional Testing:

After completion of system integration testing & review we people concentrate on the system
and functional testing through a set of BBT techniques to validates the functionality with respect
to customer requirement.

This testing technique classified into four categories:

1) Usability Testing: During this testing we people validates user friendliness of screen or build
or GUI.

2) Functional Testing: During this testing we people validates completeness and correctness of
the functionality with respect to customer requirement.

3) Security Testing: During this testing we people validates privacy to user operations.

4) Performance Testing: During this testing we people check speed of application.

Usability Testing: In general testing process starts with or we people start test execution with
usability testing. Usability testing defines user friendliness of screen or build or GUI.

It is classified in to two types:

. User Interface/GUI Testing: Less no. of event to complete task or easy validation.

E.g. To send email simple click on reply.

Ease of use (Understandable Screen).

Look & Feel (Pleasantness & Attractiveness).

Speed of interface (Less no. of event to complete task or easy validation).

. Manual Support Testing: It is done after system & functionality testing, Context sensitiveness
of user manuals.

E.g. Pune, Puna.

Functional Testing:

Q. What do you have done in your project?

Sir I have done functional testing to define customer requirement in terms of (BIEBSC) coverage
functionality testing. It is major part of BBT technique. During this testing we people validates
completeness and correctness of the functionality with respect to customer requirement.
There are two types of Functional Testing:

.Functionality Testing: During this testing we people validates completeness and correctness of
the functionality with respect to customer requirement.

This is also known as Requirement Testing.

There are several coverage’s comes under the Functionality Testing.

Behavior Coverage: During this testing we people check the property of object.

[Enabled/Disabled or ON/OFF property].

Input Domain Testing: During this testing we people check the size & type of input objects.
We are going to apply this testing.

Error Handling Coverage: Preventing negative navigation. [Testing valid as well as invalid
data].

Back End Coverage: (Database Testing) Content of backend operation, impact of content
operation.

Calculation Based Coverage: Checking output values w.r. to arithmetic calculation.

Service Level Coverage: Order of functionality, every functionality should be in proper order.

.Input Domain Testing: This is a part of functionality testing to measure the size & type of
input object. We people maintains a special structure in terms of BVA & ECP.

BVA defines Boundary Value Analysis. The size of input object. [Min & Max].

ECP defines Equivalence Class Partition. The size of data type. [Valid & Invalid].

.Non-Functionality Testing: During this testing we people check all non functional related
issues I mean to say functionality would be developed and impact of system requirement.

There are several types in .Non-Functionality Testing

Recovery Testing: It is also known as Reliable testing. During this testing we people check or
validates whether an application recover from abnormal situation to normal situation.

E.g. Normal Situation- Be-sent Require. Working fine

Abnormal Situation- Stick to be execute, system crash/server down/power fail.

Q. Are you involved in recovery testing?

Yes.
Compatibility Testing: This is also known as portability testing. During this testing we people
validates whether an application support/runs on customer expected platform or not.
Basically we people involve into browser compatibility testing.
Customer expected platforms, customer expected platform means operating system, compiler
browser &other systems.
There are two types:

Forward Compatibility: Build is ok but problem with operating system.

Back-word Compatibility: Operating system is ok but problem with build.

In general test engineers find back-word compatibility defects as maximum (Defect in build).

Q. What is browser compatibility testing?

Browser compatibility testing defines to ensure that our application runs on different types of
browser. Basically in this testing we people validates some scenarios like:

GUI of the application.

Functionality of the application (Link work properly or not).
Accessibility of application(application accessible properly or not).

In general we people map the Internet Explorer browser with other existing browser in the
market.
E.g. Safari, chrome, Mozilla fireworks, opera.

Q. Are you involved in compatibility testing?

Basically I am involved in browser & O.S. compatibility testing.
Two browser are in the market I.E. & Chrome(Using [Link]), mapping or comparing
between I.E. & chrome.

Configuration Testing/ Hardware: It is also known as hardware compatibility testing.

During this testing we people validates whether an application supports different types of
hardware devices or not.
E.g. Different types of printers, LAN, WAN, Topologies, MS word.

Intersystem/End to End/Verification & Validation Testing/ Internal system testing: It is

also known as End to End testing. During this testing we people validates, Co-existence of our
application or software with other existence software to share the resources.
Sometimes our application is interconnected with other application.

E.g. One bank ATM card (SBI ATM card) is accepted by other bank ATM to withdraw money,
during this time we people validates the functionality w.r. to interconnection to other software.

It is also known as Verification & Validation.

Basically the End to End testing never involve in system & functional testing.
Note: Generally End to End testing is being performed after the completion of functionality
testing of each & every module.

Installation Testing: During this testing we people validates installation of our application along
with co-existed software in the customer expected like configuration (environment) to validate
the functionality w.r. to customer requirement.
We people concentrate on some factors such as:

[Link] program execution before installation. (Whether all setup files are available or not).
2. Easy interface during installation.(Default Radio buttons should be there while installation).
3. Occupied disk space after installation.
4. Verify un-installation.

Parallel/Comparative Testing: It is applicable only for software product. During this we people
compare the product with other comparative product in the market. It is done after functional
testing.
Q. What is difference between product & application?
Product made for multiple customers.
Application made for specific customers.

Sanitation/Garbage Testing: During this testing we people tries to remove extra functionality
in the application w.r. to customer requirement.
When customer faces a problem they can request for modification for which he has to pay some
amount and we make business from that.

Globalization Testing: It is the testing to ensure that our application supports multiple
languages or not.

Localization Testing: To ensure that our application supports local language or not.
Nationalization Testing: To ensure that our application supports Nationalization languages or
not.
Internationalization Testing: To ensure that our application supports Internationalization
languages or not.

Q. Do you involve in globalization testing?

Actually what happen previous release I got chance to perform an globalization testing. In that
our application is develop for a north Canada user which is basically speaks a French language in
this scenario I was check our application supports French language or not.
In that application I remember one scenario that is whenever we click on French
language date format American MM/DD/YYYY has been converted into French i.e.
DD/MM/YYYY. Secondly when we select that country code country currency will be displayed.
Performance Testing: (Speed of processing): This is advance testing technique & complex to
conduct because testing team require a huge environment to conduct this testing.
E.g. Automation tools, basically performance testing is done with the help of automation tools.

Performance testing is classified into four types such as:

1. Load Testing
2. Stress Testing
3. Storage Testing
4. Data Volume Testing

[Link] Testing: Execution of our application under customer expected configuration

(environment) & customer expected load to estimate performance is called Load Testing. It is
also known as Scalability Testing.
E.g. In mechanical the load on steel chip if load exceed 500 tan, it will break.

[Link] Testing (Stress defines maximum of load): The execution of our under customer
expected configuration and an internal load & peak load to estimate performance is called Stress
Testing. It is ensure that the maximum load on the application handle.
E.g. Maximum 700 users can use the application at a time.

[Link] Testing (It defines maximum of data store): The execution of our application under
huge amount of resources to estimate storage limitation in our applications called Storage
Testing.
E.g. No of volume in byte, Mobile card handle only 2GB data as per card limit.

[Link] Volume Testing: The execution of our application under customer expected
configuration to estimate the peak limits of data is called Data Volume Testing.
E.g. No. of records in your database can be stored.

Security Testing: It is also known as advance level testing technique & complex to conduct.
During this technique we people validates the privacy to user operation as per customer
requirement.

There are three types of security testing.

[Link]
[Link] Control (Authentication)
[Link] & Decryption

[Link]: It is done by Test engineer, during this test we people validates whether the
user is valid or not.
E.g. Employee of wipro.

[Link] Control (Authentication): It is done by Test engineer, during this test we people
validates whether the user has permission for specific operation or not.
E.g. Employee of wipro but need to permission for specific operation.
3. Encryption & Decryption: Data conversion between client and server. It is done by
developer.

Encryption

Client Server

Decryption

User/Customer Acceptance Testing: (UAT/ CAT)

After completion of an system & functional testing our organization concentrate on user
acceptance testing to collect the feedback from customer.

UAT is classified into two types:

Alfa Testing: This testing is applicable for a software application. Alfa testing is a testing
technique which is done in controlled environment in the presence of developer & tester.

Beta Testing: This testing is applicable for a software product. Beta testing is done at the
customer side in the absence of developer & tester in uncontrolled environment.

Q. Have you involve in beta testing?

Basically user manual is an main part of an user acceptance testing and we involve in
some manner.

Final Regression/ Port/ Pre-Acceptance/Pre-Release/ Sanity of Release Testing:

After completion of an User Acceptance Testing & their review our organization
concentrate on released, from team formation which is known as build released team. This
release team consists of some Hardware engineer, some Developer & some Test engineer.
Basically this release team apply port testing on critical part of application to validate the
functionality before release the build. In general released testing will be performed within 2 days
and it will be performed at client side like environment (configuration).
During this testing they concentrate on some factors such as:

1 Compact installation (Installation Testing)

2 Overall functionality (System & functional Testing)

3 Input/Output device handling.

4 Secondary storage device handling.

5 Co-existence with other existing software (End to End Testing)

Q. What is change request (CR)?

After completion of port testing, release team providing training session to customer side
people and come back to our organization. During maintenance customer side people request our
organization for change or modification that is known as change request.

During change request we focus on some factors are:

# Impact analysis.

# Performance change.

# Test change process.

Change Control Board (CCB):

Analyze impact at change senior team will handle change request during an test
execution. Change Control Board comes under configuration management. Configuration
management means to handle the change request during test execution.

Testing Terminology:

It is comes under the System & Functional testing.

[Link]/Chimpanzee/Speed Testing:

Basically monkey testing defines maximum number of test cases with less number of time
for execution. During monkey testing we people concentrate on high priority test cases.
[Link] Testing:

It is done by senior tester. Test engineer do not have sufficient test data but with the help of
past experience have to conduct the test.

I mean to say we don’t have test data but we have domain knowledge.

[Link] Testing/ Orthodox Testing:

Level by level functionality coverage is called as exploratory testing. I mean to say we have
test data but don’t have domain knowledge.

[Link] Testing/Octangle Testing:

It is a initial stage of Black Box Testing. After receiving initial build from development team,
we people I mean to say test engineer estimate stability of build to cover the core functionality or
to check whether build is ready before testing or not is called as sanity testing.

We conduct a sanity testing after each build up-gradation that is after receiving new build. In
sanity testing we perform:

#All link validation

#All tab validation
#Core functionality of the application.
During sanity testing if we find any mismatch or defect then it is known as environmental issue, I
mean to say
#Run time error
#Link is not get access
#Tab not working
#Page disappears.

Smoke Testing:
Smoke testing is a extra shake(vibration) up of sanity testing. We trying to identify a
troubleshoot. During this execution if we find environmental issue/runtime error.
We are trying to identify invalid object then we are trying to identify package that means
the object belong to which package then we are going to request to database administrative
people to recompile that package that’s all about smoke testing.
Package is combination & clubbing of similar type of object & it is created by database
administrator.
Big Bag Testing/ Informal Testing:
During this test, testing team concentrate on single stage instead of multiple stages after
completion of entire system development process. It is also called as an random testing.

Incremental/ Informal Testing:

During this test, testing team concentrate on unit level to system level. During this testing
they involve multiple level testing. It is also called as Formal testing.

UAT

System & Functional Testing

Integration

Unit

Retesting:

Retesting of an same application or build with multiple test data to validate the
functionality is called as retesting.

E.g. To define multiplication table test engineer choose the different types of combination in
terms of +ve, -ve, integer, float, min, max and zero that is same build with multiple test data.

Regression Testing:

Regression comes word from regret. Re-Execution of test on modify build to ensure that
bug fix and occurrence of side effect to validate functionality is known as regression testing.

Regression Suite we performed.

#Previously failed test case

#Newly added scenario.

#High priority test case.

#If time permit independent test case.

 Testing Report the Defect Accept

Retest Modify Build Fix Defect

Modify Build

Regression
Test

Agile Methodology/ Model:

Product Owner Product Back-log Sprint Back-log Stories

Test Case Design

Test Engineer

Test Case Execution

Agile defines the characterized quickness. It is a simple philosophy, It’s not plan driven
it is a value driven. So we are going to follow some procedure I mean to say the product owner
collect the list the requirement, this is nothing but the product back-log & from this product
back-log we are going to select the list of requirement & move to corresponding sprint this is
nothing but sprint back-log. So during this time we are going to estimate that is what are the
requirement should be deliver in this sprint? and what are the requirement should be deliver in
the next sprint? & this estimation can be done by using several factors, I mean to say
Complexity, Efforts, Knowledge & this process would be done by the development lead along
with the testing lead, and each sprint back-log consist of stories and from that stories we are
going to prepare the test case.

There are several attributes such as:

#Check point after each model (Small Regression):

#Scrum Meeting: Scrum meeting is taken under control of scrum master, each day start with
this meeting and we discussed in meeting I mean to say what did yesterday? , What are going to
do today? And what is the road blocks (Issues). In scrum meeting involve people are BA, scrum
master, development team & testing team as well as all including all project members. Time span
of this meeting is 15 to 20 min but it would be extend depending upon issues.

#Sprint wise delivery: In agile we give 1 month delivery that is

1R (Release) = 1M (Month) = 1S (Sprint)

#Automation: Automation team will handle this task.

Advantages of Agile:

#Major advantages of agile methodology is Frequently changes in requirement doesn’t impact on

production process or delivery process.

# Less Cost.
# Less Resource Utilization.
# Fast Delivery.
Disadvantages of Agile: When project is complex/ big and it has lot of independent module in
that case it get difficult to implement.

Sprint Test Plan of Agile:

28 Dec = Estimation.

#1 Jan to 5 Jan (1st Week)

Requirements Analysis.
Iteration test case design.
Iteration test case review.

#8 Jan to 12 Jan (2nd Week)

Regression test case design.

Regression test case review.

#15 Jan to 20 Jan (3rd Week)

Iteration test case execution.

Regression test case execution.
Defect Analysis.
Defect Log.
Defect Fix.
#23 Jan to 28 Jan (4th Week)

UAT port testing.

Test planning for next sprint.
Test closer activity.

That’s all about the agile methodology.

STLC: Software Testing Life Cycle

Test Initiation Test Plan Test Design Test Execution Test Closer

Defect Report

rt
In many organization testing process starts with initiation process. During this stage
process manager concentrate on the scope of the project, requirement of the project & the risk
involved in the project.

After that in the test plan Test Lead mainly concentrate on job allocation in the terms
of
#What to test?
#How to test?
#When to Test?
#Who will test?

During test design we people I mean to say test engineer prepare the test case From
SRS/Functional specification to validate the functionality with respect to customer requirement.

After completion of test design we people execute the test case to validate the functionality.
During this execution if we find any mismatch or defect then we log the defect and send it to
development team to fix that defect.

After fixing that bug developer send the modified build and on that modified build we people
apply the regression testing to check whether the defect is resolve or not and to check the
occurrence of any side effect due to bug fix. This is the cyclic process and we will continue till
all defect get resolve.
After completion of test execution we prepare that report & send it to test lead.
During test closer test lead concentrate on whether all the testing process going correctly or not.
That’s all about STLC.

Defect Life Cycle:

Reject
Close
New Open Fix

Re-Open
Differed

During system and functional testing we people get new defect & its status is new and log the
defect in defect tracking tool.

After discussion with the senior tester we people set the defect status as open & assign it to
developer.

After receiving the defect from the testing team, developer analyze & verify the defect. If
developer doesn’t accept it, he set defect status as reject.

If developer decides to fix the defect in later version he set defect status as differed.

If developer accept the defect & he fix it, then he send the modified build to testing team.

Then we people I mean to say test engineer verify whether the defect has been fix or not. If
defect has been fix we people set the defect status as closed.

If defect is re-occurring any side effect due to bug fix, then we people set the defect status as
reopen and send it to developer.

That’s all about the defect life cycle.

 || Depth Part ||

Test Documentation Hierarchy Process

Quality Control (QC) Test Policy

Company Level Document (HLD)

Test Strategy
QA/PM Quality Analyst

Test Methodology

Team Lead (TL) Test Plan

Test Case

Project Level Document

Test Procedure

Test Engineer (TE)

Test Script

Defect Report

Test Summery Or
Test Summery Report send to Client.
Team Lead (TL)
Final Report Or

Software Release
Note (SRN)
Test Policy
It is a company level document and developed by quality control people. (Almost management) this
document defines “testing objective” to be achieved.

Name of Company

Address of company

Location of company

Testing Definition: Verification + Validation.

Testing Process: Proper planning before starts testing.

Testing Standard: 1 defect per 250 loc/1defect per 10fp.

Testing Measurements: QAM, TMM, PCM.

Signature of

(C.E.O)

LOC – Lines of code.

FP – Functional point.

Example: No of screens / No of forms / No. of reports / No. of inputs / No. of outputs / No. of
queries.

Using functional point we can know the size of project.

QAM – Quality Assessment Measurements.

TMM – Test Management Measurements.

PCM – Process Capability Measurements.

Basically this document shown to the client side people in according to gain new project.

Test Strategy

It is also a company level document and developed by quality analyst people (project manager level).
This strategy document defines testing approach to be followed by testing team.

1) Scope and objective: The definition & purpose of testing in our organization.
2) Business issues: Budget control for testing, cost of testing is estimated accordingly cost of a
project. (How much we are going to spend on testing).

3) Testing Approach: Mapping between development stages and testing issues (Factors) TRM is
prepared.

Ex: v-model, Test Responsibility Matrix (TRM)/ Test Matrix (TM).

Info.
Dev. Stages/Testing
Gathering & Design Coding Testing Maintenance
factors & Issues
Analysis
1. Authorization No No Yes Yes
2. Access Control No Yes Yes Yes
Depend on
3. Audit Trial Yes Yes Yes No
change
………
request.
……….
15. Methodology

Note: TRM is prepared by PM & send it to customer accordingly money will be given.

4) Test deliverables: Required testing task to be completed before start testing w.r. to H/W
configuration & resource document. Entry & Exit criteria should be define properly.

5) Roles and responsibilities: Names of jobs in testing team and responsibilities of every job during
testing.

6) Communication and status reporting: Required negotiation between every two consecutive jobs
in testing team.

7) Test automation and testing tool: Purpose of automation and availability of testing tools in your
organization.

8) Defect reporting and tracking: Required negotiation between testing team and development
team when testers got mismatches in testing. Because it may affect on production or releases.

9) Testing measurements and metrics: This is the unit to measure testing process. QAM, TMM,
PCM

10) Risks and Mitigations: If any problem occurs during testing then the solutions to overcome.

11) Change and configuration management: Ability to handle change request during execution.
How to handle sudden changes in customer requirements during testing.

12) Training plan: Required number of sessions to understand customer requirements by testing
team.

TEST FACTORS OR TESTING ISSUES

To define one quality software. Software engineering people are using 15 factors/issues.
1) Authorization: Whether a user is valid or not? To connect to application.

2) Access control: Whether a valid user have permissions to use specific service or not?

3) Audit trail: Maintains Meta data about user operations.

4) Continuity of processing: Inter-process communication.

5) Correctness: Meet customer requirements in terms of functionality.

6) Coupling: Co-existence with other software applications to share common resources.

7) Ease of use: User friendliness of screens.

8) Ease of operate: Installation, un-installation, dumping, down loading, up loading etc.

9) File integrity: Creation of back up during operations.

10) Reliability: Recover from abnormal situation to normal situation.

11) Portable: Run on different platforms.

12) Performance: Speed of processing.

13) Service levels: Order of functionalities.

14) Maintainable: Whether our application build is long time serviceable to customer site people or
not?

15) Methodology: Whether our testing team is following standards or not? (during testing)

Test Factors Vs Testing Techniques

1) Authorization:

Security testing (separate testing team)

Functionality/ requirement testing (common testing team)

2) Access control:

Security testing (separate testing team)

Functionality/ requirement testing (common testing team)

3) Audit trail:

Functionality/ requirement testing

4) Continuity of processing:

Integration testing (top down/ bottom up/ hybrid)

5) Correctness: Functionality testing

6) Coupling: Intersystem testing

7) Ease of use: User interface testing , manual support testing

8) Ease of operate: Installation testing

9) File integrity: Functionality/ requirement testing/Recovery testing

10) Reliability: Recovery testing (1 user level)

11) Portable: Compatibility testing, Configuration testing

12) Performance: Load testing, stress testing, storage testing, data volume testing.

13) Service levels: Functionality/ requirements testing (1 user level)

14) Maintainable: Compliance testing

15) Methodology: compliance testing

Compliance testing: Whether the testing team is following standards or not during testing, is called
compliance testing. Compliance means that complete plan.

Test Methodology

It is a project level document. This document defines required testing approach for corresponding
project testing. Project manager like people are developing test methodology depending on company
level test strategy. Due to this reason, test methodology is also known as a refinement form of test
strategy.

To develop test methodology, project manager/quality analyst follows below approach. Before start
every project testing.

Step 1:- Collect test strategy.

Step 2:- Identify current project type.

 Information
Gathering & System
Project Type Design Coding Maintenance
Testing
Analysis
Traditional √ √ √ √ √

Outsourcing * * √ √ *

Maintenance * * * * √

Note:- Depending on project type, project manager delete some of the columns from TRM (test
responsibility matrix) for this project testing.

Step 3:- Study project requirements.

Note: – Depending on requirements in the project, PM delete unwanted factors (rows) from TRM for
this project testing.

Step 4: – Determine the scope of project requirements.

Note: – Depending on expected future enhancements, PM is adding some of previously deleted factors
to TRM for this project testing.

Step 5: – Identify tactical risks.

Note: – Depending on analyzed risks, PM is deleting some of the factors from selected TRM for this
project testing.

CASE STUDY: 15  Test factors

-3  Requirements
——
12
+1  Scope of requirements
——
13
-4  Risks
——
9  Finalized to be applied on project

Step 6 :- Finalized TRM for current project testing.

Step 7 :- Prepare system test plan.

Step 8 :- Prepare module test plan.

Test Plan

After completion of test initiation and testing process finalization, test lead category people are
concentrating on test plan document preparation in terms of “what to test?”, “how to test?” , “when to
test?” , “who to test?” .
 Input Process Output

1. Team Formation
Development Strategy (Resource Allocation)

2. Identify Tactical Risk

Finalize TRM form (Risk Analysis)
System Test Plan
Test Methodology 3. Prepare Test Plan

4. Review Test Plan

Testing team formation: In general test planning process starts with testing team formation. In this
stage test lead is depending on below factors.

 Availability of test engineers

 Test duration the 3 factors are dependent
 Availability of test environment resources

Case study:

Test Duration: C/S, Web, ERP  3 to 5 months of system testing

System s/w  7 to 9 months of system testing
Machine critical  12 to 15 months of system testing

Team Size: Developers : Testers = 3 : 1

Identify Tactical Risks: After formation of testing team, test lead is analyzing selected team level
risks. This risk analysis is also known as Root Cause Analysis.

Ex: Risk 1 : Lack of knowledge of testing team on that domain.

Risk 2 : Lack of budget (time & cost ).

Risk 3 : Lack of resources (testing tools not available)

Risk 4 : Lack of test data (improper documents)

Risk 5 : Delays in delivery (Health problem)

Risk 6 : Lack of development process rigor.

Risk 7 : Lack of communication (in b/w testing team to development team)

Prepare test plan: after completion of testing team formation and risks analysis, test lead
concentrate on test plan document preparation in IEEE format (Institute of Electrical and Electronics
Engineers).
IEEE Format:

Test plan ID: Unique number / name.

Introduction: About project

Test items: Names of all modules in that project ex: website.

Features to be tested: New module names for test design. (What to test)

Features not to be tested: Which ones and why not? (Copy test cases from server)

Approach: Selected list of testing techniques by project manager to be applied on above

modules,(finalized TRM)

Testing tasks: Necessary operations to do before start every module testing.

Suspension criteria: Possible raised problems during above modules testing.

Ex: exception handling.

Feature pass/fail criteria: When a module is pass and when a module is fail.

Test environment: Required hardware’s and software’s to conduct testing on above modulus. Ex:
WinRunner

Test deliverables: Names of testing documents to be prepared during above modulus

EX: test cases, test procedures, test scripts, test log, defect reports for every modules.

Staff and training needs: Names of selected test engineers for this project testing

Responsibilities: Mapping between names of test engineers and names of modules.

(Work allocation)

Schedule: dates and times.

Risks and mitigations: Raised problems during testing and solutions to overcome.

Approvers: signatures of project manager and test lead.

Review test plan

After completion of first copy of test plan document development, test lead conducts a review on that
document for completeness and correctness. In this review meeting test lead concentrate on coverage
analysis.
Coverage analysis:

 Business requirement based coverage (what to test?)

 TRM based coverage (how to test?)

 Risks based coverage (when & who to test?)

After finalizations of test plan, test lead is providing some training sessions to selected testing team on
project requirements.

Test Case Design & Methods:

After finalization of test plan and after completion of training sessions, test engineers are
concentrating on test cases development for responsible modules. There are three methods to prepare
test cases such as:

Business logic based test case design (depending on SRS or FS for Application)

Input domain based test case design (design documents for Product)

User interface based test case design (Application & Product)

Business logic based test case design

In general, test engineers are preparing maximum test cases depending on use cases in SRS. Every
use case is describing functionality in terms of inputs, process and outputs.

Depending on that use case. Every use case is also known as functional specification. Every test case
describes a testable condition to be applied on build.

To study use cases, test engineers are following below approach.

Step1: Collect required use cases for responsible modules.

Step2: Selecting a use case and their dependencies from above collected list of use case.

Step2.1: Identify entry condition (base state)

Step2.2: Identify input required (test data)

Step2.3: Identify output and out come (expected)

Step2.4: Study normal flow (navigation)

Step2.5: Study end condition (end state)

Step2.6: Study alternative flow and exceptions

Step3: Prepare test cases depending on above study of use cases

Step4: Review the test cases for completeness and correctness

Step5: Go to step2 until all use cases study completion

Test case format

During test design test engineers are preparing test cases in IEEE format. Through these formats test
engineers are documenting every test case.

Format of Test Case:

Test case id: Unique number/name

Test case name: The name of test conditions.

Feature to be tested: Corresponding module or function name.

Test suit id: The corresponding batch id, in that batch this case is also member.

Priority: The importance of test case in terms of functionality.

EX: P0—Basic functionality (requirements)

P1—General functionality (recovery, Compatability, inter systems, load—)

P2—Cosmetic functionality (user inter face)

Test environment: Required hard wares and soft wares including testing tool to execute this test
case.

Test efforts: (person/hour) Time to execute this test case .

EX: 20 min average time

Test duration: Date and time

Test setup: Necessary tasks to do before start this case execution

Test procedure: This step-by-step procedure from base state to end state

Test case pass/fail criteria: When this case is pass/ when this case is fail

NOTE: In general, test engineers are not maintaining complete format for every test case. They
can try to maintain test procedure as mandatory for every test case.

Input domain based test case design

In general, test engineers are preparing test cases depending on use cases or functional specifications
in SRS. Sometimes they can go to depending on design documents also. Because, use cases are not
providing complete information about size and type of input objects. Due to this reason, test engineers
are studying data models in design documents.

EX: ER-diagrams (entity relationship diagrams)

In this study, test engineers are following below approach

Step1: Collect data models of responsible modules from design documents

Ex: ER-diagrams

Step2: Study every input attribute in terms of size and type with constraints

Step3: Prepare BVA and ECP for every input attribute in below format

I/P Attribute ECP BVA (Size/Range)

Valid Invalid Min Max

This table is called DATA MATRIX. This table is providing information about every object

Step4: Identify critical and non-critical inputs in above list

Critical inputs are involving in internal manipulations. Non-critical inputs used for printing purpose.

NOTE: If our test case is covering an operation, then test engineers are preparing step-by-step
procedure from base state to end state. If our test case is covering an object, then test engineers are
preparing data matrix.

User Interface Based Test Case Design

To conduct usability testing, test engineers are preparing test cases depending on global user inter
face convection, our organization rules and interest of customer site people.

Example test cases:

Test case1: Spelling check

Test case2: Graphics check (alignment, font, style, color and other micro soft six rules)

Test case3: Meaning full error messages

Test case4: Accuracy of data display

Test case5: Accuracy of data in the database as a result of user input

Test case6: Accuracy of data in the database as a result of external factors

Ex: file attachment, export files, import files etc

Test case7: Meaning full help messages

NOTE: test case1 to test case6 are indicating user inter face testing and test case7 is indicating
manuals support testing.
Q. What is Microsoft Six Rules?

1) Controls are init Cap

2) Controls should not be overlapped.

3) Controls should be visible

4) Controls should be alignment

5) Ok & Cancel buttons existence.

6) System menu existence.

Test Case Design Review

Before receiving build from development team to start test execution, test lead is analyzing the
completeness and correctness of prepared test cases by test engineers through a review meeting.

There are four types of Review:

1) Self Review.
2) Peer Review (Along with colleague).
3) Internal Review (PM/TL/BA/TE)
4) External Review (Customer)

In this review test lead is analysis the coverage’s.

—Business requirement based coverage

—Use cases based coverage
—Data model based coverage
—User inter face based coverage
—Test responsibility matrix based coverage

At the end of this review, test lead is preparing requirements trace ability matrix (RTM). This matrix
defines mapping between customer requirements and prepare test cases. This matrix is also known as
requirements validation matrix (RVM).

Traceability Matrix: (RVM/RTM)

Traceability Matrix define mapping between business requirement & prepared test cases to
validate the customer requirement. This matrix is prepared by the TL.

a) Forward Traceability Matrix: Mapping between prepared test case & business requirement
is called as Forward Traceability Matrix

b) Backward Traceability Matrix: It is a mapping between defects and prepared test cases
Requirement Validation Matrix: Sometimes defect is valid defect and written test case is also
correct but still defect is occurred that time we have add extra test case for that defect is called
as Requirement Validation Matrix.

Test Reporting Or Defect Tracking

During level-1 and level-2 test execution, test engineers are reporting mismatches to development
team in IEEE format.

Format:

Defect Id: Unique number and name.

Description: Summary of defect

Build Version Id: Version number of build, in which test engineers found this defect

Feature: The Corresponding module name, in which test engineer found this defect

Test Case Name: The name of test condition, during this case execution, test engineer found this
defect

Re-producible: Yes, means defect appears every time in test execution No, means defect appears
rarely in test execution

If yes, attach test procedure

If no, attach snap shot and strong reasons

Found By: The name of test engineer

Detected On: Date of submission

Assigned To: The responsible person at development side to receive this defect

Status: New – Re-reporting defect or Reopen – Reporting defect first time

Severity: The seriousness of defect in terms of functionality

High – Not able to continue test execution with out resolving that defect

Medium – Able to continue remaining testing but compulsory to solve

Low – Able to continue remaining testing but optional to resolve (may/may not)

Priority: The importance of this defect in terms of customer

Suggested Fix (Optional): Expected possibilities to resolve this defect by developers

Fixed By: Project manager/project lead

Resolved By: Programmer name

Resolved On: Date of resolving

Resolution Type:

Approved By: Signature of project manager

NOTE: In above format development people try to change priority of defect with respect to customer
importance

Defect age: The time gap between defect reported date and defect resolved date is called defect age

Defect Resolution Type: During test execution, test engineers are reporting mismatches to
development team as defects. After receiving defect reporting from testing team, development people
are conducting bug-fixing review and they will send resolution type report to corresponding testing
team. There are 12 types of resolutions to report to testing team.

Duplicate: Rejected due to this defect equal to previously reported defect.

Enhancement: rejected due to this defect related to future requirement of the customer

Hard ware limitation: rejected due to this defect related to limitations of hard ware devices

Soft ware limitation: rejected due to this defect related to limitations of soft ware technologies

Not applicable: rejected due to improper meaning of defect

Functions as designed: rejected due to coding is correct with respect to design logic

Need more information: not accepted and not rejected but developer’s required extra information to
under stand the defect

Not reproducible: not accepted and not rejected but developer’s required correct producer to
reproduce that defect

No plan to fix it: not accepted and not rejected but developer’s required extra time to fix

Fixed: accepted and ready to resolve

Fixed indirectly (deferred): accepted but postponed to future version

User miss-understanding: extra negotiation between developers and tester

Types Of Bugs: During test execution either in manual or in automation, test engineers are finding
below types of bugs.

Users Inter Face Bugs: (low severity)

Ex1: Spelling mistake (high priority)

Ex2: Improper right alignment (low priority)

Error Handling Bugs: (medium severity)

Ex1: Does not return error message (high priority)

Ex2: Complex meaning in error message (low priority)

Input Domain Bugs: (medium severity)

Ex1: Allows in valid inputs (high priority)

Ex2: Allows in valid type also (low priority)

Calculations Bugs: (high severity)

Ex1: Dependent out puts are wrong (application show stopper) (high priority)

Ex2: Find out put is wrong (module show stopper) (low priority)

Race Condition Bugs: (high severity)

Ex1: Deadlock or hang (application show stopper) (high priority)

Ex2: Improper order of functionalities (low priority)

Load Condition Bugs: (high severity)

Ex1: Does not allows multiple users (high priority)

Ex2: Does not allows customer expected load (low priority)

Hard-Ware Bugs: (high severity)

Ex1: Not able to establish connection to hard ware device (high priority)

Ex2: Wrong out put from device (low priority)

Version Control Bugs: (medium severity)

Ex1: Mis matches in between two consecutive build versions

ID-Control Bugs: (medium severity)

Ex: Wrong logo, logo missing, copy right window missing, wrong version number, soft ware title
mistake, team members names missing——etc.

Source Bugs: (medium severity)

Ex: Mistakes in help documents