Software Testing Metrics: Types, Calculation, Examples

What Are Software Testing Metrics?

Software testing metrics help measure and monitor test activities. They provide critical insights into
testing activities, team productivity, and the system's quality under test, so teams can measure different
aspects depending on what they want to track, control, or improve.
Or
Software testing metrics are quantifiable indicators of the software testing process progress, quality,
productivity, and overall health. The purpose of software testing metrics is to increase the efficiency
and effectiveness of the software testing process while also assisting in making better decisions for
future testing by providing accurate data about the testing process. A metric expresses the degree to
which a system, system component, or process possesses a certain attribute in numerical terms. A
weekly mileage of an automobile compared to its ideal mileage specified by the manufacturer is an
excellent illustration of metrics.
In essence, by providing testing teams with critical testing information, metrics help in
• Unearthing challenges and bottlenecks
• Identifying and eliminating inefficient tasks
• Making changes to testing strategies and tools to improve the testing process.
Why Are Software Testing Metrics Important?
No matter how much effort you put into software testing, unless you can measure the progress, you
cannot know what you are doing or what you need to do for better results. Software testing metrics
convey a result or a prediction based on a combination of test data. For instance, they can provide
information on the success of the testing activity, the time taken to run a set of test cases or even deliver
early warning signs of an unfavorable result.
Software testing metrics also provide clear, quantifiable answers to important questions such as:
• What is being tested?
• How much of the software is already tested?
• What is the cost of testing?
• How is the required testing effort?
• What is the number of test cases?
• How long will it take to test?
• How many bugs/defects are found?
• How can they be resolved?
• Can testing be done on time?
Importance of Metrics in Software Testing:
Test metrics are essential in determining the software’s quality and performance. Developers may use
the right software testing metrics to improve their productivity.
 • Early Problem Identification: By measuring metrics such as defect density and defect arrival
rate, testing teams can spot trends and patterns early in the development process.
• Allocation of Resources: Metrics identify regions where testing efforts are most needed, which
helps with resource allocation optimization. By ensuring that testing resources are concentrated
on important areas, this enhances the strategy for testing as a whole.
• Monitoring Progress: Metrics are useful instruments for monitoring the advancement of
testing. They offer insight into the quantity of test cases that have been run, their completion
rate, and if the testing effort is proceeding according to plan.
• Continuous Improvement: Metrics offer input on the testing procedure, which helps to foster
a culture of continuous development.
Software Testing Metrics Life Cycle
• Analysis: Identify the most suitable metrics for your testing process.
• Communication: Educate the testing team and stakeholders about the chosen metrics.
• Evaluation: Collect and verify relevant data, then use this data to assess the value of the
metrics.
• Reporting: Create a comprehensive report summarizing the findings and distribute it to
stakeholders for feedback.
How to Calculate Test Metrics?
Metric Description Formula

Defect Density Measures the number of defects Defect Density = Total Defects / Size
relative to the size of the software of the Module
module.

Test Case Evaluate the efficiency of test Test Case Effectiveness = (Defects
Effectiveness cases in detecting defects. Detected / Test Cases Run) x 100

Defect Leakage Quantifies defects missed during Defect Leakage = (Defects Found
testing but found later. Post-Testing / Defects Found Pre-
Testing) x 100

Defect Removal Assesses the effectiveness of the Defect Removal Efficiency =

Efficiency testing process in removing (Defects Removed / Total Defects at
defects. Start) x 100

Test Coverage Indicates the extent to which the Test Coverage = (Tested
software functionalities are Functionalities / Total
tested. Functionalities) x 100

Passed Test Cases Measures the proportion of test Passed Test Cases = (Passed Tests /
Percentage cases that passed. Total Tests Executed) x 100

Failed Test Cases Indicates the proportion of failed Failed Test Cases = (Failed Tests /
Percentage test cases. Total Tests Executed) x 100

Test Execution Tracks the speed of the testing Test Execution Rate = (Test Cases
Rate process. Executed / Planned Test Cases) x 100
 Defect Age Measures the time to fix a defect. Defect Age = Time of Defect Fix –
Time of Defect Discovery

Rework Effort Assesses the effort spent on Rework Effort Ratio = (Rework
Ratio reworking or correcting defects. Effort / Total Testing Effort) x 100

Examples of Software Test Metrics Calculation

Example 1:
Consider a scenario where you have the following data from test case development:
• Total Test Cases Written: 200
• Total Test Cases Executed: 180
• Passed Test Cases: 100
• Failed Test Cases: 80
Using this data, you can calculate various metrics. For example:
• Percentage of Test Cases Executed: (180 / 200) x 100 = 90%
• Test Case Effectiveness: (Number of Defects Detected / Number of Test Cases Run) x 100 =
(10 / 180) x 100 ≈ 5.55%.
Example 2:
Let's take another example for calculating software test metrics:
• Total Number of Defects Identified: 20
• Defects Accepted as Valid by the Development Team: 15
• Defects Deferred for Future Releases: 5
• Defects Fixed: 12
From this data, you can calculate metrics like:
• Fixed Defects Percentage: (12 / 20) x 100 = 60%
• Accepted Defects Percentage: (15 / 20) x 100 = 75%
• Defects Deferred Percentage: (5 / 20) x 100 = 25%
Types of Software Testing Metrics
Since software testing metrics offer quantitative measurement data to estimate the progress, quality, and
health of the software testing process, they help teams in making better decisions for future testing
activities. By getting their hands on reliable data about the testing process, they help understand the
type and scale of improvements needed and effectively plan the next steps.
There are different types of metrics that teams can use for ensuring QA. Let’s look at the top 10 metrics
for measuring quality assurance success:
Process Metrics
Process metrics provide important information on the testing process, indicating common bottlenecks
and the overall level of process efficiency.
1. Test case effectiveness is a software testing metric that measures how effectively test cases detect
problems. It helps in answering how good tests are, assessing if high-value test cases are being run, and
determining the bug-finding ability of the test case.
2. Cycle time measures the amount of time it takes to complete the entire length of the testing process.
It can be in hours, days, or even weeks. It also delivers insight into passed, failed, blocked, incomplete,
and unexecuted test executions for easy absorption of the test run status.
3. Defect fixing time showcases the amount of time it takes for testing teams to remedy a problem –
from the time of detection to the time of resolution. It also gives a better idea of how soon they can
release the product to the market – under budget and time.
Product Metrics
Product metrics shed vital light on the quality of the product and lead to better testing decisions.
1. Number of defects is a software testing metric that helps test engineers dig deeper into the many
aspects of software quality. By having information on the number of defects, teams can determine the
quality and efficiency of the software product and implement remediation measures accordingly.
2. Defect severity and cause is another metric that allows testers to see how severe defects are. It also
provides insight into their root cause and how they will affect the software’s quality.
3. Passed/failed test case metrics inform testing teams of the number of test cases that have passed. Such
insight helps ascertain the functionality, performance, stability, usability, and compatibility of the
software under test and the efforts needed to reduce the frequency of failed tests.
Project Metrics
Project metrics provide information on the productivity and efficiency of the team, tools, and the project
in general.
1. Test coverage metrics measure the test effort. They provide insight into how much of the product or
application is successfully tested. It is a clear indicator of the quality of the test plan and estimates the
amount of testing performed, including functional requirements and risk coverage.
2. Rework effort metrics help testing teams calculate the actual rework efforts spent in resolving a defect
after product delivery. It also helps tweak the product or process to ensure they don’t recur. Using this
metric, teams also get vital insight into the time and cost needed to fix a defect. As a result, they can
arrive at an approximate deadline for deployment.
3. Cost of testing metric specifies the overall expenditure of the testing process. Since people,
infrastructure, and tools contribute toward the total cost of testing, teams need to know how much they
intend to spend and how much they end up spending.
4. Budget/schedule variance is yet another important software testing metric that states the difference
between actual cost/schedule and planned cost/schedule. It helps teams improve financial and
operational planning, work within budgets, and pursue opportunities that accelerate time-to-test.
 Types of Software Testing
In this section, we are going to understand the various types of software testing, which can be
used at the time of the Software Development Life Cycle.
As we know, software testing is a process of analyzing an application's functionality as per
the customer prerequisite.
If we want to ensure that our software is bug-free or stable, we must perform the various types
of software testing because testing is the only method that makes our application bug-free.

The different types of Software Testing

The categorization of software testing is a part of diverse testing activities, such as test
strategy, test deliverables, a defined test objective, etc. And software testing is the execution
of the software to find defects.
The purpose of having a testing type is to confirm the AUT (Application Under Test).
To start testing, we should have a requirement, application-ready, necessary resources
available. To maintain accountability, we should assign a respective module to different test
engineers.
The software testing mainly divided into two parts, which are as follows:
o Manual Testing
o Automation Testing
What is Manual Testing?
Testing any software or an application according to the client's needs without using any
automation tool is known as manual testing.
In other words, we can say that it is a procedure of verification and validation. Manual testing
is used to verify the behavior of an application or software in contradiction of requirements
specification.

We do not require any precise knowledge of any testing tool to execute the manual test cases.
We can easily prepare the test document while performing manual testing on any application.
Classification of Manual Testing
In software testing, manual testing can be further classified into three different types of
testing, which are as follows:
o White Box Testing
o Black Box Testing
o Grey Box Testing

For our better understanding let's see them one by one:

White Box Testing
In white-box testing, the developer will inspect every line of code before handing it over to the
testing team or the concerned test engineers.

Subsequently, the code is noticeable for developers throughout testing; that's why this process
is known as WBT (White Box Testing).
In other words, we can say that the developer will execute the complete white-box testing for
the particular software and send the specific application to the testing team.
The purpose of implementing the white box testing is to emphasize the flow of inputs and
outputs over the software and enhance the security of an application.

White box testing is also known as open box testing, glass box testing, structural testing,
clear box testing, and transparent box testing.
Black Box Testing
Another type of manual testing is black-box testing. In this testing, the test engineer will
analyze the software against requirements, identify the defects or bug, and sends it back to the

development team.

Then, the developers will fix those defects, do one round of White box testing, and send it to
the testing team.
Here, fixing the bugs means the defect is resolved, and the particular feature is working
according to the given requirement.
The main objective of implementing the black box testing is to specify the business needs or
the customer's requirements.
In other words, we can say that black box testing is a process of checking the functionality of
an application as per the customer requirement. The source code is not visible in this testing;
that's why it is known as black-box testing.
For more information about Black box testing, refers to the below
link: [Link]
Types of Black Box Testing
Black box testing further categorizes into two parts, which are as discussed below:
o Functional Testing
o Non-function Testing

Functional Testing
o The test engineer will check all the components systematically against requirement
specifications is known as functional testing. Functional testing is also known
as Component testing.
o In functional testing, all the components are tested by giving the value, defining the
output, and validating the actual output with the expected value.

Functional testing is a part of black-box testing as its emphases on application requirement

rather than actual code. The test engineer has to test only the program instead of the system.
Types of Functional Testing
Just like another type of testing is divided into several parts, functional testing is also classified
into various categories.
The diverse types of Functional Testing contain the following:
 o Unit Testing
o Integration Testing
o System Testing

Now, Let's understand them one by one:

1. Unit Testing
Unit testing is the first level of functional testing in order to test any software. In this, the test
engineer will test the module of an application independently or test all the module
functionality is called unit testing.
The primary objective of executing the unit testing is to confirm the unit components with their
performance. Here, a unit is defined as a single testable function of a software or an application.
And it is verified throughout the specified application development phase.
Integration Testing
Once we are successfully implementing the unit testing, we will go integration testing. It is the
second level of functional testing, where we test the data flow between dependent modules or
interface between two features is called integration testing.
The purpose of executing the integration testing is to test the statement's accuracy between each
module.
Types of Integration Testing

Integration testing is also further divided into the following parts:

o Incremental Testing
o Non-Incremental Testing

Incremental Integration Testing

Whenever there is a clear relationship between modules, we go for incremental integration
testing. Suppose, we take two modules and analysis the data flow between them if they are
working fine or not.
If these modules are working fine, then we can add one more module and test again. And we
can continue with the same process to get better results.
In other words, we can say that incrementally adding up the modules and test the data flow
between the modules is known as Incremental integration testing.
Types of Incremental Integration Testing
Incremental integration testing can further classify into two parts, which are as follows:
1. Top-down Incremental Integration Testing
2. Bottom-up Incremental Integration Testing

Let's see a brief introduction of these types of integration testing:

1. Top-down Incremental Integration Testing
In this approach, we will add the modules step by step or incrementally and test the data flow
between them. We have to ensure that the modules we are adding are the child of the earlier
ones.
2. Bottom-up Incremental Integration Testing
In the bottom-up approach, we will add the modules incrementally and check the data flow
between modules. And also, ensure that the module we are adding is the parent of the earlier
ones.
Non-Incremental Integration Testing/ Big Bang Method
Whenever the data flow is complex and very difficult to classify a parent and a child, we will
go for the non-incremental integration approach. The non-incremental method is also known
as the Big Bang method.
3. System Testing
Whenever we are done with the unit and integration testing, we can proceed with the system
testing.
In system testing, the test environment is parallel to the production environment. It is also
known as end-to-end testing.
In this type of testing, we will undergo each attribute of the software and test if the end feature
works according to the business requirement. And analysis the software product as a complete
system.
Non-function Testing
The next part of black-box testing is non-functional testing. It provides detailed information
on software product performance and used technologies.
Non-functional testing will help us minimize the risk of production and related costs of the
software.
Non-functional testing is a combination of performance, load, stress, usability and,
compatibility testing.
Types of Non-functional Testing
Non-functional testing categorized into different parts of testing, which we are going to discuss
further:
o Performance Testing
o Usability Testing
 o Compatibility Testing

1. Performance Testing
In performance testing, the test engineer will test the working of an application by applying
some load.
In this type of non-functional testing, the test engineer will only focus on several aspects, such
as Response time, Load, scalability, and Stability of the software or an application.
Classification of Performance Testing
Performance testing includes the various types of testing, which are as follows:
o Load Testing
o Stress Testing
o Scalability Testing
o Stability Testing
 o Load Testing
While executing the performance testing, we will apply some load on the particular application
to check the application's performance, known as load testing. Here, the load could be less
than or equal to the desired load.
It will help us to detect the highest operating volume of the software and bottlenecks.
o Stress Testing
It is used to analyze the user-friendliness and robustness of the software beyond the common
functional limits.
Primarily, stress testing is used for critical software, but it can also be used for all types of
software applications.
o Scalability Testing
To analysis, the application's performance by enhancing or reducing the load in particular
balances is known as scalability testing.
In scalability testing, we can also check the system, processes, or database's ability to meet
an upward need. And in this, the Test Cases are designed and implemented efficiently.
o Stability Testing
Stability testing is a procedure where we evaluate the application's performance by applying
the load for a precise time.
It mainly checks the constancy problems of the application and the efficiency of a developed
product. In this type of testing, we can rapidly find the system's defect even in a stressful
situation.
2. Usability Testing
Another type of non-functional testing is usability testing. In usability testing, we will
analyze the user-friendliness of an application and detect the bugs in the software's end-user
interface.
Here, the term user-friendliness defines the following aspects of an application:
 o The application should be easy to understand, which means that all the features must
be visible to end-users.
o The application's look and feel should be good that means the application should be
pleasant looking and make a feel to the end-user to use it.
3. Compatibility Testing
In compatibility testing, we will check the functionality of an application in specific hardware
and software environments. Once the application is functionally stable then only, we go
for compatibility testing.
Here, software means we can test the application on the different operating systems and other
browsers, and hardware means we can test the application on different sizes.
Grey Box Testing
Another part of manual testing is Grey box testing. It is a collaboration of black box and
white box testing.
Since, the grey box testing includes access to internal coding for designing test cases. Grey box
testing is performed by a person who knows coding as well as testing.

In other words, we can say that if a single-person team done both white box and black-box
testing, it is considered grey box testing.
Automation Testing
The most significant part of Software testing is Automation testing. It uses specific tools to
automate manual design test cases without any human interference.
Automation testing is the best way to enhance the efficiency, productivity, and coverage of
Software testing.
It is used to re-run the test scenarios, which were executed manually, quickly, and repeatedly.
In other words, we can say that whenever we are testing an application by using some tools is
known as automation testing.
We will go for automation testing when various releases or several regression cycles goes on
the application or software. We cannot write the test script or perform the automation testing
without understanding the programming language.

Some other types of Software Testing

In software testing, we also have some other types of testing that are not part of any above
discussed testing, but those testing are required while testing any software or an application.
o Smoke Testing
o Sanity Testing
o Regression Testing
o User Acceptance Testing
o Exploratory Testing
o Adhoc Testing
o Security Testing
o Globalization Testing
Let's understand those types of testing one by one:
In smoke testing, we will test an application's basic and critical features before doing one
round of deep and rigorous testing.
Or before checking all possible positive and negative values is known as smoke testing.
Analyzing the workflow of the application's core and main functions is the main objective of
performing the smoke testing.
Sanity Testing
It is used to ensure that all the bugs have been fixed and no added issues come into existence
due to these changes. Sanity testing is unscripted, which means we cannot documented it. It
checks the correctness of the newly added features and components.
Regression Testing
Regression testing is the most commonly used type of software testing. Here, the
term regression implies that we have to re-test those parts of an unaffected application.
Regression testing is the most suitable testing for automation tools. As per the project type and
accessibility of resources, regression testing can be similar to Retesting.
Whenever a bug is fixed by the developers and then testing the other features of the applications
that might be simulated because of the bug fixing is known as regression testing.
In other words, we can say that whenever there is a new release for some project, then we can
perform Regression Testing, and due to a new feature may affect the old features in the earlier
releases.
User Acceptance Testing
The User acceptance testing (UAT) is done by the individual team known as domain
expert/customer or the client. And knowing the application before accepting the final product
is called as user acceptance testing.
In user acceptance testing, we analyze the business scenarios, and real-time scenarios on the
distinct environment called the UAT environment. In this testing, we will test the application
before UAI for customer approval.
Exploratory Testing
Whenever the requirement is missing, early iteration is required, and the testing team has
experienced testers when we have a critical application. New test engineer entered into the
team then we go for the exploratory testing.
To execute the exploratory testing, we will first go through the application in all possible ways,
make a test document, understand the flow of the application, and then test the application.
Adhoc Testing
Testing the application randomly as soon as the build is in the checked sequence is known
as Adhoc testing.
It is also called Monkey testing and Gorilla testing. In Adhoc testing, we will check the
application in contradiction of the client's requirements; that's why it is also known as negative
testing.
When the end-user using the application casually, and he/she may detect a bug. Still, the
specialized test engineer uses the software thoroughly, so he/she may not identify a similar
detection.
Security Testing
It is an essential part of software testing, used to determine the weakness, risks, or threats in
the software application.
The execution of security testing will help us to avoid the nasty attack from outsiders and ensure
our software applications' security.
In other words, we can say that security testing is mainly used to define that the data will be
safe and endure the software's working process.
Globalization Testing
Another type of software testing is Globalization testing. Globalization testing is used to
check the developed software for multiple languages or not. Here, the
words globalization means enlightening the application or software for various languages.
Globalization testing is used to make sure that the application will support multiple languages
and multiple features.
In present scenarios, we can see the enhancement in several technologies as the applications
are prepared to be used globally.
 VERIFICATION AND VALIDATION TESTING

V-Model
V-Model also referred to as the Verification and Validation Model. In this, each phase of SDLC
must complete before the next phase starts. It follows a sequential design process same as the
waterfall model. Testing of the device is planned in parallel with a corresponding stage of
development.

Verification: It involves a static analysis method (review) done without executing code. It is
the process of evaluation of the product development process to find whether specified
requirements meet.
Validation: It involves dynamic analysis method (functional, non-functional), testing is done
by executing code. Validation is the process to classify the software after the completion of the
development process to determine whether the software meets the customer expectations and
requirements.
So V-Model contains Verification phases on one side of the Validation phases on the other side.
Verification and Validation process is joined by coding phase in V-shape. Thus it is known as
V-Model.
There are the various phases of Verification Phase of V-model:
 1. Business requirement analysis: This is the first step where product requirements
understood from the customer's side. This phase contains detailed communication to
understand customer's expectations and exact requirements.
2. System Design: In this stage system engineers analyze and interpret the business of the
proposed system by studying the user requirements document.
3. Architecture Design: The baseline in selecting the architecture is that it should
understand all which typically consists of the list of modules, brief functionality of each
module, their interface relationships, dependencies, database tables, architecture
diagrams, technology detail, etc. The integration testing model is carried out in a
particular phase.
4. Module Design: In the module design phase, the system breaks down into small
modules. The detailed design of the modules is specified, which is known as Low-Level
Design
5. Coding Phase: After designing, the coding phase is started. Based on the requirements,
a suitable programming language is decided. There are some guidelines and standards
for coding. Before checking in the repository, the final build is optimized for better
performance, and the code goes through many code reviews to check the performance.
There are the various phases of Validation Phase of V-model:
1. Unit Testing: In the V-Model, Unit Test Plans (UTPs) are developed during the module
design phase. These UTPs are executed to eliminate errors at code level or unit level. A
unit is the smallest entity which can independently exist, e.g., a program module. Unit
testing verifies that the smallest entity can function correctly when isolated from the
rest of the codes/ units.
2. Integration Testing: Integration Test Plans are developed during the Architectural
Design Phase. These tests verify that groups created and tested independently can
coexist and communicate among themselves.
3. System Testing: System Tests Plans are developed during System Design Phase.
Unlike Unit and Integration Test Plans, System Tests Plans are composed by the client?s
business team. System Test ensures that expectations from an application developer are
met.
4. Acceptance Testing: Acceptance testing is related to the business requirement analysis
part. It includes testing the software product in user atmosphere. Acceptance tests reveal
the compatibility problems with the different systems, which is available within the user
atmosphere. It conjointly discovers the non-functional problems like load and
performance defects within the real user atmosphere.
When to use V-Model?
o When the requirement is well defined and not ambiguous.
o The V-shaped model should be used for small to medium-sized projects where
requirements are clearly defined and fixed.
 o The V-shaped model should be chosen when sample technical resources are available
with essential technical expertise.
Advantage (Pros) of V-Model:
1. Easy to Understand.
2. Testing Methods like planning, test designing happens well before coding.
3. This saves a lot of time. Hence a higher chance of success over the waterfall model.
4. Avoids the downward flow of the defects.
5. Works well for small plans where requirements are easily understood.
Disadvantage (Cons) of V-Model:
1. Very rigid and least flexible.
2. Not a good for a complex project.
3. Software is developed during the implementation stage, so no early prototypes of the
software are produced.
4. If any changes happen in the midway, then the test documents along with the required
documents, has to be updated.
What is Verification Testing?
Verification testing is performed to check if a system meets its specified requirements at
different stages of its creation.
The IEEE-STD-610 definition of verification testing is:
“Verification is a test of a system to prove that it meets all its specified requirements at a
particular stage of its development.”
In simpler terms, verification testing involves checking whether a system or component meets
the requirements and specifications set for it, ensuring that it is built correctly according to the
defined standards.
It involves reviewing various documents like requirements specification, design blueprints, ER
diagrams, database table design, test cases, test scenarios, traceability matrix, etc. to ensure
everything aligns with the project’s standards and protocols.
This process helps confirm that the system is designed correctly and functions as intended,
making sure that all elements—software, hardware, documentation, and team practices—are
error-free and meet the necessary standards.
Code reviews, walkthroughs, inspections, design, and specification analysis are common
components of verification testing.
Verification tests ensure that all development elements (software, hardware, documentation,
and human resources) adhere to organizational and team-specific standards and protocols. It
checks to authenticate that the system design and architecture are accurately engineered and
error-free. Verification checks are often like studying the specifications and checking them
against the code logic.

Advantages of Verification Testing

• Early and frequent verification reduces the number of bugs and defects that may show
up in later stages.
• By verifying at each stage, devs, product managers, and stakeholders can get more
insight into what the product may need to be developed better in the coming stages.
• Even if they can’t solve all bugs immediately, verifying helps QAs estimate the
emerging issues and help them better prepare to handle those when they appear.
• Verification helps keep software closely aligned with customers and business
requirements at every stage. This ensures that devs have to put in less unnecessary work
as development continues
When to use Verification Testing?
Some common scenarios where verification testing is used are:
• Before Code Integration: When developers complete a module of code, verification
testing ensures that this module meets its design specifications and works correctly
before integrating it into the larger system.
• During Design Reviews: After creating a design blueprint for a new feature,
verification testing checks if the design aligns with the initial requirements and design
documents.
• Before User Acceptance Testing: Before a system is handed over for user acceptance
testing, verification testing confirms that the system meets all specified requirements
and is free from critical errors.
• After Software Updates: When a software update or patch is applied, verification
testing ensures that the changes have been implemented correctly and that they meet
the specified requirements without introducing new issues.
• During System Integration: When different system components are integrated,
verification testing checks if the integrated system meets all the defined requirements
and functions as expected.
• When Requirements Change: If project requirements are updated or revised,
verification testing ensures that the system still complies with the new requirements.
What is Validation Testing?
Validation testing is the process used to ensure that a final product meets the true needs and
expectations of stakeholders.
The IEEE-STD-610 definition of validation testing is:
“Validation is an activity that ensures that an end product stakeholder’s true needs and
expectations are met.”
In other words, validation testing is used to verify that the final product fulfills the requirements
and expectations of the stakeholders and customers. This process typically occurs after the
product is fully developed, focusing on confirming that it works as intended in real-world
scenarios.
Unlike verification testing, which happens throughout the development process, validation
testing typically occurs at the end of a development phase or after the entire system is built. Its
main goal is to confirm that the final product aligns with what stakeholders and customers
wanted.
Unlike verification testing, which occurs at every stage in development, validation testing
occurs at the end of a specific module or even after the software has been entirely built. Its
primary intent is to ensure the final product matches the stakeholder and customer
requirements.
Most forms of QA fall under this category. This includes various types of quality assurance
tests, such as unit tests and User Acceptance Tests, which check that the system functions as
intended and delivers measurable results. Some of the standard tests under this heading:
• Unit Testing
• Integration Testing
• Smoke Testing
• Functional Testing
• Performance Testing
• Regression Testing
• Security Testing
• System Testing
• Accessibility Testing
• Cross Browser Testing
• User Acceptance Testing
All validation tests ensure that a system works as planned by running all its functions and
tracking tangible, quantifiable results.

Advantages of Validation Testing

• Any bugs missed during verification will be detected while running validation tests.
• If specifications were incorrect and inadequate, validation tests would reveal their
inefficacy. Teams will have to spend time and effort fixing them, but it will prevent a
bad product from hitting the market.
• Validation tests ensure that the product matches and adheres to customer demands,
preferences, and expectations under different conditions (slow connectivity, low
battery, etc.)
• These tests are also required to ensure the software functions flawlessly across different
browser-device-OS combinations. In other words, it authenticates software for cross
browser compatibility.

When to use Validation Testing?

Some popular scenarios where validation testing is used:
• After System Completion: Once the entire system or software is fully developed and
all components are integrated, validation testing ensures that the final product meets the
requirements and expectations of stakeholders.
• During User Acceptance Testing (UAT): When the product is ready for end-users to
test, validation testing is used to confirm that it meets their needs and functions correctly
in real-world scenarios.
• Before Product Launch: Prior to releasing a product to the market, validation testing
is conducted to ensure that it satisfies customer requirements and performs as expected
under actual usage conditions.
 • After Major Changes or Enhancements: When significant updates or new features
are added to a system, validation testing checks that these changes align with
stakeholder expectations and do not negatively impact the overall functionality.
• During Beta Testing: In beta testing phases, where a product is released to a select
group of users for feedback, validation testing is used to gather insights on whether the
product meets user needs and expectations.
• Post-Implementation Review: After the product is deployed and in use, validation
testing can be performed to verify that it continues to meet the stakeholders’ needs and
functions as intended.
These scenarios help ensure that the final product delivers value to the end-users and fulfills
the initial requirements and expectations.
Difference between Verification and Validation in Software engineering
Verification testing ensures a product is built according to specified requirements, while
validation testing ensures the final product meets the actual needs and expectations of
stakeholders.
This table outlines the key distinctions between verification and validation in software
engineering.
Parameter Verification Testing Validation Testing
Definition Ensures the product meets Ensures the product meets the
specified requirements at stakeholders’ true needs and
various stages of development. expectations.
Objective To confirm that the product is To confirm that the right
being built correctly according product has been built and
to requirements and design meets the end-user needs and
specifications. requirements.
Timing Performed at each stage of Performed after the
development (e.g., during development is complete, often
coding, design, etc.). at the end of the project or
phase.
Activities Reviews, inspections, code User acceptance testing, system
analysis, unit testing. testing, beta testing.
Type of Testing Static and dynamic testing Primarily dynamic testing
activities. activities.
Documentation Reviewed Design documents, Final product, user feedback,
requirement specifications, system performance.
code.
Example Ensuring that a software Ensuring that a software
module conforms to the design application meets the
document. requirements and expectations
of the users.
Example of Verification and Validation Testing
We can better understand the concept of verification and validation testing using some
scenarios
Scenario 1: Developing a New Online Shopping Application
Verification Testing Example
• Requirement Review: During the development phase, the team reviews the
requirements document to ensure that the feature for user login meets all specified
criteria, such as password strength and security protocols.
• Code Inspection: The development team conducts a code review to ensure that the
implementation of the user login feature aligns with the design specifications and
adheres to coding standards.
Validation Testing Example
• User Acceptance Testing (UAT): Once the user login feature is fully implemented and
integrated into the application, end-users test the login functionality to confirm it meets
their expectations, such as ease of use and secure access.
• Beta Testing: The application is released to a group of real users in a controlled
environment to gather feedback on whether the login process works effectively and
meets the users’ needs for convenience and security.
Summary
In this scenario, Verification Testing ensures that the user login feature is built correctly
according to the design and requirements. Validation Testing ensures that the user login feature
actually meets the end-users’ needs and works as expected in real-world scenarios.
Scenario 2: Developing a New Mobile Banking App
Verification Testing Example
• Design Review: The development team reviews the design document for the mobile
banking app to ensure that the features, such as transaction history and balance
checking, are properly designed according to the specifications.
• Unit Testing: Individual components, like the transaction processing module, are tested
to confirm they function correctly and adhere to the technical requirements and design
specifications.
Validation Testing Example
• System Testing: After the mobile banking app is fully developed, the complete app is
tested to ensure that the transaction history and balance checking features meet the real-
world needs of users, such as ease of access and accurate data display.
• User Feedback: The app is released to a group of actual users to gather feedback on
whether the features are intuitive, useful, and align with their expectations for managing
their finances effectively.
Summary : In this scenario, Verification Testing ensures that the mobile banking app’s features
are built correctly according to the design and requirements.
Validation Testing ensures that the app’s features meet users’ needs and expectations in real-
world use, confirming that the app provides a satisfactory and functional user experience.
DEFECT TRACKING
What is Defect Tracking?
Defect tracking, aka Bug tracking, is the systematic process of identifying, recording, monitoring, and
managing defects or issues in a product or system throughout its development lifecycle. These defects
can encompass various aspects, including software bugs, hardware malfunctions, design flaws, or other
imperfections that may hinder the product’s functionality, performance, or quality.
Why is Defect Tracking Required?
A defect tracker is like a digital journal that records all the problems or errors in a product or project.
It’s like having a list of everything that’s not working correctly.
Here’s why it’s important and how it works:
1. Preventing Errors from Being Missed: Imagine you’re building a house, and there are many
different parts to it—plumbing, electrical, walls, and so on. If you don’t keep a list of every
issue you discover (like a leaky faucet or a cracked wall), you might forget to fix some of them.
A defect tracker is like that list; it ensures that every problem, or “defect,” is written down so
that none of them are missed.
2. Saving Time: When building something or creating a project, you want to avoid checking the
same things repeatedly. That’s not only frustrating but also a waste of time. A defect tracker
helps with this by tracking what issues have already been fixed. It’s like putting a checkmark
next to a problem once it’s solved. This way, you don’t have to keep looking at the same issues,
and you can focus on what still needs to be fixed.
3. Working on the Right Problems: Sometimes, there are so many issues that it can be
overwhelming. Imagine you have a pile of homework and need help figuring out where to start.
A defect tracker helps you decide what’s most important. It can categorize the problems based on how
severe they are. For example, a major issue like a broken door is more important to fix than a small
scratch on the wall. So, it helps ensure that the correct problems are being worked on first.
Let me put it all in simple terms: A defect tracker is like a to-do list for fixing problems in a project. It
ensures that everything is remembered, keeps track of what’s already set, and helps you focus on the
most important issues. This way, you can build or create things more efficiently and ensure they work
as they should.
Objectives
Here are some objectives of defect tracking:
1. Keeping Track of All Defects: Imagine you have a notebook where you write down every
mistake you find in your homework. Bug tracking is like that notebook for software. It ensures
that every little error or “defect” is written down so none are forgotten. This is super important
because missing a defect could cause problems later when people use the software.
2. Finding the Best Solutions and Preventing More Defects: Once you’ve written down all the
defects, it’s like having a list of things to fix. But it’s not just about setting them; it’s also about
figuring out the best way. Think of it as having a guide telling you the smartest way to solve a
problem. Plus, it helps you learn from your mistakes and prevent similar issues from happening
again. It’s like learning not to spill your juice after you’ve cleaned up a spill once.
 3. Saving Time and Doing Better Work: Imagine you’re building a puzzle. If you keep looking
for the same piece repeatedly, it will take forever to finish the puzzle. Defect tracking ensures
you don’t waste time looking at the same problems repeatedly. It’s like having a helper who
says, “You’ve already solved that part; now let’s work on the parts that still need fixing.” This
saves time and helps you do a better job because you can focus on what’s most important.
Tracking software defects saves time and improves quality. It’s like having a superhero assistant to find
solutions and avoid repeating mistakes. Get superpowers for better software!
Key features
Here are some key features listed below:
• Effective communication: Defect tracking is a team effort, so it is essential to have effective
communication between all stakeholders. This includes the developers, testers, and managers.
• Proper documentation: All defects should be properly documented, including the steps to
reproduce the defect, the severity of the defect, and the impact of the defect. This documentation
will help the developers to fix the defects quickly and efficiently.
• Regular reporting: Defect tracking should be a continuous process. Regularly reporting
defects’ status will help identify any trends or patterns. This information can be used to improve
the defect tracking process and to prevent defects from occurring in the future.
• Prioritization: Not all defects are created equal. Some defects are more critical than others. It
is essential to prioritize defects so that the most critical defects are fixed first.
• Tracking of defects: It is important to track the progress of defects. This includes tracking the
status of the defect, the assigned engineer, and the estimated time to fix the defect. This tracking
information will help to ensure that defects are fixed on time.
• Resolution of defects: The goal of is to resolve defects. This means that the defect should be
fixed, and the fix should be verified.
• Continuous improvement: Defect tracking is an ongoing process. It is essential to improve
the defect-tracking process continuously. This can be done by identifying and addressing gaps
or weaknesses in the process.
By following these key factors, you can effectively track defects and ensure the quality of your software.
Defect tracking Parameters
Defect tracking parameters are the attributes that are used to track defects. These parameters can be
used to identify, prioritize, and manage defects.
Some of the common defect-tracking parameters include:
• ID: A unique identifier for each defect.
• Title: A brief description of the defect.
• Description: A precise description of the defect, including the steps to reproduce the defect.
• Severity: The severity of the defect, such as critical, major, or minor.
• Priority: The priority of the defect, such as high, medium, or low.
• Status: The status of the defect, such as open, closed, or deferred.
 • Assigned to: The engineer who is assigned to fix the defect.
• Due date: The date by which the defect should be fixed.
• Comments: Any comments about the defect.
What you are looking at are just some of the common defect-tracking parameters. These parameters can
vary depending on the project and the team’s needs.
How to Design a Defect Tracking System/Process?
Here are the steps on how to design a defect-tracking system/process:
1. Define the goals of the defect tracking system. What do you want to achieve with the defect-
tracking system? Do you want to track, prioritize, manage, or generate reports on defects?
2. Identify the stakeholders. Who will be using the defect tracking system? Will it be used by
developers, testers, managers, or other stakeholders?
3. Define the parameters. What information will be tracked about each defect? This could
include the defect ID, title, description, severity, priority, status, assigned to, due date, and
comments.
4. Select a defect tracking tool. There are many defect-tracking tools available. Pick a tool that
meets the needs of your project and team.
5. Configure the defect tracking tool. Once you have selected a defect-tracking tool, you need
to configure it to meet the needs of your project. This includes setting up the defect tracking
parameters and defining the workflows for defect submission, prioritization, and management.
6. Train the users. Once the defect tracking system is in place, you must train the users to use it.
This includes how to submit defects, prioritize defects, manage defects, and generate reports.
7. Monitor and improve the defect tracking system. Once the system is used, you must monitor
it to ensure it meets your project’s needs. This includes identifying gaps or weaknesses in the
system and making necessary improvements.
Let me whisper to you some more extra tips for designing a defect-tracking system/process:
• Make sure the system is easy to use and navigate.
• Let users customize the system to meet their specific needs.
• Integrate the system with other tools and systems, such as version control and build automation
tools.
• Automate as much of the process as possible.
• Provide regular reports on the status of defects.
• Continuously improve the system based on feedback from users.
By following these steps, you can design a defect-tracking system/process that will help you improve
the quality of your software.
How does Defect Tracking/Bug Tracking Work?
A defect tracking or monitoring tool keeps track of all the defects in an application to ensure all errors
get noticed during the evaluation process. However, to understand how the tracking is helping improve
the quality of software, it is crucial to know how defect tracking functions.
It initiates with the bug-tracking software, maintaining a database of all reported bugs on a central
server. These bugs are then categorized based on the software company’s criteria. Depending on the
severity of the error, each bug is prioritized, and further action is taken to resolve it.
Challenges
Defect tracking is the process of identifying, documenting, and managing defects in software. It is
an important part of the software development process, as it helps to ensure that software is released
with a high level of quality.
Yet, defect tracking can be challenging. Here are some of the most common challenges:
• Only complete or accurate defect reports: Defect reports needing more essential details or
inaccuracies can make tracking and managing defects difficult. Clear and precise reporting is
necessary for efficient defect resolution.
• Poor communication: Inadequate communication among stakeholders can result in defects
being overlooked or left unresolved for extended periods. Effective collaboration and
information sharing are vital to ensure timely defect fixes.
• Inefficient workflow: An inefficient workflow can lead to delays in defect resolution and even
result in some defects slipping through the cracks. Streamlining processes is key to preventing
such issues.
• Lack of automation: Relying on manual defect tracking processes can be time-consuming and
lead to errors. Automation tools can significantly improve efficiency and accuracy in defect
tracking.
• Limited visibility: With clear visibility into defects’ status, monitoring progress and
identifying emerging patterns or trends becomes easier. Robust reporting and tracking
mechanisms are essential for better visibility.
• Ownership ambiguity: When defects lack clear ownership or responsibility, they may be
addressed. Assigning ownership ensures accountability and timely resolution.
• Production defects: Defects that escape into the production environment can disrupt user
experiences and tarnish the software’s reputation. Rigorous testing and proactive defect
management are essential to prevent production issues.
Let me give you some bonuses for overcoming these challenges:
• Use a defect tracking tool: A defect tracking tool can help to automate the tracking process
and improve accuracy.
• Define clear defect reporting guidelines: Clear defect reporting guidelines will help to ensure
that defect reports are complete and accurate.
• Establish a communication plan: This will help to ensure that stakeholders are kept informed
of the status of defects.
• Regularly review the defect tracking process: Regularly reviewing the process will help
identify and address any gaps or weaknesses.
• Train the users: Training the users on how to use the defect tracking system will help ensure
the system is used effectively.
What is defect logging and tracking?
 • Defect logging is the process of identifying and documenting software defects. This includes
capturing the defect’s description, severity, and steps to reproduce.
• Defect tracking is the process of managing defects from identification to resolution. This
includes prioritizing defects, assigning them to developers, and tracking their progress.
What Is a Defect/Bug Life Cycle?
A defect/bug life cycle is the sequence of steps a bug or defect goes through from its identification to
its resolution in software development. This life cycle standardizes the bug management process,
ensuring teams can manage and resolve them more effectively. Detecting bugs in the development phase
allows the developers to address issues as early as possible, not letting them get entrenched into the
code and become harder to fix. There are two ways to describe the defect life cycle:
• By bug workflow
• By bug status
1. Bug Life Cycle Workflow
The flow chart below gives you some idea of the process that unfolds when a bug is identified. It is
based on personal experience with various testing teams, so it’s not representative of every bug process,
but it is still a typical cycle for a bug in software testing.

The table below provides a more detailed breakdown of the activities at each stage and the staff
members who are likely to be involved. There are three distinct groups: testers, developers, and project
managers or leads.
The table below provides a more detailed breakdown of the activities at each stage and the staff
members who are likely to be involved. There are three distinct groups: testers, developers, and project
managers or leads.

Bug Life Cycle Activities People Involved

1. Execute tests. Tester

2. Report and submit the newly found bug to a bug management system. Tester
Set bug status to new.

3. Review and decide whether to fix the bug. Assign developers if Project lead, project
necessary. Set bug status to in progress, under investigation, or similar. manager, tester

4. Investigate and reproduce the bug. Developer

 5. If the bug can be successfully reproduced, fix it. Otherwise, request more Developer, tester
information from the tester. Update bug status accordingly.

6. Provide further description or use bug-reporting tools to elaborate the Tester

bug.

7. Verify the fix by executing steps described in the bug report. Tester

8. Close the bug if the fix is verified. Otherwise, update bug status and Tester
provide further explanation.

2. Bug Life Cycle Stages

We can also look at the bug life cycle from the status of bugs. Each bug status corresponds to a specific
activity in the bug life cycle:
1. New/Open
2. Deferred/Postponed
3. Assigned
4. In Progress
5. Fixed
6. Pending Retest
7. Verified/Closed
8. Reopened

Bug Status Description

New/Open The bug has been reported or identified and is awaiting review. It is typically
assigned a unique identifier and entered into the bug tracking system.

Deferred/Postponed There are two scenarios here:

• The bug can’t be reproduced.
• The bug is considered valid but not critical or prioritized for
immediate resolution.
It will be marked as "Deferred" or "Postponed" to be addressed in a future
release or development cycle.
 Assigned The bug has been reviewed and assigned to a specific developer or team
responsible for fixing it. This stage involves triaging the bug, prioritizing it,
and allocating resources for resolution.

In Progress The assigned developer has started working on fixing the bug. They are
analyzing the bug's root cause and implementing necessary code changes or
fixes.

Fixed The developer has made the required code changes to address the bug.

Pending Retest After the bug fix, the bug is marked as "Pending Retest." The QA team will
make final verification if the bug has truly been fixed.

Verified/Closed The bug has been re-tested by the QA team and confirmed to have been
resolved completely.

Reopened If the bug is found to be unresolved or if new issues arise after verification, it
is reopened. The development process continues from the "In Progress" stage
to address the remaining issues.

Continuous Improvement in the Bug Life Cycle

The defect life cycle as shown above is the standard process, but QA teams should strive to optimize it
and improve the bug resolution efficiency. Based on observations and analytics generated from test
runs, teams can make organization-specific tweaks to this workflow. Below are several
recommendations to make your bug life cycle better:
• Establish clear bug taxonomies to better categorize newly found bugs. It is essentially a shared
language between the testers and developers, allowing them to describe the bug in a structured
manner. Here are several bug criteria to get started with:
o Severity (High - Medium - Low impact to system performance/security)
o Priority (High - Medium - Low urgency)
o Reproducibility (Reproducible, Intermittent, Non-Reproducible, or Cannot Reproduce)
o Root Cause (Coding Error, Design Flaw, Configuration Issue, or User Error, etc.)
o Bug Type (Functional Bugs, Performance Issues, Usability Problems, Security
Vulnerabilities, Compatibility Errors, etc.)
o Areas of Impact
o Frequency of Occurrence
• Establish clear bug reporting guidelines (context, bug categories as listed above, step-by-step
guide to reproduce bugs, environment details, screenshots (if needed), expected results versus
current results, etc.).
• Adopt bug tracking tools to automate certain stages of the bug life cycle (bug logging,
assignment, etc.) and improve efficiency.
• Perform root cause analysis to prevent similar issues from happening in the future. The root
cause analysis involves the following steps:
o Gather data from previous test runs, system logs, user feedback, and available
documentation to prove that there is a bigger issue to be tackled.
o Brainstorm and identify potential root causes for the bug (use popular techniques like
the 5 Whys, Fishbone Diagrams, or Fault Trees).
o Determine the likelihood and impact of each root cause.
• Adopt continuous testing and ensure that the bug life cycle fits seamlessly into this process
by leveraging CI/CD tools.