UNIT III Software Engineering

Chapter 13
Software Testing

Software testing is an important process in the Software Development Lifecycle (SDLC).

It involves verifying and validating that a Software Application is free of bugs, meets the
technical requirements set by its Design and Development, and satisfies user requirements
efficiently and effectively.

➢ What is Software Testing?

Software Testing is a process of verifying and validating whether the Software
Product or Application is working as expected or not. The complete testing includes
identifying errors and bugs that cause future problems for the performance of an
application.

Software Testing Can be Divided into Two Steps:

Software testing mainly divides into the two parts, which is used in the Software
Development Process:
1. Verification: This step involves checking if the software is doing what is
supposed to do. Its like asking, "Are we building the product the right way?"
2. Validation: This step verifies that the software actually meets the customer's
needs and requirements. Its like asking, "Are we building the right product?"

What is Verification?
Verification is the process of ensuring that the software work products (such as
requirements, design, and code) conform to specified requirements and standards. It focuses
on checking correctness and consistency without necessarily executing the software.
What is Validation?
Validation is the process of evaluating the software during or after development to ensure it
fulfills its intended use and meets user and stakeholder needs. It typically involves
executing the software and observing its behavior in real or simulated environments.

Differences between Verification and Validation

Here is the Differences between Verification and Validation
Verification Validation

Ensures work products conform to Ensures the final product meets

Definition specified requirements user and stakeholder needs

Requirements, design, and Functional behavior and intended

Focus implementation correctness use

1
Tanvi Nerurkar
UNIT III Software Engineering

Verification Validation

Typically evaluation- and

Often analytical and review-based
Nature execution-based

May or may not involve executing Usually involves executing the

Execution software software

Methods used in verification are

Functional testing, system testing,
Methods reviews, walkthroughs,
acceptance testing
Used inspections and desk-checking.

It checks whether the software

It checks whether the software
meets the requirements and
conforms to specifications or not.
Purpose expectations of a customer or not.

Helps detect defects early by Detects defects during execution,

checking conformance to including functional, usability,
Bug requirements and standards. and integration issues.

The goal of verification is

The goal of validation is an actual
application and software
product.
Goal architecture and specification.

Involves developers, reviewers, Involves testers, users, QA teams,

Responsibility QA, and engineers and stakeholders.

Performed throughout the

Performed throughout the software
lifecycle when the system or
development lifecycle.
Timing components are executable.

Applied continuously across all Applied iteratively as the system

Lifecycle lifecycle phases. evolves and becomes usable.

Verification is for prevention of Validation is for detection of

Error Focus errors. errors.

Based on defined criteria, Based on observed system

Stability standards, and expert judgment. behavior and acceptance criteria.

2
Tanvi Nerurkar
UNIT III Software Engineering

Real-World Example of Verification vs Validation

• Verification Example: Imagine a team is developing a new mobile banking
app. During the verification phase, they review the requirements and design
documents. They check if all the specified features like fund transfer, account
balance check, and transaction history are included and correctly detailed in the
design. They also perform peer reviews and inspections to ensure the design
aligns with the requirements. This step ensures that the app is being built
according to the initial plan and specifications without actually running the app.
• Validation Example: In the validation phase, the team starts testing the mobile
banking app on actual devices. They check if users can log in, transfer money,
and view their transaction history as intended. Testers perform usability tests to
ensure the app is user-friendly and functional tests to ensure all features work
correctly. They might also involve real users to provide feedback on the app's
performance. This phase ensures that the app works as expected and meets user
needs in real-world scenarios.

Advantages of Differentiating Verification and Validation

Differentiating between verification and validation in software testing offers several
advantages:
1. Clear Communication: It ensures that team members understand which aspects
of the software development process are focused on checking requirements
(verification) and which are focused on ensuring functionality (validation).
2. Efficiency: By clearly defining verification as checking documents and designs
without executing code, and validation as testing the actual software for
functionality and usability, teams avoid redundant efforts and streamline their
testing processes.
3. Minimized Errors: It reduces the chances of overlooking critical requirements
or functionalities during testing, leading to a more thorough evaluation of the
software's capabilities.
4. Cost Savings: Optimizing resource allocation and focusing efforts on the right
testing activities based on whether they fall under verification or validation helps
in managing costs effectively.
5. Client Satisfaction: Ensuring that software meets or exceeds client and user
expectations by conducting both verification and validation processes rigorously
improves overall software quality and user satisfaction.
6. Process Improvement: By distinguishing between verification and validation,
organizations can refine their testing methodologies, identify areas for
improvement, and enhance the overall Software development lifecycle
(SDLC).

➢ Introduction to Testing:

A model of the software testing process

3
Tanvi Nerurkar
UNIT III Software Engineering

A general model of the testing process is shown in above Figure.

● Test cases are specifications of the inputs to the test and the expected output from the
system plus a statement of what is being tested.
● Test data are the inputs that have been devised to test the system.
● Test data can sometimes be generated automatically. Automatic test case generation is
impossible.
● The output of the tests can only be predicted by people who understand what the system
should do.
● Exhaustive testing, where every possible program execution sequence is tested, is
impossible.
● Testing, therefore, has to be based on a subset of possible test cases.
● Ideally, software companies should have policies for choosing this subset rather than leave
this to the development team.
● These policies might be based on general testing policies, such as a policy that all program
statements should be executed at least once.
● Alternatively, the testing policies may be based on experience of system usage and may
focus on testing the features of the operational system.

For example:
1. All system functions that are accessed through menus should be tested.
2. Combinations of functions (e.g., text formatting) that are accessed through the same menu
must be tested.
3. Where user input is provided, all functions must be tested with both correct and incorrect
input.

➢ Need for Software Testing

Software bugs can cause potential monetary and human loss. There are many examples in
history that clearly depicts that without the testing phase in software development lot of
damage was incurred. Below are some examples:
• 1985: Canada's Therac-25 radiation therapy malfunctioned due to a software bug
and resulted in lethal radiation doses to patients leaving 3 injured and 3 people
dead.
• 1994: China Airlines Airbus A300 crashed due to a software bug killing 264
people.
• 1996: A software bug caused U.S. bank accounts of 823 customers to be
credited with 920 million US dollars.
• 1999: A software bug caused the failure of a $1.2 billion military satellite
launch.
• 2015: A software bug in fighter plane F-35 resulted in making it unable to detect
targets correctly.

4
Tanvi Nerurkar
UNIT III Software Engineering

• 2015: Bloomberg terminal in London crashed due to a software bug affecting

300,000 traders on the financial market and forcing the government to postpone
the 3bn pound debt sale.
• Starbucks was forced to close more than 60% of its outlet in the U.S. and
Canada due to a software failure in its POS system.
• Nissan cars were forced to recall 1 million cars from the market due to a
software failure in the car's airbag sensory detectors.

➢ Principle of Testing
Software testing is an important aspect of software development, ensuring that
applications function correctly and meet user expectations. From test planning to execution,
analysis and understanding these principles help testers in creating a more structured and
focused approach to software testing, resulting in a higher-quality product.

Here are the Seven Principles of Software Testing:

1. Testing shows the Presence of Defects
The goal of software testing is to make the software fail. Software testing reduces the
presence of defects. Software testing talks about the presence of defects and doesn't talk
about the absence of defects. Software testing can ensure that defects are present but it can
not prove that software is defect-free. Even multiple tests can never ensure that software is
100% bug-free. Testing can reduce the number of defects but not remove all defects.
2. Exhaustive Testing is not Possible
It is the process of testing the functionality of the software in all possible inputs (valid or
invalid) and pre-conditions is known as exhaustive testing. Exhaustive testing is impossible
means the software can never test at every test case. It can test only some test cases and
assume that the software is correct and it will produce the correct output in every test case.

5
Tanvi Nerurkar
UNIT III Software Engineering

If the software will test every test case then it will take more cost, effort, etc., which is
impractical.
3. Early Testing
To find the defect in the software, early test activity shall be started. The defect detected in
the early phases of SDLC will be very less expensive. For better performance of software,
software testing will start at the initial phase i.e. testing will perform at the requirement
analysis phase.
4. Defect Clustering
In a project, a small number of modules can contain most of the defects. The Pareto
Principle for software testing states that 80% of software defects come from 20% of
modules.
5. Pesticide Paradox
Repeating the same test cases, again and again, will not find new bugs. So it is necessary to
review the test cases and add or update test cases to find new bugs.
6. Testing is Context-Dependent
The testing approach depends on the context of the software developed. Different types of
software need to perform different types of testing. For example, The testing of the e-
commerce site is different from the testing of the Android application.
7. Absence of Errors Fallacy
If a built software is 99% bug-free but does not follow the user requirement then it is
unusable. It is not only necessary that software is 99% bug-free but it is also mandatory to
fulfill all the customer requirements.

➢ Level of Testing
Software Testing is an important part of the Software Development Life Cycle which is
help to verify the product is working as expected or not. In SDLC, we used different
levels of testing to find bugs and errors.

What Are the Levels of Software Testing?

Software Testing is essential to verify an application is working properly or not. To achieve
this, testing can be done through various stages of development from each component to all
system. Those "stages", called Levels of Testing, help to make sure all components of
system are working properly.

6
Tanvi Nerurkar
UNIT III Software Engineering

1. Unit Testing
Unit Testing is the first step in testing your software. It focuses on checking individual
components or functions of the application to make sure they work correctly on their own.
The goal here is to catch any issues early before those small components are integrated with
the rest of the system.
Unit tests help developers spot bugs early in the development process, making it easier and
quicker to fix them. It's the first layer of defense to verify that each part of the application
performs as expected.
Mainly features are:
• Testing individual functions or components.
• Ensuring correct behavior at the smallest level.
• Catching errors early.

2. Integration Testing
After unit testing, Integration Testing takes over. This level checks how different modules
or components of the software work together. It's important because even if individual parts
work perfectly, they might face issues when interacting with one another.
Integration testing ensures that data flows correctly between modules and that they
communicate seamlessly. It helps catch problems that might arise when different parts of
the system interact.
3. System testing
System Testing is when you test the software as a system. This stage checks whether the
entire system functions as expected in a real-world environment. It includes both functional
and non-functional tests to ensure that the software meets customer needs.
• Full end-to-end testing of the software.
• Verifying both functional and non-functional requirements.
• Testing the software's behavior in real-world conditions.

4. Acceptance Testing
Acceptance Testing, also known as User Acceptance Testing (UAT), is the final test
before releasing the software to the end-users. In this phase, the customer or end-users
verify if the software meets their needs and expectations.
UAT is crucial because it verify the software is ready for production. It’s the last chance to
catch any overlooked issues before deployment. If the software passes this stage, the
customer gives the green light for release.
• Validating the software against business requirements.

7
Tanvi Nerurkar
UNIT III Software Engineering

• Ensuring the software meets customer expectations.

• Getting final approval from the customer.

➢ Types of Software Testing

Here are the Types of Software Testing mainly categorized into the two domain, which are
below.

Fig:Software Testing Types

1. Manual Testing
Manual Testing is a technique to test the software that is carried out using the functions
and features of an application. Which means manual testing will check the defect manually
with trying one by one function is working as expected.
2. Automation Testing
Automation Testing It is a technique where the Tester writes scripts independently and
uses suitable Software or Automation Tools to test the software. It is an Automation
Process of a Manual Process. It allows for executing repetitive tasks without the use of a
Manual Tester.
Types of Manual Testing
Manual testing will be divided into further types which is following:
1. White Box Testing
White Box Testing is a software testing technique that involves testing the internal
structure and workings of a software application. The tester has access to the source code
and uses this knowledge to design test cases that can verify the correctness of the software
at the code level.

2. Black Box Testing

Black-Box Testing is a type of software testing in which the tester is not concerned with
the internal knowledge or implementation details of the software but rather focuses on
validating the functionality based on the provided specifications or requirements.

8
Tanvi Nerurkar
UNIT III Software Engineering

Differentiate between Black Box Testing and White Box Testing

S. No. Black Box Testing White Box Testing

This testing has high-level

1. This testing has Low granularity.
granularity.

It is done by end-users and also by It is generally done by testers and

2.
the tester and developers. developers.

Here, Internals are not required to be Here, the Internal code of the
3.
known. application and database is known.

It is likely to be less exhaustive than

4. Most exhaustive among all three.
the other two.

It is based on requirements, and test

cases on the functional It can exercise code with a relevant
5.
specifications, as the internals are variety of data.
not known.

Why is Importance of Software Testing?

Software testing is important for the several reasons and it mainly classified in the
following reason which are affecting the most.
• Defects can be Identified Early: Software testing is important because if there
are any bugs they can be identified early and can be fixed before the delivery of
the software.
• Improves Quality of Software: Software Testing uncovers the defects in the
software, and fixing them improves the quality of the software.
• Increased Customer Satisfaction: Software testing ensures reliability, security,
and high performance which results in saving time, costs, and customer
satisfaction.
• Helps with Scalability: Software testing type non-functional testing helps to
identify the scalability issues and the point where an application might stop
working.
• Saves Time and Money: After the application is launched it will be very
difficult to trace and resolve the issues, as performing this activity will incur
more costs and time. Thus, it is better to conduct software testing at regular
intervals during software development.

9
Tanvi Nerurkar
UNIT III Software Engineering

Best Practices for Software Testing

Here are the best practices for software testing that help to verify the testing process:
• Continuous Testing: Project teams test each build as it becomes available thus
it enables software to be validated in real environments earlier in the
development cycle, reducing risks and improving the functionality and design.
• Involve Users: It is very important for the developers to involve users in the
process and open-ended questions about the functionality required in the
application. This will help to develop and test the software from the customer's
perspective.
• Divide Tests into Smaller Parts: Dividing tests into smaller fractions save time
and other resources in environments where frequent testing needs to be
conducted. This also helps teams to make better analyses of the tests and the test
results.
• Metrics and Reporting: Reporting enables the team members to share goals
and test results. Advanced tools integrate the project metrics and present an
integrated report in the dashboard that can be easily reviewed by the team
members to see the overall health of the project.
• Don't Skip Regression Testing: Regression testing is one of the most important
steps as it encourages the validation of the application. Thus, it should not be
skipped.
• Programmers Should Avoid Writing Tests: Test cases are usually written
before the start of the coding phase so it is considered a best practice for
programmers to avoid writing test cases as they can be biased towards their code
and the application.
• Service Virtualization: Service virtualization simulates the systems and
services that are not yet developed or are missing. Thus, enabling teams to
reduce dependency and start the testing process sooner. They can modify, and
reuse the configuration to test different scenarios without having to alter the
original environment.
Benefits of Software Testing
Here are some Benefits which give Software Testing:
• Product Quality: Testing ensures the delivery of a high-quality product as the
errors are discovered and fixed early in the development cycle.
• Customer Satisfaction: Software testing aims to detect the errors or
vulnerabilities in the software early in the development phase so that the
detected bugs can be fixed before the delivery of the product. Usability testing is
a type of software testing that checks the application for how easily usable it is
for the users to use the application.
• Cost-Effective: Testing any project on time helps to save money and time for
the long term. If the bugs are caught in the early phases of software testing, it
costs less to fix those errors.
• Security: Security testing is a type of software testing that is focused on testing
the application for security vulnerabilities from internal or external sources.

10
Tanvi Nerurkar
UNIT III Software Engineering

➢ Test-Case design

● Test case design is a part of system and component testing where you design the test cases
(inputs and predicted outputs) that test the system.
● The goal of the test case design process is to create a set of test cases that are effective in
discovering program defects and showing that the system meets its requirements.
● To design a test case, you select a feature of the system or component that you are testing.
● You then select a set of inputs that execute that feature, document the expected outputs or
output ranges and, where possible, design an automated check that tests that the actual and
expected outputs are the same.

● There are various approaches that you can take to test case design:

1. Requirements-based testing where test cases are designed to test the system requirements. This
is mostly used at the system-testing stage as system requirements are usually implemented by
several components. For each requirement, you identify test cases that can demonstrate that the
system meets that requirement.

2. Partition testing where you identify input and output partitions and design tests so that the
system executes inputs from all partitions and generates outputs in all partitions. Partitions are
groups of data that have common characteristics such as all negative numbers, all names less than
30 characters, all events arising from choosing items on a menu, and so on.

3. Structural testing where you use knowledge of the program’s structure to design tests that
exercise all parts of the program. Essentially, when testing a program, you should try to execute
each statement at least once. Structural testing helps identify test cases that can make this
possible.

● In general, when designing test cases, you should start with the highest-level tests from the
requirements then progressively add more detailed tests using partition and structural testing.

1 Requirements-based testing:
A general principle of requirements engineering is that requirements should be testable. That is,
the requirement should be written in such a way that a test can be designed so that an observer
can check that the requirement has been satisfied. Requirements-based testing, therefore, is a
systematic approach to test case design where you consider each requirement and derive a set of
tests for it.

Requirements-based testing is validation rather than defect testing demonstrates that the system
has properly implemented its requirements.

Process of Requirements based Testing:

● Defining Test Completion Criteria –

● Testing is completed only when all the functional and non-functional testing is complete.
● Design Test Cases –
● A Test case has five parameters namely the initial state or precondition, data setup, the inputs,
expected outcomes and actual outcomes.
● Execute Tests –
● Execute the test cases against the system under test and document the results.
● Verify Test Results –
● Verify if the expected and actual results match each other.

11
Tanvi Nerurkar
UNIT III Software Engineering

● Verify Test Coverage –

● Verify if the tests cover both functional and non-functional aspects of the requirement.
● Track and Manage Defects –
● Any defects detected during the testing process goes through the defect life cycle and are
tracked to resolution. Defect Statistics are maintained which will give us the overall status of the
project.

Requirements Testing process:

● Testing must be carried out in a timely manner.
● Testing process should add value to the software life cycle, hence it needs to be effective.
● Testing the system exhaustively is impossible hence the testing process needs to be efficient as
well.
● Testing must provide the overall status of the project; hence it should be manageable.
○ The Requirements based testing process starts at the very early phase of the software
development, as correcting issues/errors is easier at this phase.
○ It begins at the requirements phase as the chances of occurrence of bugs have its roots
here.
○ It aims at quality improvement of requirements. Insufficient requirements leads to failed
projects.

2 Partition testing:
One systematic approach to test case design is based on identifying all partitions for a system or
component.

Test cases are designed so that the inputs or outputs lie within these partitions. Partition testing
can be used to design test cases for both systems and components. Input equivalence partitions
are sets of data where all of the set members should be processed in an equivalent way.

Output equivalence partitions are program outputs that have common characteristics, so they can
be considered as a distinct class.

You also identify partitions where the inputs are outside the other partitions that you have chosen.

These test whether the program handles invalid input correctly. Valid and invalid inputs also form
equivalence partitions.

You identify partitions by using the program specification or user documentation and, from
experience, where you predict the classes of input value that are likely to detect errors.

For example, say a program specification states that the program accepts 4 to 8 inputs that are
five-digit integers greater than 10,000.

12
Tanvi Nerurkar
UNIT III Software Engineering

3. Structural testing:
● Structural testing is an approach to test case design where the tests are derived from knowledge
of the software’s structure and implementation.
● This approach is sometimes called ‘white-box’, ‘glass-box’ testing, or ‘’clear-box’ test-
● ing to distinguish it from black-box testing.
● Understanding the algorithm used in a component can help you identify further partitions and
test cases.

● Structural testing, also known as glass box testing or white box testing is an approach where the
tests are derived from the knowledge of the software's structure or internal implementation.

● Structural Testing Techniques:

○ Statement Coverage - This technique is aimed at exercising all programming statements with
minimal tests.
○ Branch Coverage - This technique is running a series of tests to ensure that all branches are
tested at least once.
○ Path Coverage - This technique corresponds to testing all possible paths which means that each
statement and branch are covered.

4. Path testing:

● Path Testing is a structural testing method based on the source code or algorithm and NOT
based on the specifications.

13
Tanvi Nerurkar
UNIT III Software Engineering

● It can be applied at different levels of granularity.

● Path Testing Assumptions:

○ The Specifications are Accurate
○ The Data is defined and accessed properly
○ There are no defects that exist in the system other than those that affect control flow

● Path Testing Techniques:

○ Control Flow Graph (CFG) - The Program is converted into Flow graphs by representing the
code into nodes, regions and edges.
○ Decision to Decision path (D-D) - The CFG can be broken into various Decision to Decision
paths and then collapsed into individual nodes.
○ Independent (basis) paths - Independent path is a path through a DD-path graph which cannot
be reproduced from other paths by other methods.

1. Control Flow Graph –

A control flow graph (or simply, flow graph) is a directed graph which represents the control
structure of a program or module.
A control flow graph (V, E) has V number of nodes/vertices and E number of edges in it. A
control graph can also have :

Junction Node – a node with more than one arrow entering it.
Decision Node – a node with more than one arrow leaving it.
Region – area bounded by edges and nodes (area outside the graph is also counted as a region.).

14
Tanvi Nerurkar
UNIT III Software Engineering

15
Tanvi Nerurkar
UNIT III Software Engineering

16
Tanvi Nerurkar
UNIT III Software Engineering

Independent Paths :

An independent path in the control flow graph is the one which introduces at least one new edge
that has not been traversed before the path is defined. The cyclomatic complexity gives the
number of independent paths present in a flow graph. This is because the cyclomatic complexity

17
Tanvi Nerurkar
UNIT III Software Engineering

is used as an upper-bound for the number of tests that should be executed in order to make sure
that all the statements in the program have been executed at least once. Consider first graph given
above here the independent paths would be 2 because number of independent paths is equal to the
cyclomatic complexity.

So, the independent paths in above first given graph :

● Path 1: A -> B
● Path 2: C -> D

Note – Independent paths are not unique. In other words, if for a graph the cyclomatic complexity
comes out be N, then there is a possibility of obtaining two different sets of paths which are
independent in nature.

Design Test Cases :

Finally, after obtaining the independent paths, test cases can be designed where each test case
represents one or more independent paths.

Advantages :

Basis Path Testing can be applicable in the following cases:

1. More Coverage – Basis path testing provides the best code coverage as it aims to achieve
maximum logic coverage instead of maximum path coverage. This results in an overall thorough
testing of the code.

2. Maintenance Testing – When a software is modified, it is still necessary to test the changes
made in the software which as a result, requires path testing.

3. Unit Testing – When a developer writes the code, he or she tests the structure of the program
or module themselves first. This is why basis path testing requires enough knowledge about the
structure of the code.

4. Integration Testing – When one module calls other modules, there are high chances of
Interface errors. In order to avoid the case of such errors, path testing is performed to test all the
paths on the interfaces of the modules.

5. Testing Effort – Since the basis path testing technique takes into account the complexity of the
software (i.e., program or module) while computing the cyclomatic complexity, therefore it is
intuitive to note that testing effort in case of basis path testing is directly proportional to the
complexity of the software or program.

➢ References:

• [Link]
• Software Engineering by Ian Somerville

18
Tanvi Nerurkar