CHAPTER 1: BASICS OF SOFTWARE TESTING AND TESTING METHODS

(14 marks)
Objectives:
Understand the importance of Quality Software
1.1 Definition of Software Testing, Objectives of Testing
1.2 Failure, Error, Fault, Defect, Bug Terminology
1.3 Test Case, When to Start and Stop Testing of Software (Entry and Exit Criteria)
1.4 Verification and Validation, V Model,Quality Assurance, Quality Control
1.5 Methods of testing : static and dynamic testing
1.6 The box approach :white box testing: inspections , walkthroughs, Technical reviews ,
Functional testing, code coverage testing, code complexity testing

1.1 Definition of Software Testing

Software testing is a method of assessing the functionality of a software program.
OR
Software testing is the process of validating and verifying that a software program or application
or product, meets the business and technical requirements that guided its design and development.
“Testing is the process of executing a program or software with the intention of finding
errors.” – Myers
“Testing can show the presence of bugs but never their absence.” – Dijkstra
“The process of exercising or evaluating a system or software by manual or automated means
to verify that satisfies requirements or to identify differences between expected and actual
results” - IEEE
Effective software testing delivers quality software products satisfying user’s requirements, needs
and expectations.
Testing Principles:
1. Testing shows presence of defects: Testing can show the defects are present, but cannot prove
that there are no defects. Even after testing the application or product thoroughly we cannot say
that the product is 100% defect free.
2. Exhaustive testing is impossible: Testing everything including all combinations of inputs and
preconditions is not possible. So, instead of doing the exhaustive testing we can use risks and
priorities to focus testing efforts.
3. Early testing: In the software development life cycle testing activities should start as early as
possible and should be focused on defined objectives.
4. Defect clustering: A small number of modules contains most of the defects discovered during
pre-release testing or shows the most operational failures.
5. Pesticide paradox: If the same kinds of tests are repeated again and again, eventually the same
set of test cases will no longer be able to find any new bugs. To overcome this ―Pesticide
Paradox‖, it is really very important to review the test cases regularly and new and different tests
need to be written to exercise different parts of the software or system to potentially find more
defects.
6. Testing is basically context dependent: Different kinds of sites are tested differently. For
example, safety – critical software is tested differently from an e-commerce site.
7. Absence – of – errors fallacy: If the system built is unusable and does not fulfill the user‘s
needs and expectations then finding and fixing defects does not help.
Role of Testing

Software Testing chapter 1 Vaishali Rane Page 1

 Immediate goals
Bug Discovery
Bug prevention

Long term goals

Reliability
Software Testing Quality
Customer Satisfaction
Risk management

Post Implementation goals

Reduced maintenance cost

Improved testing process

Software testing is done for many reasons

 Check reliability of the software
 Ensure software does not contain bug which becomes a reason for failure
 Software is in accordance to its specification and meets its requirements.
 User is capable of using the software
 Check software works with other software and hardware it needs to work with
 Improve quality of software by removing maximum possible errors or defects
from it.

1.3 Objectives of software testing

1. Finding error: Finding defects which may get created by the programmer while developing
the software.
2. Quality improvement: Gaining confidence in and providing information about the level of
quality.
3. Creating good test cases: Good test case is one that has a high probability of finding
undiscovered error.
4. Meets users requirements : To make sure that the end result meets the business and user
requirements.
5. Satisfying Requirements: To ensure that it satisfies the BRS that is Business Requirement
Specification and SRS that is System Requirement Specifications.
6. To gain the confidence of the customers by providing them a quality product

1.2 Failure, Error, Fault, Defect, Bug Terminology

Fault: It is a condition that causes the software to fail to perform its required function.
Error: Refers to difference between Actual Output and Expected output.
Failure: It is the inability of a system or component to perform required function according to its
specification.
IEEE Definitions
Failure: External behavior is incorrect
Fault: Discrepancy in code that causes a failure.
Error: Human mistake that caused fault

Software Testing chapter 1 Vaishali Rane Page 2

Note: Error is terminology of Developer. Bug is terminology of Tester
The various terms related to software failure with respect to the area of application are listed as
Defect, Variance, Fault, Failure, Problem, Inconsistency, Error, Feature, Incident, Bug, and
Anomaly. Failure: the inability of a system or component to perform its required functions within
specified performance requirements.
Fault: An incorrect step, process, or data definition in a computer program.
Error: A human action that produces an incorrect result.
An error can be a grammatical error in one or more of the code lines, or a logical error in
carrying out one or more of the client‘s requirements.
Not all software errors become software faults. in some cases, the software error can cause
improper functioning of the software. In many other cases, erroneous code lines will not affect the
functionality of the software as a whole.
A failure is said to occur whenever the external behaviour of a system does not conform to that
prescribed in the system specification. A software fault becomes a software failure only when it is
―activated.
The various terms related to software failure with respect to the area of application are listed as
Defect, Variance, Fault, Failure, Problem, Inconsistency, Error, Feature, Incident, Bug, and
Anomaly.
Problem, error, and bug are probably the most generic terms used.
Anomaly, incident, and variance don‘t sound quite so negative and infer more unintended
operation than an all-out failure.
Fault, failure, and defect tend to imply a condition that‘s really severe, maybe even dangerous. It
doesn‘t sound right to call an incorrectly coloured icon a fault. These words also tend to imply
blame: ―It‘s his fault that the software failed.
As all the words sound the same they are distinguished based on the severity and the area in which
the software failure has occurred.
When we run a program the error that we get during execution is termed on the basis of runtime
error, compile time error, computational error, and assignment error.
The error can be removed by debugging, if not resolved leads to a problem and if the problem
becomes large leads to software failure.
A bug can be defined as the initiation of error or a problem due to which fault, failure,
incident or an anomaly occurs.
The program to find the factorial of a number given below lists few errors problem and failure in
a program.

For example
#include<stdio.h>
void main()
{
int i , fact, n;
printf(“enter the number “);
scanf(“%d”,&n);
for(i =1 ;i <=n;i++)
fact = fact * i;
printf (“the factorial of a number is %d”, fact);
}
As in line number 4 the fact is not initialized to 1, so it takes garbage value and gives a wrong
output, this is an example of a bug. If fact is initialized to zero (fact = 0) than the output will be
zero as anything multiplied by zero will give the output as zero. This is a bug which can be removed
by initializing fact = 1 during initializing. As the fact is declared as integer, for the

Software Testing chapter 1 Vaishali Rane Page 3

number till 7! will work perfectly. When the number entered is 8, the output is garbage value as
the integer limit is from – 32767 to +32768, so in declaration change the initialization to long int
fact

1.3 Test Case

IEEE defines test case as “a set of inputs values, execution preconditions and expected
outcomes developed for a particular objects”
Test case is a well-documented procedure designed to test the functionality of the feature in the
system.
A test case is a set of conditions or variables under which a tester will determine whether a system
under test satisfies requirements or works correctly. For designing the test case, it needs to provide
set of inputs and its corresponding expected outputs.
Parameters:
1. Test case ID: is the identification number given to each test case.
2. Purpose: defines why the case is being designed.
3. Precondition: The prerequisite for running in the system.
4. Input: Actual inputs must be provided, instead of general inputs.
5. Expected outputs: which should be produced when there is no failure?
6. Actual outputs: What is the actual output when the code is executed.
7. Status: If Expected and actual result is same status is Pass otherwise it is Fail.
Two approaches to testing Software -
Test to pass: A test case producing expected result is called test to pass, its capabilities are not
pushed.
Test to fail/ error forcing: A test case that doesn’t produce expected result is called test to fail.
Sole purpose is to break the software.

How to write a good test case?

Test case is an important document from validation of system perspective. These must be written
at each stage of validation from unit test till acceptance testing.

Basic principles of writing a good test case are:

 Test case must be testable.
 Tester should know what is to be done when to wait for system to do it.
 Inform tester each transaction displayed/ replied by the system on screen at each step.
And wait for user response.
 Use simple conversational language for writing test case, which improves clarity and
avoid communication losses.
 Use consistent names of fields must be used in place of generic names. Any change in
field name must be incorporated in test cases.
 Tester should be aware of windows basics.
 Order of the test cases must follow business scenario. Avoid time wastage,
Common Mistakes in writing test cases:
 Making test cases too long and combining two or more test cases in single test should
be avoided
 Incomplete, incorrect, and incoherent test cases can create confusion and frustrate
testers.
 Steps should be made very clear in test case steps.

Software Testing chapter 1 Vaishali Rane Page 4

  Test case changes must be updated in software user interface.
 Define pass/fail criteria correctly, i.e. test are successful or not, there is a defect or not?

Software Testing chapter 1 Vaishali Rane Page 5

 Attribute for each test case are:
a. Test pre-condition b. Priority
c. Test sequence d. Test data
e. Traceability to test scenario f. Steps to reproduce
g. Test case name and number h. Expected results
i. Types of testing j. Actual results
k. Objectives l. Test result
m. Valid/Invalid conditions n. Remarks
1.5 When to Start and Stop Testing of Software (Entry and Exit Criteria)
Process model is a way to represent any given phase of software development that prevent and
minimize the delay between defect injection and defect detection/correction. Each phase of
software project is characterised by-
 Entry criteria, specifies when that phase can be started also included the inputs for the
phase.
 Tasks or steps that need to be carried out in that phase, along with measurements that
characterize the tasks.
 Verification, which specifies methods of checking that tasks have been carried out correctly.
 Exit criteria, which stipulate the conditions under which one can consider the phases as
done. Also included are the outputs for only the phase.

This is known as the Entry Task Verification exit or EVTX model which offers several
advantages for effective verification and validation.
 Clear entry criteria make sure that a given phase does not start prematurely.
 The verification for each phase helps to prevent defects. At least defects can be
minimized.
 Documentation of the detailed tasks that comprise each phase reduces the ambiguity
in interpretations and thus minimises the variation.

Entry criteria may include

 Test environment such as lab, hardware , software and system administration support
should be ready
 Proper test data is available
 All personnel involved in testing must be trained in the tools to be used in testing process
 All standard hardware platform and software tools must have been successfully
installed, configured and functioning properly
 All the necessary documentation, design and requirement information should be
available that will allow the testers to operate the system and judge the correct behaviour.

Exit criteria may include

 Completion of all test case execution.
 All requirements covered. Completion of functional and code coverage to a certain
point.

Software Testing chapter 1 Vaishali Rane Page 6

  All severe bugs are resolved. Bug rate falls below a certain level.
 Management decision. - budget is spent - schedule is achieved

Exit Criteria:
Entry Criteria:
Complete traceability between design and SRS
Approval of SRS by customer
Steps: Development team ready to start
Evolve an architecture programming
Perform high level design
Perform detailed/ low level
design
Input: Write program spaces
Approved SRS Output:
Architecture documents
Design documents
Program specifications

GOALS OF SOFTWARE TESTING

IMPORTANT GOALS OF SOFTWARE TESTING
1. Always Identifying the bugs as early as possible.
2. Preventing the bugs in a project and product.
3. Check whether the customer requirements criterion is met or not.
4. And finally main goal of testing to measure the quality of the product and project.

1. Short-term or immediate goals of software testing: -

These goals are the immediate results after performing testing. These goals even may be set in
the individual phases of SDLC. Some of them are completely discussed below:

a) Bug discovery: The immediate goal about software testing is to find errors at any stage of
software development. More the bugs discovered at early stage, better will be the success rate
about software testing.

b) Bug prevention: It is the consequent action of bug discovery. From the behavior and
analysis of bugs discovered, everyone in the software development team gets to learn how to
code so that bugs discovered should not be repeated in later stages or future projects.

Though errors always cannot be prevented to zero, they can be minimized. In this sense
prevention of a bug is a superior goal of testing.

2. Long-term goals of software testing: -

These goals affect the product quality in the deep run, when one cycle of the SDLC is over.
Some of them are completely discussed below:

a) Quality: Since software is also a product, so its quality is primary from the user’s point of
view. Thorough testing ensures superior quality.
Quality depends on various factors, such as correctness, integrity, efficiency, and reliability. So
to achieve quality you have to achieve all the above mentioned factors of Quality.
b) Customer satisfaction: From the user’s perspective, the prime goal of software testing is
customer satisfaction only. If we want the client and customer to be satisfied with the software
product, then testing should be complete and thorough.

Software Testing chapter 1 Vaishali Rane Page 7

c) Reliability : is the confidence on software that it will never fail while working. This also
means to gain the confidence of the customers by providing them, a good quality product.
A complete testing process achieves reliability, reliability enhances the quality, and quality in
turn, increases the customer satisfaction.
4. Risk management : it is done to reduce the failure rate of the product and manage risk in
different situations.

3. Post-implementation goals of software testing: - These goals are become essential after the
product is released. Some of them are completely discussed below:
a) Reduced maintenance cost: The maintenance cost about any software product is not its
physical cost, as effective software does not wear out. The only maintenance cost in a software
product is its failure due to errors.
Post- release errors are always costlier to fix, as they are difficult to detect. Thus, if testing has
been done rigorously and effectively, then the chances about failure are minimized and as a
result of this maintenance cost is reduced.
b) Improved software testing process: A testing process for one project may not be
blooming successful and there may be area for improvement. Therefore, the bug history and
post-implementation results can be analyzed to find out snags in the present testing process,
which can be determine in future projects.
Thus, the long-term post-implementation goal is to improve the testing process for future
projects.

Skills for Software Tester

1. He should possess Communication & listening Skill
2. He should be a Troubleshooter ,explorer and relentless
3. He should have Knowledge of SDLC and Domain knowledge
4. He should have knowledge of Risk assessment and management
5. He should be Innovative in testing skill
6. He should be Tactful, diplomatic and persuasive
7. He should posses Knowledge of selecting & using testing tools

1.4 Quality Assurance, Quality Control, Verification and Validation, V Model

Verification
1. It makes sure that the product is designed to deliver all functionality to the customer.
2. Verification is done at the starting of the development process. It includes reviews and
meetings, walkthroughs, inspection, etc. to evaluate documents, plans, code, requirements and
specifications.
3. It answers the questions like: Am I building the product right?
4. Am I accessing the data right (in the right place; in the right way).
5. It is a Low level activity
6. Performed during development on key art facts, like walkthroughs, reviews and inspections,
mentor feedback, training, checklists and standards.
7. Demonstration of consistency, completeness, and correctness of the software at each stage and
between each stage of the development life cycle.
Advantages of Software Verification :
1. Verification helps in lowering down the count of the defect in the later stages of
development.
2. Verifying the product at the starting phase of the development will help in understanding
the product in a better way.
3. It reduces the chances of failures in the software application or product.

Software Testing chapter 1 Vaishali Rane Page 8

 4. It helps in building the product as per the customer specifications and needs.

Disadvantages of software verification

1. It is a long and time consuming process.
2. Verification does not mean that the errors are not present.
Validation
1. Determining if the system complies with the requirements and performs functions for which it
is intended and meets the organization‘s goals and user needs.
2. Validation is done at the end of the development process and takes place after verifications are
completed.
3. It answers the question like: Am I building the right product? Am I accessing the right data (in
terms of the data required to satisfy the requirement).
4. It is a High level activity.
5. Performed after a work product is produced against established criteria ensuring that the
product integrates correctly into the environment.
6. Determination of correctness of the final software product by a development project with
respect to the user needs and requirements.

Advantages of Validation:
1. During verification if some defects are missed then during validation process it can be
caught as failures.
2. If during verification some specification is misunderstood and development had happened
then during validation process while executing that functionality the difference between
the actual result and expected result can be understood.
3. Validation is done during testing like feature testing, integration testing, system testing,
load testing, compatibility testing, stress testing, etc.
4. Validation helps in building the right product as per the customer’s requirement and helps
in satisfying their needs.
Disadvantages of Validation:
1. It may result in redundant testing as much of the tests are carried out earlier.
2. Validation also does not show that errors are 100 % gone.
DIFFERENCES : VERIFICATION AND VALIDATION

Verification Validation

Are we building the system right? Are we building the right system?

Verification is the process of evaluating products of Validation is the process of evaluating software at
a development phase to find out whether they meet the end of the development process to determine
the specified requirements. whether software meets the customer expectations
and requirements.

The objective of Verification is to make sure that The objective of Validation is to make sure that the
the product being develop is as per the requirements product actually meet up the user’s requirements, and
and design specifications. check whether the specifications were correct in the
first place.

Following activities are involved in Verification: Following activities are involved in Validation:

Software Testing chapter 1 Vaishali Rane Page 9

Reviews, Meetings and Inspections. Testing like black box testing, white box testing, gray
box testing etc.

Verification is carried out by QA team to check Validation is carried out by testing team.
whether implementation software is as per
specification document or not.

Execution of code is not comes under Verification. Execution of code is comes under Validation.

Verification process explains whether the outputs Validation process describes whether the software is
are according to inputs or not. accepted by the user or not.

Verification is carried out before the Validation. Validation activity is carried out just after the
Verification.

Following items are evaluated during Verification: Following item is evaluated during Validation:
Plans, Requirement Specifications, Design Actual product or Software under test.
Specifications, Code, Test Cases etc,

Cost of errors caught in Verification is less than Cost of errors caught in Validation is more than
errors found in Validation. errors found in Verification.

It is basically manually checking the of documents It is basically checking of developed program based
and files like requirement specifications etc. on the requirement specifications documents & files.

V-model with labelled diagram.

V- model means Verification and Validation model. Just like the waterfall model, the V-Shaped
life cycle is a sequential path of execution of processes. Each phase must be completed before the
next phase begins. Under V-model, the corresponding testing phase of the development phase is
planned in parallel. So there is verification on one side of V & validation phase on the other side
of V.

Verification Phase:
1. Overall Business Requirement: In this first phase of the development cycle, the product
requirements are understood from customer perspective. This phase involves detailed
communication with the customer to understand his expectations and exact requirements. The
acceptance test design planning is done at this stage as business requirements can be used as an
input for acceptance testing.
2. Software Requirement: Once the product requirements are clearly known, the system can be
designed. The system design comprises of understanding & detailing the complete hardware,
software & communication set up for the product under development. System test plan is designed
based on system design. Doing this at earlier stage leaves more time for actual test execution later.
3. High level design: High level specification are understood & designed in this phase. Usually
more than one technical approach is proposed & based on the technical & financial feasibility, the
final decision is taken. System design is broken down further into modules taking up different
functionality.

Software Testing chapter 1 Vaishali Rane Page 10

4. Low level design: In this phase the detailed integral design for all the system modules is
specified. It is important that the design is compatible with the other modules in the system & other
external system. Components tests can be designed at this stage based on the internal module
design,
5. Coding: The actual coding of the system modules designed in the design phase is taken up in
the coding phase. The base suitable programming language is decided base on requirements.
Coding is done based on the coding guidelines & standards.

Validation:
1. Unit Testing: Unit testing designed in coding are executed on the code during this validation
phase. This helps to eliminate bugs at an early stage.
2. Components testing: This is associated with module design helps to eliminate defects in
individual modules.
3. Integration Testing: It is associated with high level design phase & it is performed to test the
coexistence & communication of the internal modules within the system
4. System Testing: It is associated with system design phase. It checks the entire system
functionality & the communication of the system under development with external systems. Most
of the software & hardware compatibility issues can be uncovered using system test execution.
5. Acceptance Testing: It is associated with overall & involves testing the product in user
environment. These tests uncover the compatibility issues with the other systems available in the
user environment. It also uncovers the non-functional issues such as load & performance defects
in the actual user environment.

Advantages of V-model:
 Simple and easy to use.
 Testing activities like planning, test designing happens well before coding. This saves a
lot of time. Hence higher chance of success over the waterfall model.
 Proactive defect tracking – that is defects are found at early stage.
 Avoids the downward flow of the defects.
 Works well for small projects where requirements are easily understood

Disadvantages of V-model:

Software Testing chapter 1 Vaishali Rane Page 11

  Very rigid and least flexible.
 Software is developed during the implementation phase, so no early prototypes of the
software are produced.
 If any changes happen in midway, then the test documents along with requirement
documents has to be updated.
When to use the V-model:
 The V-shaped model should be used for small to medium sized projects where
requirements are clearly defined and fixed. 
 The V-Shaped model should be chosen when ample technical resources are available
with needed technical expertise.

High confidence of customer is required for choosing the V-Shaped model approach. Since, no
prototypes are produced, there is a very high risk involved in meeting customer expectations.

Quality assurance and Quality control

Quality refers to “the degree of which a system or component or software or process meets
expected or specified requirements” IEEE DEFINITION.

Above fig. shows the relationship between quality control, quality assurance and software
testing.
1. Quality assurance :
Quality Assurance is the set of activities that ensures , the development and maintenance process
is adequate to ensure whether the system will meet its objectives or goals.
2. Quality control :
A set of activities designed to evaluate a developed work product.
3. Testing :
Process of execution of a system with the intention of finding the [Link] assurance
- Process of determining whether product meets customers requirements
- QA activities are process oriented . They measure the process, identify deficiencies and
suggest improvements.
- Responsible for managing quality.

The various objectives of SQA are as follows:

1. Quality management approach.

2. Measurement and reporting mechanisms.

Software Testing chapter 1 Vaishali Rane Page 12

 3. Effective software-engineering technology.
4. A procedure to assure compliance with software-development standards where
applicable.
5. A multi-testing strategy is drawn.
6. Formal technical reviews that are applied throughout the software process.

The major goals of SQA are as follows:

1. SQA activities are planned.
2. Non-compliance issues that cannot be resolved within the software project are addressed
by senior management.
3. Adherence of software products and activities to the applicable standards, procedures,
and requirements is verified objectively.
4. Affected groups and individuals are informed of SQA activities and results.
SQA has 2 main aspects :
1. Quality control and
2. Quality assurance
The process of software quality control is also governed by SQA.
QUALITY CONTROL
SOFTWARE QUALITY CONTROL (SQC) is a set of activities for ensuring quality in software
products.
Software Quality Control is limited to the Review/Testing phases of the Software Development
Life Cycle and the goal is to ensure that the products meet specifications/requirements.

SQC Activities
It includes the following activities:
Reviews
1. Requirement Review
2. Design Review
3. Code Review
4. Deployment Plan Review
5. Test Plan Review
6. Test Cases Review
Testing
1. Unit Testing
2. Integration Testing
3. System Testing
4. Acceptance Testing
The process of Software Quality Control (SQC) is governed by Software Quality Assurance
(SQA). While SQA is oriented towards prevention, SQC is oriented towards detection.
Difference between quality assurance and quality control :

Software Testing chapter 1 Vaishali Rane Page 13

 Quality Assurance Quality Control

Ensures quality in processes by which Ensures quality in products by focus on identifying defects
product are developed. in actual products
Identifies weaknesses in processes and
Identifies defects for the primary purpose of correcting
improves them. It is a proactive quality
defects. It is a reactive quality process
process
QC is the responsibility of the specific team that tests
QA is the responsibility of the entire team.
fodefects.

Prevents the introduction of issues or

Detects, reports and corrects defects
defects

QA personnel should not perform quality

QC personnel may perform quality assurance tasks if and
control unless doing it to validate quality
when required.
control is working.

QA is a managerial tool QC is a corrective tool

Software Testing chapter 1 Vaishali Rane Page 14

 QC is the responsibility of the specific team that tests
QA is the responsibility of the entire team.
fodefects.
Prevents the introduction of issues or
Detects, reports and corrects defects
defects
QA personnel should not perform quality
QC personnel may perform quality assurance tasks if and
control unless doing it to validate quality
when required.
control is working.
QA is a managerial tool QC is a corrective tool

1.5 METHODS OF TESTING : STATIC AND DYNAMIC TESTING

Static testing and dynamic testing are important testing methods available for developers and
testers in Software Development lifecycle. These are software testing techniques which the
organisation must choose carefully which to implement on the software application. In order to
get the most out of each type of testing, and choose the right tools for a given situation, it’s
crucial to understand the benefits and limitations of each type of testing.

What is Static Testing?

Static Testing is type of testing in which the code is not executed. It can be done manually or by
a set of tools. This type of testing checks the code, requirement documents and design documents
and puts review comments on the work document. When the software is non –operational and
inactive, we perform security testing to analyse the software in non-runtime environment. With
static testing, we try to find out the errors, code flaws and potentially malicious code in the
software application. It starts earlier in development life cycle and hence it is also called
verification testing. Static testing can be done on work documents like requirement
specifications, design documents, source code, test plans, test scripts and test cases, web page
content.
The Static test techniques include:
Inspection: Here the main purpose is to find defects. Code walkthroughs are conducted by
moderator. It is a formal type of review where a checklist is prepared to review the work
documents.
Walkthrough: In this type of technique a meeting is lead by author to explain the product.
Participants can ask questions and a scribe is assigned to make notes.
Technical reviews: In this type of static testing a technical round of review is conducted to
check if the code is made according to technical specifications and standards. Generally the test
plans, test strategy and test scripts are reviewed here.
Informal reviews: Static testing technique in which the document is reviewed informally and
informal comments are provided.

What is Dynamic Testing?

Dynamic testing is done when the code is in operation mode. Dynamic testing is performed in
runtime environment. When the code being executed is input with a value, the result or the
output of the code is checked and compared with the expected output. With this we can observe
the functional behaviour of the software, monitor the system memory, CPU response time,
performance of the system. Dynamic testing is also known as validation testing , evaluating the
finished product.
Dynamic testing is of two types:
Functional Testing and Non functional testing.
Types of Dynamic Testing techniques are as follows:
Software Testing chapter 1 Vaishali Rane Page 15
Unit Testing: Testing of individual modules by developers.. The source code is tested in it.
Integration Testing: Testing the interface between different modules then they are joined..
System Testing: Testing performed on the system as a whole.
Acceptance Testing: Testing done from user point of view at user’s end.
However, both Static Testing and Dynamic Testing are important for the software application.
There are number of strengths and weaknesses associated with both types of testing which
should be considered while implementing these testing on code:
Difference between Static Testing and Dynamic Testing
Static Testing Dynamic Testing

1. Static Testing is white box testing which is done at 1. Dynamic Testing on the other hand is done at the
early stage if development life cycle. It is more cost later stage of development lifecycle.
effective than dynamic testing

2. Static testing has more statement coverage than 2. Dynamic Testing has less statement stage because
dynamic testing in shorter time it is covers limited area of code

3. It is done before code deployment 3. It is done after code deployment

4. It is performed in Verification Stage 4. It is done in Validation Stage

5. This type of testing is done without the execution 5. This type of execution is done with the execution
of code. of code.

6. Static testing gives assessment of code as well as 6. Dynamic Testing gives bottlenecks of the
documentation. software system.

7. In Static Testing techniques a checklist is 7. In Dynamic Testing technique the test cases are
prepared for testing process executed.

8. Static Testing Methods include Walkthroughs, 8. Dynamic testing involves functional and
code review. nonfunctional testing

THE BOX APPROACH:

WHITE BOX TESTING:
WHITE BOX TESTING (also known as Clear Box Testing, Open Box Testing, Glass Box
Testing, Transparent Box Testing, Code-Based Testing or Structural Testing) is a software
testing method in which the internal structure/design/implementation of the item being tested is
known to the tester.
The tester chooses inputs to exercise paths through the code and determines the appropriate
outputs.
Programming know-how and the implementation knowledge is essential.
White box testing is testing beyond the user interface and into the details of a system.
This method is named so because the software program, in the eyes of the tester, is like a
white/transparent box; inside which one clearly sees.
Software Testing chapter 1 Vaishali Rane Page 16
 Advantages of White Box testing
1. Testing can commence even before the GUI is ready.
2. As internal functionality is considered, all the possible conditions are considered and test
cases are generated. Hence all the functionalities are being tested.
3. It identifies the specific procedure accuracy within the application.
4. It minutely verifies whether the program can be successfully executed with other parts of
the application.
5. It identifies error in the hidden code and thus makes debugging process swift.
6. It removes extra lines of code which are not required in the program thereby optimizing
the program and increases the efficiency.
7. As the internal coding of the application is considered while preparing test cases, it
becomes very easy to identify the input and the expected output data.
8. It helps in evaluating all the loops and paths.
9. It can provide stability and usability of the test cases.
10. Thoroughness achieved in white box testing is far more than black box testing.
Disadvantages of White Box testing
1. As the internal code of the application has to be considered while preparing the
test cases, skilled testers are required who have knowledge of programming also. Hence
the cost of the resources is high.
2. It is not possible for the tester to look into every bit of the code and identify the
hidden errors. This may result in failure of the application.
3. Sometimes a change in the code may be required and thus all the scenarios may
need to be tested again.
4. White box testing is an exhaustive method.
5. It takes time to tester to develop the test cases.
6. Test cases are a waste if changes in the implementation code are done frequently.
7. If the application is large then complete testing through white box techniques is
not feasible.
White box testing can be classified as shown :
1. Static white box testing : type of testing where the program code is tested without
running it.
2. Structural testing : it involves testing a running program. Internal logic is tested and test
cases are exerted.

White box testing can be classified as below :

White box testing

Structural testing
Static testing
Statement/line
Inspection Code functional testing
Branch coverage
Walkthrough Code coverage testing
Static
Condition coverage
Technical review Code complexity testing
Function coverage
Software Testing chapter 1 Vaishali Rane Page 16
Static Testing :

Formal Reviews
A formal review is the process under which static white-box testing is performed.
A formal review can range from a simple meeting between two programmers to a detailed,
rigorous inspection of the code.
There are four essential elements to a formal review:
1. Identify Problems. The goal of the review is to find problems with the software—not
just items that are wrong, but missing items as well. All criticism should be directed at
the code, not the person who created it. Participants shouldn’t take any criticism
personally. Leave your egos, emotions, and sensitive feelings at the door.
2. Follow Rules. A fixed set of rules should be followed. They may set the amount of code
to be reviewed (usually a couple hundred lines), how much time will be spent (a couple
hours), what can be commented on, and so on. This is important so that the participants
know what their roles are and what they should expect. It helps the review run more
smoothly.
3. Prepare. Each participant is expected to prepare for and contribute to the review.
Depending on the type of review, participants may have different roles. They need to
vary in who has the responsibility for static white-box testing. In some teams the
programmers are the ones who organize and run the reviews, inviting the software testers
as independent observers. In other teams the software testers are the ones who perform
this task, asking the programmer who wrote the code and a couple of his peers to assist in
the reviews. Ultimately, either approach can work. It’s up to the development team to
choose what works best for them.
4. Write a Report. The review group must produce a written report summarizing the results
of the review and make that report available to the rest of the product development team.
It’s imperative that others are told the results of the meeting—how many problems were
found, where they were found, and so they need to know what their duties and
responsibilities are and be ready to actively fulfill them at the review. Most of the
problems found through the review process are found during preparation, not at the actual
review.

Software Testing chapter 1 Vaishali Rane Page 17

Peer Reviews
The easiest way to get team members together and doing their first formal reviews of the
software is through peer reviews, the least formal method. Sometimes called buddy reviews, this
method is really more of an “I’ll show you mine if you show me yours” type discussion.
 Peer reviews are often held with just the programmer who wrote the code and one or two
other programmers or testers acting as reviewers.
 That small group simply reviews the code together and looks for problems and
oversights.
 To assure that the review is highly effective (and does not turn into a coffee break) all the
participants need to make sure that the four key elements of a formal review are in place:
Look for problems, follow rules, prepare for the review, and write a report.
 Because peer reviews are informal, these elements are often scaled back. Still, just getting
together to discuss the code can find bugs.

Walkthrough / Structured Walkthrough

Walkthroughs are the next step up in formality from peer reviews. In a walkthrough, the
programmer who wrote the code formally presents (walks through) it to a small group of five or
so other programmers and testers. The reviewers should receive copies of the software in advance
of the review so they can examine it and write comments and questions that they want to ask at
the review. Having at least one senior programmer as a reviewer is very important. The presenter
reads through the code line by line, or function by function, explaining what the code does and
why. The reviewers listen and question anything that looks suspicious. Because of the larger
number of participants involved in a walkthrough compared to a peer review, it’s much more
important for them to prepare for the review and to follow the rules. It’s also very important that
after the review the presenter write a report telling what was found and how he plans to address
any bugs discovered.
A structured walkthrough, a static testing technique performed in an organized manner between a
group of peers to review and discuss the technical aspects of software development process. The
main objective in a structured walkthrough is to find defects in order to improve the quality of
the product.
Structured walkthroughs are usually NOT used for technical discussions or to discuss the
solutions for the issues found. As explained, the aim is to detect error and not to correct errors.
When the walkthrough is finished, the author of the output is responsible for fixing the issues.
Benefits:
1. Saves time and money as defects are found and rectified very early in the lifecycle.
2. This provides value-added comments from reviewers with different technical
backgrounds and experience.
3. It notifies the project management team about the progress of the development process.
4. It creates awareness about different development or maintenance methodologies which
can provide a professional growth to participants.
Inspections
 Inspections are the most formal type of reviews. They are highly structured and require
training for each participant. Inspections are different from peer reviews and
walkthroughs in that the person who presents the code, the presenter or reader, isn’t the
original programmer.
 This forces someone else to learn and understand the material being presented,
potentially giving a different slant and interpretation at the inspection meeting.
 The other participants are called inspectors. Each is tasked with reviewing the code from
a different perspective, such as a user, a tester, or a product support person. This helps
bring different views of the product under review and very often identifies different bugs.
Software Testing chapter 1 Vaishali Rane Page 18
  One inspector is even tasked with reviewing the code backward—that is, from the end to
the beginning—to make sure that the material is covered evenly and completely.
 Some inspectors are also assigned tasks such as moderator and recorder to assure that the
rules are followed and that the review is run effectively.
 After the inspection meeting is held, the inspectors might meet again to discuss the
defects they found and to work with the moderator to prepare a written report that
identifies the rework necessary to address the problems.
 The programmer then makes the changes and the moderator verifies that they were
properly made.
 Depending on the scope and magnitude of the changes and on how critical the software
is, a re-inspection may be needed to locate any remaining bugs.
 Inspections have proven to be very effective in finding bugs in any software deliverable,
especially design documents and code, and are gaining popularity as companies and
product development teams discover their benefits.

Technical review
A Technical review is a static white-box testing technique which is conducted to spot the defects
early in the life cycle that cannot be detected by black box testing techniques.
Technical Review Characteristics:
1. Technical Reviews are documented and uses a defect detection process that has peers and
technical specialist as part of the review process.
2. The Review process doesn't involve management participation.
3. It is usually led by trained moderator who is NOT the author.
4. The report is prepared with the list of issues that needs to be addressed.

Coding standards:
It’s important, though, and there are three reasons for adherence to a standard or guideline:
• Reliability. It’s been shown that code written to a specific standard or guideline is more
reliable and bug-free than code that isn’t.
• Readability/Maintainability. Code that follows set standards and guidelines is easier to
read, understand, and maintain.
• Portability. Code often has to run on different hardware or be compiled with different
compilers. If it follows a set standard, it will likely be easier or even completely painless to move
it to a different platform.

National and international standards for most computer languages and information
technology can be obtained from:
• American National Standards Institute (ANSI), [Link]
• International Engineering Consortium (IEC), [Link]
• International Organization for Standardization (ISO), [Link]
• National Committee for Information Technology Standards (NCITS), [Link]
There are also documents that demonstrate programming guidelines and best practices available
from professional organizations such as
• Association for Computing Machinery (ACM), [Link]
• Institute of Electrical and Electronics Engineers, Inc (IEEE), [Link]

Generic code review checklist :

The review of the code is conducted with respect to checking the code with respect to the
following errors:
 Data Reference Errors :
Software Testing chapter 1 Vaishali Rane Page 19
 Data reference errors are bugs caused by using a variable, constant, array, string, or
record that hasn’t been properly initialized for how it’s being used and referenced.
 Data Declaration Errors
Data declaration bugs are caused by improperly declaring or using variables or constants.

 Computation Errors
Computational or calculation errors are essentially bad math. The calculations don’t
result in the expected result.

 Comparison Errors
Less than, greater than, equal, not equal, true, false. Comparison and decision errors are
very susceptible to boundary condition problems.

 Control Flow Errors

Control flow errors are the result of loops and other control constructs in the language not
behaving as expected. They are usually caused, directly or indirectly, by computational or
comparison errors.

 Input/Output Errors
These errors include anything related to reading from a file, accepting input from a
keyboard or mouse, and writing to an output device such as a printer or screen. T

Dynamic White Box Testing :

Dynamic white-box testing, in a nutshell, is using information gained from seeing what the
code does and how it works to determine what to test, what not to test, and how to approach
the testing.
Another name commonly used for dynamic white-box testing is structural testing
because you can see and use the underlying structure of the code to design and run your tests.
The four areas that dynamic white-box testing encompasses are
• Directly testing low-level functions, procedures, subroutines, or libraries. In Microsoft
Windows, these are called Application Programming Interfaces (APIs).
• Testing the software at the top level, as a completed program, but adjusting your test
cases based on what you know about the software’s operation.
• Gaining access to read variables and state information from the software to help you
determine whether your tests are doing what you thought. And, being able to force the
software to do things that would be difficult if you tested it normally.
• Measuring how much of the code and specifically what code you “hit” when you run
your tests and then adjusting your tests to remove redundant test cases and add missing
ones.

1. Code functional testing

This is a initial part of a structural testing .
The developer performs some quick checks before exerting the code coverage and code
complexity testing.
The methods followed are:
 The developer can perform certain obvious tests, knowing the input variables and he
corresponding expected output variables. This can be a quick test that checks out any
obvious mistakes. By repeating these tests for multiple values of input variables, the
confidence level of the developer to go to the next level.

Software Testing chapter 1 Vaishali Rane Page 20

  The developer can built a “debug versions” of the product by putting intermediate print
statements and making sure the programs is passing through the right loops and iterations
the right number of times.
 Run the product under a debugger or an Integrated Development Environment (IDE).

2. Code coverage testing

Code coverage is a white box testing methodology, that is it requires knowledge of and access to
the code itself rather than simply using the interface provided.

Code coverage is probably most useful during the module testing phase, though it also has
benefit during integration testing and probably at other times, depending on how and what you
are testing.

Code coverage is a measurement of how many lines/blocks/arcs of your code are executed while
the automated tests are running.

Code coverage is collected by using a specialized tool to instrument the binaries to add tracing
calls and run a full set of automated tests against the instrumented product. A good tool will give
you not only the percentage of the code that is executed, but also will allow you to drill into the
data and see exactly which lines of code were executed during a particular test.

Code coverage measures how much code of the application is being exercised when the tests are
run. This makes sure that the tests being run are actually testing the code of application. The basic
measure of Code Coverage is the “Coverage Item” , which can by anything we have been able to
count and see whether it has been tested or not.
Measurement of Coverage can be determined by the following formula.
Coverage= Number of coverage items exercised / Total number of coverage items *100%.
It should be kept in mind that 100% code coverage does not mean that the application is 100%
tested.
An application with high code coverage means it has been more thoroughly tested and would
contain less software bugs than an application with low code coverage.
There are numerous ways to calculate code coverage like program subroutines and program
statements called during the execution of test suite.

Coverage criteria are method to measure how much code has been exercised by the test suite.
The test suite must satisfy the rule of the coverage criteria. The various Coverage criteria are as
follows:

1. Statement/line Coverage
2. Branch / Decision Coverage
3. Condition Coverage/ predicate coverage
4. Path Coverage

1. Statement /Line coverage

Statement coverage is a white box testing technique, which involves the execution of all
the statements at least once in the source code.
It is a metric, which is used to calculate and measure the number of statements in the
source code which have been executed.

Software Testing chapter 1 Vaishali Rane Page 21

 Using this technique we can check what the source code is expected to do and what it
should not.
It can also be used to check the quality of the code and the flow of different paths in the
program.
The main drawback of this technique is that we cannot test the false condition in it.
It can be calculated as :
Statement coverage = (No of statements Executed/exercised /Total no of statements
in the source code ) x 100
Example:
Read A
Read B
if A>B
Print “A is greater than B”
else
Print "B is greater than A"
endif
Set1 :If A =5, B =2
No of statements Executed: 5
Total no of statements in the source code: 7
Statement coverage =5/7*100 = 71.00 %
Advantage of statement coverage:
 It verifies what the written code is expected to do and not to do
 It measures the quality of code written
 It checks the flow of different paths in the program and it also ensure that whether those
path are tested or not.

Disadvantage of statement coverage:

 It cannot test the false conditions.

 It does not report that whether the loop reaches its termination condition.
 It does not understand the logical operators.

2. Branch coverage / Predicate coverage /Decision coverage

Branch coverage is also known as Decision coverage or all-edges coverage.
It covers both the true and false conditions unlikely the statement coverage.
 A branch is the outcome of a decision, so branch coverage simply measures which
decision outcomes have been tested. This sounds great because it takes a more in-depth
view of the source code than simple statement coverage
 A decision is an IF statement, a loop control statement (e.g. DO-WHILE or REPEAT-
UNTIL), or a CASE statement, where there are two or more outcomes from the
statement. With an IF statement, the exit can either be TRUE or FALSE, depending on
the value of the logical condition that comes after IF.
The decision coverage can be calculated as given below:

Branch / decision coverage = (Number of decisions outcomes tested /Total Number

of decision outcomes ) x 100

Software Testing chapter 1 Vaishali Rane Page 22

 decision coverage requires each decision to have had both a True and False
outcome. Therefore, to achieve 100% decision coverage, a second test case is necessary where A
is less than or equal to B which ensures that the decision statement ‘IF A > B’ has a False outcome.
So one test is sufficient for 100% statement coverage, but two tests are needed for 100% decision
coverage. It is really very important to note that 100% decision coverage guarantees 100%
statement coverage, but not the other way around.

1 READ A
2 READ B
3 C = A – 2 *B
4 IFC < 0THEN
5 PRINT “ C negative”
6 ENDIF
Let’s suppose that we already have the following test, which gives us 100% statement coverage
for code sample shown above

TEST SET 2 Test 2_1: A = 20, B = 15

The value of C is -10, so the condition ‘C < 0’ is True, so we will print ‘C negative’ and we have
executed the True outcome from that decision statement. But we have not executed the False
outcome of the decision statement. What other test would we need to exercise the False outcome
and to achieve 100% decision coverage?

Before we answer that question, let’s have a look at another way to represent this code. Sometimes
the decision structure is easier to see in a control flow diagram

The dotted line shows where Test 2_1 has gone and clearly shows that we haven’t yet had a test
that takes the False exit from the IF statement.
Let’s modify our existing test set by adding another test:

TEST SET 2
Test 2_1: A = 20, B = 15
Test 2_2: A = 10, B = 2

This now covers both of the decision outcomes, True (with Test 2_1) and False (with Test 2_2). If
we were to draw the path taken by Test 2_2, it would be a straight line from the read statement
down the False exit and through the ENDIF. We could also have chosen other numbers to achieve
either the True or False outcomes.

Advantages of decision coverage:

Software Testing chapter 1 Vaishali Rane Page 23

  To validate that all the branches in the code are reached
 To ensure that no branches lead to any abnormality of the program’s operation
 It eliminate problems that occur with statement coverage testing

Disadvantages of decision coverage:

 This metric ignores branches within Boolean expressions which occur due to short-circuit
operators.
3. Condition coverage /predicate coverage
 This is closely related to decision coverage but has better sensitivity to the control flow.
However, full condition coverage does not guarantee full decision coverage.
 Condition coverage reports the true or false outcome of each condition.
 Condition coverage measures the conditions independently of each other.
 Other control-flow code-coverage measures include linear code sequence and jump
(LCSAJ) coverage, multiple condition coverage (also known as condition combination
coverage) and condition determination coverage (also known as multiple condition
decision coverage or modified condition decision coverage, MCDC).
 This technique requires the coverage of all conditions that can affect or determine the
decision outcome.

 Conditional coverage or expression coverage will reveal how the variables or sub-
expressions in the conditional statement are evaluated. In this coverage expressions with
logical operands are only considered.
 For example, if an expression has Boolean operations like AND, OR, XOR, which
indicated total possibilities.
 Conditional coverage offers better sensitivity to the control flow than decision coverage.
Condition coverage does not give a guarantee about full decision coverage
 The formula to calculate Condition Coverage:
Condition coverage = (Number of conditions exerted / Total number of conditions in
program) x 100
4. Function coverage testing
This is done to identify how many functions within the program are covered for testing.
The requirements of product are mapped into functions during the design phase and each
of the functions form a logical unit.
While providing the function coverage , test cases can be written so as to exercise each of
the different functions in the code.
We can also measure how many times the functions is called. This will indicate which
function is used most often and hence these functions become the target of any
performance testing. This will improve the performance as well as quality of the product.

Function coverage = (total number of functions exercised / total number functions in

a program) X 100

Advantages of function coverage are :

1. Functions are easier to identify in a program.

Software Testing chapter 1 Vaishali Rane Page 24

 2. Since the functions are at much higher level of abstraction that code, it is easier to
achieve 100 percent functions coverage.
3. Functions have more logical mapping to requirements and hence can provide a
more direct correlation to the test coverage of the product.
4. Since functions are means of realizing requirement, the importance of functions
can be prioritized based on the importance of the requirements they realize. So it
would be easy to prioritize the functions for testing.

5. Path Coverage testing :

Path Coverage tells us whether all the paths in each function have been followed. The path
coverage determines the paths are possible combinations of logical operators.
A path is a sequence of branches from entry point to exit point. It is useful as it requires thorough
testing. Path Coverage has disadvantages as the number of paths is exponential to number of
branches. Also due to relationship of data many paths are not possible.
The formula below is used to calculate the path coverage :

Path coverage= total paths exerted/ executed / total number of paths X 100

Code Complexity Testing :

It is also called as Basis Path Testing.

This enables the tester to drive a logical complexity measure of a procedural design and use this
measure as a guide for defining a basis set of execution paths. Test cases derived to exercise the
basis set are guaranteed to execute every during testing.

We may use ‘FLOW GRAPH NOTATION’ as a useful loop for understanding control flow and
illustrating the approach.

We can use ‘cyclomatic complexity “ is a software metric that provides a quantitative measure
of the logical complexity of a program. Here, the value computed for cyclomatic complexity
defines the No. of independent paths is the basis set of program and provides as with the upped
bound for the No. of tests that be conducted to ensure that all statements have been exercise at
least once.
1. Flow graph notation :
Flow Graph is defined as a function in a program that can be represented as a control flow graph
and the nodes in the flow graph are defined as program statements while the directed edges are
the flow of control. A Flow Graph consists of nodes and edges. The two nodes in the Flow Graph
can be either unconnected or connected by an edge in either direction or connected by an edge in
all directions.

While tracing a path from a source to a sink a back edge is an edge that leads back to a node that
has already been visited. The Flow Graph contains one source node and one sink.
A source node is the node that has no incoming edges while a sink node is the node with no
outgoing edges. A program's function may contain more than one sink node, but this graph can
be converted into a graph with only one sink. There are some languages that allow more than one

Software Testing chapter 1 Vaishali Rane Page 25

source. This construct is very rare and not used in Structured Programming.

1. Used as a main tool for test case identification.

2. Represents the relationship between program segments, that is the sequence of statements
having the property that if the first member of the sequence is executed then all other
statements in that sequence will also be executed.
3. Nodes represent one program segment.
4. The area bounded by edges and nodes are called regions.

Flow graph Notation

Before basic path procedure is discussed, it is important to know the simple notation used for the
repres4enttation of control flow. This notation is known as flow graph. Flow graph depicts
control flow and uses the following constructs.
These individual constructs combine together to produce the flow graph for a particular
procedure

Sequence - Until -

If - While -

Case

-
Basic terminology associated with the flow graph
Node: Each flow graph node represents one or more procedural statements. Each node that
contains a condition is called a predicate node.
Edge: Edge is the connection between two nodes. The edges between nodes represent flow of
control. An edge must terminate at a node, even if the node does not represent any useful
procedural statements.
Region: A region in a flow graph is an area bounded by edges and nodes. Cyclomatic
complexity: Independent path is an execution flow from the start point to the end point. Since a
Software Testing chapter 1 Vaishali Rane Page 39
procedure contains control statements, there are various execution paths depending upon decision
taken on the control statement. So Cyclomatic complexity provides the number of such execution
independent paths. Thus it provides a upper bound for number of tests that must be produced
because for each independent path, a test should be conducted to see if it is actually reaching the
end point of the procedure or not.
2. Cyclomatic Complexity
Cyclomatic Complexity for a flow graph is computed in one of three ways:
1. The numbers of regions of the flow graph correspond to the Cyclomatic complexity.
2. Cyclomatic complexity, V(G), for a flow graph G is defined as

V(G) = E – N + 2

where E is the number of flow graph edges and N is the number of flow graph nodes.
3. Cyclomatic complexity, V(G), for a graph flow G is also defined as

V(G) = P + 1

Where P is the number of predicate nodes contained in the flow graph G.

Example: Consider the following flow graph

Region, R= 6
Number of Nodes = 13
Number of edges = 17
Number of Predicate Nodes = 5
Cyclomatic Complexity, V( C) :
V( C ) = R = 6;
Or
V(C) = Predicate Nodes + 1
=5+1 =6
Or
V( C)= E-N+2
= 17-13+2
Deriving Test Cases
The main objective of basic path testing is to derive the test cases for the procedure under test.
The process of deriving test cases is following
1. From the design or source code, derive a flow graph.
2. Determine the Cyclomatic complexity, V(G) of this flow graph using any of the formula
discussed above.
· Even without a flow graph, V(G) can be determined by counting the number of
conditional statements in the code and adding one to it.
3. Prepare test cases that will force execution of each path in the basis set.
Software Testing chapter 1 Vaishali Rane Page 40
 · Each test case is executed and compared to the expected results.
Graph Matrices

Graph matrix is a two dimensional matrix that helps in determining the basic set. It has rows and columns
each equal to number of nodes in flow graph. Entry corresponding to each node-node pair represents an
edge in flow graph. Each edge is represented by some letter (as given in the flow chart) to distinguish it
from other edges. Then each edge is provided with some link weight, 0 if there is no connection and 1 if
there is connection.
For providing weights each letter is replaced by 1 indicating a connection. Now the graph matrix
is called connection matrix. Each row with two entries represents a predicate node. Then for each
row sum of the entries is obtained and 1 is subtracted from it. Now the value so obtained for each
row is added and 1 is again added to get the cyclomatic complexity.
3. Compound logic
Compound Condition Coverage requires that all combinations of condition values at every
branch statement will have been covered, and that every entry point will have been taken,
at least once.
Also known as Multiple Condition Coverage.
Subsumes Branch/Condition Coverage, regardless of the order in which conditions are evaluated.
Where one or more Boolean operators such as the logical OR, AND, NAND, NOR are present is
a conditional statement:
1. IF a OR b
2. then procedure x
3. else procedure y
4. ENDIF
A predicate node will be created for each statement.

4. Independent program paths

Linearly independent path The path-coverage testing does not require coverage of
all paths but only coverage of linearly independent paths.
A linearly independent path is any path through the program that introduces at
least one new edge that is not included in any other linearly independent paths. If
a path has one new node compared to all other linearly independent paths, then the
path is also linearly independent. This is because, any path having a new node
automatically implies that it has a new edge. Thus, a path that is subpath of
another path is not considered to be a linearly independent path.
It’s important to understand the idea of paths, so let’s go through that idea really
quick with an isolated example. Say you have these paths:
 path 1: 1-11
 path 2: 1-2-3-4-5-10-1-11
 path 3: 1-2-3-6-8-9-10-1-11
Software Testing chapter 1 Vaishali Rane Page 41
  path 4: 1-2-3-6-7-9-10-1-11
Note that each new path introduces a new edge — meaning, a line to new nodes on the path.
Now consider this path:
 path 5: 1-2-3-4-5-10-1-2-3-6-8-9-10-1-11
Path 5 is not considered to be an independent path because it’s simply a
combination of the already specified paths (paths 2 and 3) and does not traverse
any new edges.
Paths 1, 2, 3, and 4 constitute a basis set for a possible control flow graph. That is,
if tests can be designed to force execution of those initial four paths, which make
up a basis set, then everystatement in the program will have been guaranteed to be
executed at least one time and every condition will have been executed on its true
and false sides.
Path testing is sometimes referred to as basis path testing and now you know why.
A basis set is a set of linearly independent test paths. Any path through the control
flow graph can be formed as a combination of paths in the basis set. I should note
that the basis set is not unique. In fact, a number of different basis sets can be
derived for a given procedural design.
With that out of the way, consider this control flow graph:

Let’s calculate the independent paths through this part of the application. An
important thing to take note of here before I start is that I’m going to do this
calculation without having to know what the actual logic I’m modeling is. (This,
by the way, is a great way to impress people you meet in a bar.) [Right now you
are no doubt wondering what kind of bars I frequent.]
First you have to know the total number of nodes. This is simple to calculate:
 Nodes = Decisions + Processes
Nodes are where anything can happen and so the full count of nodes are those
places where decisions are being made and those places where a specific
algorithm or bit of logic is being performed. There are then three equations that
you can use to calculate the independent paths.
 Independent Paths = Edges – Nodes + 2
 Independent Paths = Regions + 1
 Independent Paths = Decisions + 1
Technical Note: Those three equations are from graphing theory and they are
Software Testing chapter 1 Vaishali Rane Page 42
 used to calculate the number of linearly independent paths through any structured
system. This cyclomatic number is the number of linearly independent paths
through the system. I bring all this up not because it really matters but mainly
because you might have heard the term “cyclomatic complexity” thrown around.
The number of nodes is 6. These are the processes and decisions of the graph (p1,
p2, p3, p4 and d1, d2). The number of edges is 7, which are indicated by e1
through e7 on the graph. So, taking the first calculation path above:
Independent Paths = Edges - Nodes + 2
Independent Paths =7-6+2
Independent Paths =3
You could also calculate by the number of regions. Areas bounded by edges and
nodes are called regions. There are two regions in the graph. So:
Independent Paths = Regions + 1
Independent Paths =2+1
Independent Paths =3
You could also calculate by the number of decision nodes. There are two such
nodes in the graph. So:
Independent Paths = Decisions + 1
Independent Paths =2+1
Independent Paths =3
I bring up these variations because different resources out there use different
calculations and I’ve found people getting confused when what they were reading
didn’t seem to add up. It’s essentially looking at it in three different ways but,
note, getting the same answer. In fact, all three equations must be equal to the
same number for the “logic circuit” of the application to be valid. If the system is
not a valid logic circuit, it can’t work. I should also note that the first equation,
regarding edges minus nodes, is usually given as:
Independent Paths = Edges - Nodes + 2P
Here P refers to the number of connected components. In this case, P is the
number of unconnected parts of the graph. Usually this is going to be 1.
So now let’s go back to the Greatest Common Divisor example. Here’s the control
flow graph again for easy reference:

Software Testing chapter 1 Vaishali Rane Page 43

 5. Liming V(G)
The researches show that program module having V(G) greater than 10 is considered to be complex. Overly
complex module reduces maintainability and testability. The standard ranges of this metrics are as shown
below:

Cyclomatic complexity complexity level and risk

1-10-------------------------- simple program , without much risk
11-20 ----------------------  more complex , moderate risk
21-50  more complex, moderate risk
Greater than 50 ------------  untestable program, very high risk

Black Box Testing

Black box testing is defined as a testing technique in which functionality of the Application
Under Test (AUT) is tested without looking at the internal code structure, implementation details
and knowledge of internal paths of the software. This type of testing is based entirely on software
requirements and specifications.
n BlackBox Testing we just focus on inputs and output of the software system without bothering
about internal knowledge of the software program.
The above Black-Box can be any software system you want to test. For Example, an operating
system like Windows, a website like Google, a database like Oracle or even your own custom
application. Under Black Box Testing, you can test these applications by just focusing on the
inputs and outputs without knowing their internal code implementation.

How to do BlackBox Testing

Here are the generic steps followed to carry out any type of Black Box Testing.
 Initially, the requirements and specifications of the system are examined.
 Tester chooses valid inputs (positive test scenario) to check whether SUT processes them
correctly. Also, some invalid inputs (negative test scenario) are chosen to verify that the
SUT is able to detect them.
 Tester determines expected outputs for all those inputs.
 Software tester constructs test cases with the selected inputs.
 The test cases are executed.
 Software tester compares the actual outputs with the expected outputs.
 Defects if any are fixed and re-tested.

Advantages and Disadvantages of Black Box Testing

Black Box Testing has its own set of advantages and disadvantages, which we will hover below:
⇒ Advantages of Black Box Testing
 No need for implementation knowledge, programming knowledge – Tester can be non-
technical person
 Tester and developer are independent and there are no dependencies between them
 Assess the application quality by analyzing the actual outcome with expected outcome
 Helps in developing software in a more user-friendly way than mentioned in requirement
document as a tester is supposed to test the application from user’s point of view.
Software Testing chapter 1 Vaishali Rane Page 44
  Testing can be done once the application is released by the development team and does
not have any dependency on the development team for testing.
 Once the functional specification is complete, test cases can be designed.
 More effective when the application is large and complex
 Ambiguities, inconsistencies, and incompleteness can be identified at the early stage of
testing
⇒ Disadvantages of Black Box Testing
 There are chances of leaving unidentified conditions in absence of knowledge of
programming structure.
 In short span of time, it is quite difficult for the testers to identify all possible inputs to
write test cases for a particular scenario and test it.
 Practically, only limited number of possible inputs can be tested through manual testing
 Not all the paths/logics are tested
 Results in repeated tests if the developer has already tested the feature, and tester is not
aware of it
 Majority of the bugs are expected to be covered from test cases, which is quite difficult of
the application is large and complex
 Difficult to achieve 100% Test coverage if the application is complex

Black box testing techniques

Black box techniques are divided into static black box and dynamic black box technique.
Static black box technique is done with “Requirement based testing”
Dynamic black box technique is done with help of
[Link] and negative testing,
2. Boundary value analysis,
3. Decision tables,
4. equivalence partitioning,
5. user documentation testing and
6. Graph based testing

1. Requirement Based Testing.

Requirements-based testing is a testing approach in which test cases, conditions
and data are derived from requirements. It includes functional tests and also
nonfunctional attributes such as performance, reliability or usability. i. Stages in
Requirements based Testing:
Defining Test Completion Criteria - Testing is completed only when all the
functional and non-functional testing is complete.
Design Test Cases - A Test case has five parameters namely the initial state or
precondition, data setup, the inputs, expected outcomes and actual outcomes. 
Execute Tests - Execute the test cases against the system under test and document
the results.
Verify Test Results - Verify if the expected and actual results match each other.
Verify Test Coverage - Verify if the tests cover both functional and nonfunctional
aspects of the requirement.
Track and Manage Defects - Any defects detected during the testing process goes
through the defect life cycle and are tracked to resolution. Defect Statistics are
maintained which will give us the overall status of the project. ii. Requirements
testing process: Testing must be carried out in a timely manner. Testing process
should add value to the software life cycle, hence it needs to be effective. Testing
the system exhaustively is impossible hence the testing process needs to be
Software Testing chapter 1 Vaishali Rane Page 45
 efficient as well. Testing must provide the overall status of the project, hence it
should be manageable.
Requirements are traced by a RTM(requirements traceability matrix). Requirement
Traceability Matrix or RTM captures all requirements proposed by the client or software
development team and their traceability in a single
document delivered at the conclusion of the life-cycle.
In other words, it is a document that maps and traces user requirement with test
cases. The main purpose of Requirement Traceability Matrix is to see that all test
cases are covered so that no functionality should miss while doing Software
testing.
Which Parameters to include in Requirement Traceability Matrix?
 Requirement ID
 Requirement Type and Description
 Test Cases with Status

Above is a sample requirement traceability matrix.

Test Requirements have a separate set of fields and templates that can be
configured for them.
Test Cases can be linked to Requirements, in one-to-one, many-to-one, and one-
to-many arrangements.
One to one : for each requirement there is one set of test case.
One to many : for each requirement there are many test cases
Many to one : a set of requirements can be tested with one test case
Many to many : many requirements are tested by many test cases.
One to none : the set of requirements are not tested by any test case.

Dynamic black box testing :

1. Positive and Negative Testing
i. Positive Testing:
Positive Testing is testing process where the system validated against the
valid input data.
In this testing tester always check for only valid set of values and check if
an application behaves as expected with its expected inputs.
Software Testing chapter 1 Vaishali Rane Page 46
 The main intention of this testing is to check whether software application
not showing error when not supposed to & showing error when supposed
to.
Such testing is to be carried out keeping positive point of view & only
execute the positive scenario.
Positive Testing always tries to prove that a given product and project
always meets the requirements and specifications.
Under Positive testing is test the normal day to day life scenarios and check
the expected behaviour of application.
Example of Positive testing:
To test the text box for integer numbers only. As we are testing for positive
testing the input range is valid numbers only.

2. Negative Testing:
Negative Testing is testing process where the system validated against the
invalid input data.
A negative test checks if an application behaves as expected with its
negative inputs.
The main intention of this testing is to check whether software application
not showing error when supposed to & showing error when not supposed
to. Such testing is to be carried out keeping negative point of view & only
execute the test cases for only invalid set of input data.
Negative testing is a testing process to identify the inputs where system is
not designed or un-handled inputs by providing different invalid. The main
reason behind Negative testing is to check the stability of the software
application against the influences of different variety of incorrect
validation data set.
NEGATIVE TESTING

AGE : ABC

Enter only numbers

Software Testing chapter 1 Vaishali Rane Page 47

Differences between positive and negative testing

2. Boundary value analysis

i. Boundary conditions are special because programming, by its nature, is

susceptible to problems at its edges.
ii. The boundary conditions are defined as the initial and the final data ranges
of the variables declared.
iii. If an operation is performed on a range of numbers, odds are the
programmer got it right for the vast majority of the numbers in the middle, but
maybe made a mistake at the edges.
iv. The edges are the minimum and the maximum values for that identifier.
1. #include<stdio.h>
2. void main()
3. {
4. int i , fact=1, n;
5. printf(“enter the number “);
6. scanf(“%d”,&n);
7. for(i =1 ;i <=n;i++)
8. fact = fact * i;

10. }
The boundary condition in the above example is for the integer variable.
Sub-Boundary Conditions
i. They’re the ones defined in the specification or evident when using the
software.
ii. Some boundaries, though, that are internal to the software aren’t necessarily
apparent to an end user but still need to be checked by the software tester.
iii. These are known as sub-boundary conditions or internal boundary
conditions.
iv. In the given example the sub boundary condition is the value of factorial

For example
Software Testing chapter 1 Vaishali Rane Page 48
 1. #include<stdio.h>
2. void main()
3. {
4. int i , fact=1, n;
5. printf(“enter the number “);
6. scanf(“%d”,&n);
7. for(i =1 ;i <=n;i++)
8. fact = fact * i;
9. printf (
10. }

Boundary testing is the process of testing between extreme ends or boundaries

between partitions of the input values.
 So these extreme ends like Start- End, Lower- Upper, Maximum-Minimum, Just Inside-
Just Outside values are called boundary values and the testing is called "boundary
testing".
 The basic idea in boundary value testing is to select input variable values at their:
1. Minimum
2. Just below the Minimum
3. Just above the minimum
4. A nominal value
5. Just below the maximum
6. Maximum
Boundary value analysis:
It's widely recognized that input values at the extreme ends of input domain cause more errors in
the system. More application errors occur at the boundaries of input domain. ‘Boundary value
analysis' testing technique is used to identify errors at boundaries rather than finding those exist
in center of input domain.
Boundary value analysis is the next part of Equivalence partitioning for designing test cases
where test cases are selected at the edges of the equivalence classes.
Test cases for input box accepting numbers between 1 and 1000 using Boundary value
analysis:
1) Test cases with test data exactly as the input boundaries of input domain i.e. values 1 and 1000
in our case.
2) Test data with values just below the extreme edges of input domains i.e. values 0 and 999.
3) Test data with values just above the extreme edges of input domain i.e. values 2 and 1001.
Boundary value analysis is often called as a part of stress and negative testing.
Note: There is no hard-and-fast rule to test only one value from each equivalence class you
created for input domains. You can select multiple valid and invalid values from each
equivalence class according to your needs and previous judgments.
E.g. if you divided 1 to 1000 input values invalid data equivalence class, then you can select test
case values like 1, 11, 100, 950 etc. Same case for other test cases having invalid data classes.
This should be a very basic and simple example to understand the Boundary value analysis and
Equivalence partitioning concept.
Example :
 Consider the behavior of flight reservation application.
 1-10 ticket values will be considered valid.
 Only 10 tickets can be booked at one time.
 A number between 1-10 is valid, any number below 1 and above 10 is invalid.

Software Testing chapter 1 Vaishali Rane Page 49

Test Scenario Description Expected Outcome

Boundary Value = 0 System should NOT accept

Boundary Value = 1 System should accept
Boundary Value = 2 System should accept
Boundary Value = 9 System should accept
Boundary Value = 10 System should accept
Boundary Value = 11 System should NOT accept

This testing is used to reduce the very large number of test cases into minimum chunks.
4. Equivalence Partitioning
i. Type of black box testing that divides that divides the input domain of the program into classes
of data from which the test cases can be divided.
ii. This method is typically used to reduce the total number of test cases to a finite set of testable
test cases, still covering maximum requirements.
iii. It is the process of taking all possible test cases and placing them into classes. One test value
is picked from each class while testing.

For example:
If you are testing for an input box accepting numbers from 1 to 1000 then there is no use in
writing thousand test cases for all 1000 valid input numbers plus other test cases for invalid data.
i. Using equivalence partitioning method above test cases can be divided into three sets of input
data called as classes. Each test case is a representative of respective class.
ii. For the above example we can divide our test cases into three equivalence classes of some
valid and invalid inputs.

Test cases for input box accepting numbers between 1 and 1000 using Equivalence
Partitioning:
1. One input data class with all valid inputs. Pick a single value from range 1 to 1000 as a valid
test case. If you select other values between 1 and 1000 then result is going to be same. So one
test case for valid input data should be sufficient.
2. Input data class with all values below lower limit. i.e. any value below 1, as an invalid input
data test case.
3. Input data with any value greater than 1000 to represent third invalid input class.
4. So using equivalence partitioning all possible test cases is partitioned into three classes. Test
cases with other values from any class should give you the same result.
5. We have selected one representative from every input class to design our test cases. Test case
values are selected in such a way that largest number of attributes of equivalence class can be
exercised.
6. Equivalence partitioning uses fewest test cases to cover maximum requirements.

4. Decision Tables.
i. Decision table testing is black box test design technique to determine the test scenarios for
complex business logic.
ii. Decision tables provide a systematic way of stating complex business rules, which is useful for
developers as well as for testers.
iii. Decision tables can be used in test design whether or not they are used in specifications, as
they help testers explore the effects of combinations of different inputs and other software states
that must correctly implement business rules.
Software Testing chapter 1 Vaishali Rane Page 50
iv. It helps the developers to do a better job can also lead to better relationships with them.
v. Testing combinations can be a challenge, as the number of combinations can often be huge. vi.
Testing all combinations may be impractical if not impossible.

vii. We have to be satisfied with testing just a small subset of combinations but making the
choice of which combinations to test and which to leave out is also important.
viii. If you do not have a systematic way of selecting combinations, an arbitrary subset will be
used and this may well result in an ineffective test effort.

Decision table examples :

1. Chemical vending

2. Printer Troubleshooting

3. Salary of employees on hourly basis

Software Testing chapter 1 Vaishali Rane Page 51

6. User Documentation Testing.
i. Documentation testing is a non-functional type of software testing.
ii. It is a type of non-functional testing.
iii. Any written or pictorial information describing, defining, specifying, reporting,
or certifying activities, requirements, procedures, or results’. Documentation is as important to a
product’s success as the product itself.
iv. If the documentation is poor, non-existent, or wrong, it reflects on the quality of the roduct
and the vendor.
v. As per the IEEE Documentation describing plans for, or results of, the testing of a system or
component, Types include test case specification, test incident report, test log, test plan, test
procedure, test report. Hence the testing of all the above mentioned documents is known as
documentation testing.
vi. This is one of the most cost effective approaches to testing. If the documentation is not right:
there will be major and costly problems.
vii. The documentation can be tested in a number of different ways to many different degrees of
complexity.
viii. These range from running the documents through a spelling and grammar checking device,
to manually reviewing the documentation to remove any ambiguity or inconsistency.
ix. Documentation testing can start at the very beginning of the software process and hence save
large amounts of money, since the earlier a defect is found the less it will cost to be fixed.

7. Graph Based Testing.

i. Each and every application is buildup of some objects.
ii. All such objects are identified and graph is prepared.
iii. From this object graph each object relationship is identified and test cases written accordingly
to discover the errors.

Software Testing chapter 1 Vaishali Rane Page 52

Graph notations :

Nodes/objects are represented as circles

Links are categorized as :
1. Directed link (one direction)
2. Bidirectional link or undirected link
3. Parallel links

State Testing
o The data gets tested on the numbers, words, inputs, and outputs of the software.
o The product or the software should be tested for the program’s logic flow through its
various states.
o A software state is a condition or mode that the software is currently in.
o Consider the below Figures, which illustrates the software mode of the paint
software in airbrush mode.

Software Testing chapter 1 Vaishali Rane Page 53

Testing Software’s Logic Flow
1. Testing the software’s states and logic flow has the same problems. It’s usually possible to visit
2. Check for the all possible states for test to pass condition
3. The complexity of the software, especially due to the richness of today’s user interfaces, provides
so many choices and options that the number of paths grows exponentially.

Creating a State Transition Map

i. There are various techniques for state transition diagrams.

ii. Figure shows two examples. One uses boxes and arrows and the other uses circles (bubbles)
and arrows.
iii. The state transition diagram should be easily understandable and should be able to explain
clearly about the various stages the software passes through.

Figure : state transition diagram

Figure: State Transition Diagram

A state transition map should show the following items:

Software Testing chapter 1 Vaishali Rane Page 54

probably is.
if you find out later that it isn’t.

fic reason is what you’re looking for

here.

a calculation being performed, and so on.

Software Testing chapter 1 Vaishali Rane Page 55

Software Testing Question Bank
SNO CHAPTER 1 Question bank MARKS Year
1 What are entry and exit criteria of software testing? 4 S15, S16 , SP
2 Explain V model with labeled diagram 6 S15, W16 ,S17, SP
3 Give difference between Quality Assurance and Quality 4 S15, S17, W16
Control.(Any Four)
4 What is software testing? State objectives of software testing. 4 S16, SP
5 What is a ‘test case’? State its specification parameter 4 S16
6 State any four testing principles 4 S16
7 Differentiate between verification and validation. 4 S16
8 Explain the terms Mistake, Error, Defect, Bug, Fault and Failure 6 S16
in relation with software testing.
9 Define software testing. List all skills of software tester. 4 W16,S17,SP,S19
10 Describe alpha testing with its entry & exit criteria. 4 S17
11 List & explain techniques of finding bugs. 4 S17
12 Draw labeled diagram and describe Entry Task Verification Exit 4 SP
model for effective verification and validation.
13 Explain inspection and walkthrough. / differences 4 S19,W18
14 Explain when to start and stop testing? 4 S19
15 Explain quality control and quality assurance. 4 S19
16 Explain verification and validation with the help of neat 8 S19
diagram.
17 Explain white box testing and technical review. 4 S19
18 Explain code functional testing and code coverage testing with 8 S19
example.
19 What is user documentation testing ? State any three objectives of 4 W18
user documentation testing.
20 Explain requirement based testing with it’s stages and requirement 4 W18
testing process.
21 Draw classification of white box testing. Explain the purpose of code 4 W18
coverage testing.
22 Write four test cases for railway reservation system. 4 W18
23 Explain V-Model with labelled diagram. State it’s any two advantages 6 W19
and disadvantages.
24 Give any six difference between white box testing and black box 6 W18
testing.
25 With the help of suitable example, explain the use of decision table 8 W18
in the black box testing.
Software Testing chapter 1 Vaishali Rane Page 56
26 What is test case ? Which parameters are to be considered while 4 W18
documenting test cases ?
27 Prepare and write six test cases for simple calculator application. 6 S18
28 Define static and dynamic testing 4 W-19
29 Enlist objectives of software testing 4 w-19
30 Define bug, error, fault, and failure 4 w-19
31 Describe boundary value analysis with suitable example. 4 w-19

32 Describe the use of decision table in black box testing with the help of 4 w-19
suitable example.
33 Design test cases for railway reservation system 6 w-19
34 Design test cases for hostel admission form of your institute. 6 w-19

Software Testing chapter 1 Vaishali Rane Page 57

28 What is Boundary Value Analysis ? List any three guidelines for 4 S18
boundary value analysis.
29 Prepare and write four test cases for Library Management System of 4 S18
college.
30 Explain Equivalence partitioning with respect to equivalence classes. 4 S18
31 Explain any one type of structural white box testing in detail. 4 S18
32 Differentiate between verification and validation. 6 S18
33 With the help of an example, explain Graph-based testing. 8 S18
34 What is Black Box testing ? List any four techniques of Black Box 4 W17
testing.
35 Write any four test cases to test login form. 4 W17
36 Describe use of load testing and stress testing to test online result 6 W17
display facility of MSBTE website.
37 What is White Box testing ? Explain any one technique of static 4 W17
White Box testing.
38 With the help of suitable example explain decision table. Why 4 W17
decision table is important ?
39 What is test case ? Which parameters are to be considered while 4 W17
documenting test cases ?
40 Describe V-model with labelled diagram. State its any two 6 W17
advantages and disadvantages. Also write where it is applicable.
41 With the help of example explain Boundary Value Analysis. 6 W17
42 Prepare any eight test cases for admission form of college admission. 8 W17

43 With the suitable example, explain how ‘Basis Path Testing’ is used to 8 W17
derive the code complexity for the testing.

44 Define the terms Error : Defect, Fault and Bug in relation with 4 W17
Software testing.

Software Testing chapter 1 Vaishali Rane Page 58