Unit - 1

Basic definitions:
Error:
● A good synonym of Error is a mistake. When mistakes are made while coding, these are called
bugs.
● Errors tend to propagate. An example is a requirements error that may be magnified during design
and amplified more during coding.
Fault:
● A fault is the result of an error. A fault is the representation of an error, where representation is the
mode of expression, such as narrative text, dataflow diagrams, hierarchy charts, source code, and so
on.
● Defect is a good synonym for fault.
● Faults can be elusive. When a designer makes an error of omission, the resulting fault is that
something is missing that should be present in the representation.
● A fault of commission occurs when we enter something into an incorrect representation.
● Faults of omission occur when we fail to enter correct information. Of these two types, faults of
omission are more difficult to detect and resolve.
Failure:
● A failure occurs when a fault executes.
Two subtleties arise here:
1. Failures only occur in an executable representation, which is usually taken to be source code or
loaded object code.
2. This definition relates failures only to faults of commission. How to deal with failures that
correspond to faults of omission? What about faults that never happens to execute, or do not
execute for a long time? Reviews prevent many failures by finding faults and well-done reviews
can find faults of omission.
Incident:
● When a failure occurs, it may or may not be readily apparent to the user. An incident is the
symptom associated with a failure that alerts the user to the occurrence of a failure.
Test:
● A test is the act of exercising software with test cases.
● A test has two distinct goals: to find failures and to demonstrate correct execution.
Test case:
● The test case has an identity and is associated with a program behavior.
● A test case also has a set of inputs and expected outputs.
Identifying Test Cases
Two fundamental approaches are used to identify test cases,
1. Functional Testing
2. Structural testing.
Functional Testing
Functional testing considers any program to be a function that maps values from its input domain to values
in its output range. This leads to the term black box testing, in which the content(implementation) of a
black box is not known, and the function of the black box is understood completely in terms of its inputs
and outputs.

In the functional approach to test case identification, the only information used is the specification of the
software.
Functional test cases have two distinct advantages:
1. They are independent of how the software is implemented, so if the implementation changes, the
test cases are still useful.
2. Test case development can occur in parallel with the implementation, thereby reducing the overall
project development interval.
Functional test cases frequently suffer from two problems:
Significant redundancies may exist among test cases, compounded by the possibility of gaps of untested
software.
Figure 1.6 shows the results of test cases identified by two functional methods. Method A identifies a
larger set of test cases than does Method B. Notice that, for both methods, the set of test cases is
completely contained within the set of specified behavior. Because functional methods are based on the
specified behavior, it is hard to imagine these methods identifying behaviors that are not specified.
Structural Testing
Structural testing is the other fundamental approach to test case identification. To contrast it with
functional testing, it is sometimes called white box (or even clear box) testing. The clear box metaphoric
probably more appropriate, because the essential difference is that the implementation (of the black box) is
known and used to identify test cases. The ability to "see inside" the black box allows the tester to identify
test cases based on how the function is implemented. Structural testing has been the subject of some fairly
strong theories. To understand structural testing, familiarity with the concepts of linear graph theory is
essential.

With these concepts, the tester can rigorously describe exactly what is tested. Because of its strong
theoretical basis, structural testing lends itself to the definition and use of test coverage metrics. Test
coverage metrics provide a way to explicitly state the extent to which a software item has been tested, and
this in turn makes testing management more meaningful.
Figure 1.7 shows the results of test cases identified by two structural methods. As before, Method A
identifies a larger set of test cases than does Method B. Is a larger set of test cases necessarily better? This
is an excellent question, and structural testing provides important ways to develop an answer. Notice that,
for both methods, the set of test cases is completely contained within the set of programmed behaviours.
Because structural methods are based on the program, it is hard to imagine these methods identifying
behaviours that are not programmed. It is easy to imagine, however, that a set of structural test cases is
relatively small concerning the full set of programmed behaviours. In Chapter 11 we will see direct
comparisons of test cases generated by various structural methods.

The Functional versus Structural Debate

Given two fundamentally different approaches to test case identification, it is natural to question which is
better. If you read much of the literature, you will find strong adherents to either choice. Referring to
structural testing, Robert Poston writes, "This tool has been wasting tester's time since the 1970s ... [it]does
not support good software testing practice and should not be in the tester's toolkit" (Poston, 1991).
In defense of structural testing, Edward Miller writes, "Branch coverage [a structural test coverage metric],
if attained at the 85% or better level, tends to identify twice the number of defects that would have been
found by 'intuitive' [functional] testing" (Miller, 1991).
The Venn diagrams presented earlier yield a strong resolution to this debate. Recall that the goal of both
approaches is to identify test cases. Functional testing uses only the specification to identify test cases,
while structural testing uses the program source code (implementation) as the basis of test case
identification. Our earlier discussion forces the conclusion that neither approach alone is sufficient.
Consider program behaviors: if all specified behaviors have not been implemented, structural test
cases will never be able to recognize this. Conversely, if the program implements behaviors that have
not been specified, this will never be revealed by functional test cases. (A Trojan horse is a good
example of such unspecified behavior.) The quick answer is that both approaches are needed; the
testing crafts person's answer is that a judicious combination will provide the confidence of
functional testing and the measurement of structured testing.

Earlier, we asserted that functional testing often suffers from twin problems of redundancies and gaps.
When functional test cases are executed in combination with structural test coverage metrics, both of these
problems can be recognized and resolved (Figure 1.8).
The Venn diagram view of testing provides one final insight. What is the relationship between the set T of
test cases and the sets S and P of specified and implemented behaviours? The test cases in Tare are
determined by the test case identification method used. A very good question to ask is how appropriate (or
effective) is this method? To close a loop from an earlier discussion, recall the causal trail from error to
fault, failure, and incident. If we know what kind of errors we are prone to make, and if we know what
kinds of faults are likely to reside in the software to be tested, we can use this to employ more appropriate
test case identification methods. This is the point at which testing becomes a craft.

Error and Fault Taxonomies

Our definitions of error and fault hinge on the distinction between process and product: process refers to
how we do something, and product is the end result of a process. The point at which testing and software
quality assurance (SQA) meet is that SQA typically tries to improve the product by improving the process.
In that sense, testing is clearly more product oriented. SQA is more concerned with reducing errors
endemic in the development process, while testing is more concerned with discovering faults in a product.
Both disciplines benefit from a clearer definition of types of faults.
Faults can be classified in several ways: the development phase in which the corresponding error occurred,
the consequences of corresponding failures, difficulty to resolve, risk of no resolution, and soon. My
favorite is based on anomaly occurrence: one time only, intermittent, recurring, or [Link] 1.9
contains a fault taxonomy (Beizer, 1984) that distinguishes faults by the severity of their [Link]
a comprehensive treatment of types of faults, see the
IEEE Standard Classification for Software Anomalies
(IEEE, 1993). (A software anomaly is defined in that document as "a departure from the expected," which
is pretty close to our definition.) The IEEE standard defines a detailed anomaly resolution process built
around four phases (another life cycle): recognition, investigation, action, and disposition. Some of the
more useful anomalies are given in Table 1.1 through Table 1.5; most of these are from the IEEE standard,
but I have added some of my favorites.

Verification and Validation Testing

Verification testing
Verification testing includes different activities such as business requirements, system requirements,
design review, and code walkthroughs while developing a product.
It is also known as static testing, where we are ensuring that "we are developing the right product or
not". It also checks that the developed application fulfills all the requirements given by the client.
Validation testing
Validation testing is testing where the tester performs functional and non-functional testing.
Here functional testing includes Unit Testing (UT), Integration Testing (IT), and System Testing (ST),
and non-functional testing includes User acceptance testing (UAT).
Validation testing is also known as dynamic testing, where we are ensuring that "we have developed the
product right." It also checks that the software meets the business needs of the client.

Difference between verification and validation testing

Verification Validation

We check whether we are developing the right We check whether the developed product is right.
product or not.

Verification is also known as static testing. Validation is also known as dynamic testing.

Verification includes different methods like Validation includes testing like functional testing,
Inspections, Reviews, and Walkthroughs. system testing, integration, and User acceptance
testing.
 It is a process of checking the work-4products It is a process of checking the software during or at
(not the final product) of a development cycle to the end of the development cycle to decide whether
decide whether the product meets the specified the software follow the specified business
requirements. requirements.

Quality assurance comes under verification Quality control comes under validation testing.
testing.

The execution of code does not happen in the In validation testing, the execution of code happens.
verification testing.

In verification testing, we can find the bugs early In the validation testing, we can find those bugs,
in the development phase of the product. which are not caught in the verification process.

Verification testing is executed by the Quality Validation testing is executed by the testing team to
assurance team to make sure that the product is test the application.
developed according to customers' requirements.

Verification is done before the validation testing. After verification testing, validation testing takes
place.

In this type of testing, we can verify that the In this type of testing, we can validate whether the
inputs follow the outputs or not. user accepts the product or not.

Levels of Testing
Thus far, we have said nothing about one of the key concepts of testing— levels of abstraction. Levels of
testing echo the levels of abstraction found in the waterfall model of the software development lifecycle.
Although this model has its drawbacks, it is useful for testing as a means of identifying distinct levels of
testing and for clarifying the objectives that pertain to each level.
A diagrammatic variation of the waterfall model is given in Figure 1.10; this variation emphasizes the
correspondence between testing and design levels. Notice that, especially in terms of functional testing, the
three levels of definition (specification, preliminary design, and detailed design) correspond directly to
three levels of testing system, integration, and unit testing.
A practical relationship exists between levels of testing versus functional and structural testing. Most
practitioners agree that structural testing is most appropriate at the unit level, while functional testing is
most appropriate at the system level. This is generally true, but it is also a likely consequence of the base
information produced during the requirements specification, preliminary design, and detailed design
phases. The constructs defined for structural testing make the most sense at the unit level, and similar
constructs are only now becoming available for the integration and system levels of testing. We develop
such structures in Part IV to support structural testing at the integration and system levels for both
traditional and object-oriented software.

Examples
Three examples will be used to illustrate the various unit testing methods. They are the triangle problem (a
venerable example in testing circles); a logically complex function, NextDate; and an example that typifies
Management Information Systems (MIS) applications, known here as the commission [Link]
together, these examples raise most of the issues that testing craftspersons will encounter at the unit level.
The discussion of integration and system testing in Part IV uses three other examples: as implified version
of an automated teller machine (ATM), known here as the simple ATM (SATM)system; the currency
converter, an event-driven application typical of graphical user interface (GUI)applications; and the wind
shield wiper control device from the Saturn™ automobile. Finally, an object-oriented version of Next Date
is provided, called o-oCalendar, which is used to illustrate aspects of testing object-oriented software.
For the purposes of structural testing, pseudocode implementations of the three unit-level examples
aregiven in this chapter. System-level descriptions of the SATM system, the currency converter, and
theSaturn windshield wiper system. These applications are described both traditionally (with E/R
diagrams, dataflow diagrams, and finite state machines) and with the de facto object-oriented standard, the
Unified Modeling Language (UML).
Generalized Pseudocode
Pseudocode provides a "language neutral" way to express program source code. This version is loosely
based on Visual Basic and has constructs at two levels: unit and program components. Units can
beinterpreted either as traditional components (procedures and functions) or as object-oriented
components(classes and objects). This definition is somewhat informal; terms such as expression, variable
list, andfield description are used with no formal definition. Items in angle brackets indicate language
elementsthat can be used at the identified positions. Part of the value of any pseudocode is the suppression
ofunwanted detail; here, we illustrate this by allowing natural language phrases in place of more
formal,complex conditions (see Table 2.1).
The Triangle Problem
The triangle problem is the most widely used example in software testing literature. Some of the
morenotable entries in three decades of testing literature are Gruenberger (1973), Brown and Lipov
(1975),Myers (1979), Pressman (1982, and subsequent editions), Clarke (1983, 1984), Chellappa (1987),
andHetzel (1988). There are others, but this list makes the point.
Problem Statement
Simple version:The triangle program accepts three integers, a, b, and c, as input. These are taken to
besides of a triangle. The output of the program is the type of triangle determined by the three
sides:Equilateral, Isosceles, Scalene, or Not A Triangle. Sometimes this problem is extended to include
righttriangles as a fifth type; we will use this extension in some of the exercises.

Discussion
Perhaps one of the reasons for the longevity of this example is that it contains clear but complex logic.
Italso typifies some of the incomplete definitions that impair communication among customers,developers,
and testers. The first specification presumes the developers know some details abouttriangles, particularly
the triangle inequality: the sum of any pair of sides must be strictly greater than the third side. The upper
limit of 200 is both arbitrary and convenient; it will be used when we develop boundary value test cases.
Traditional Implementation
The "traditional" implementation of this grandfather of all examples has a rather Fortran-like style.
Theflowchart for this implementation appears in Figure 2.1. The flowchart box numbers correspond
tocomment numbers in the (Fortran-like) pseudocode program given next.
The variable "match" is used to record equality among pairs of the sides. A classical intricacy of the
FRORTRAN style is connected with the variable "match": notice that all three tests for the triangle quality
do not occur. If two sides are equal, say, a and c, it is only necessary to compare a + c with b. (Because b
must be greater than zero, a + b must be greater than c because c equals a.) This observation reduces the
number of comparisons that must be made. The efficiency of this version is obtained at the expense of
clarity (and ease of testing). Notice that six ways are used to reach the NotATriangle box, and three ways
are used to reach the Isosceles box.
Structured Implementation
Figure 2.2 is a dataflow diagram description of the triangle program. We could implement it as a main
program with the three indicated procedures. We will use this example later for unit
testing; therefore, the three procedures have been merged into one pseudocode program. Comment lines
relate sections of the codeto the decomposition given in Figure 2.2