Chapter 7

Each verification activity is a phase of the testing life cycle. The testing objective in each
verification activity is to detect as many errors as possible. The testing team should
leverage its efforts by participating in any inspections and walkthroughs conducted by
development and by initiating verification, especially at the early stages of development.

In addition, the testing team should develop its own verification "test- ware" in the form of
generic and testing-specific checklists for all kinds of documents, so that verification
expertise becomes an asset of the organization. The testware itself, like all other software,
should also be verified.

Testing should use verification practices as a springboard for improving interdisciplinary

communication about essential matters as well as promoting the general maturity of the
development environment.

You should do as much and as thorough a verification as possible. It has proven to be one
of the surest and most cost-effective routes to quality improvement in both the short and
the long term.

Basic verification methods

Verification is a "human" examination or review of the work product. There are various
types of reviews. Inspections, walkthroughs, technical reviews and other methods are not
always referred to consistently, but inspections are generally considered the most formal.

Formal structured types of verification

Formal reviews, technical reviews, and inspections are various expressions used for the
more structured types of verification. In the following sections we shall refer mostly to
inspections, simply because they are the most structured, but this does not imply the
exclusion of the other methods or that we question their value.

The central "event" in these methods is a meeting at which defects in the product under
review are discussed and discovered.

Formal methods - key elements

(1) Everyone in the review group participates openly and actively, and participation is
governed by traditions, customs and written rules about how such a review is to be
conducted.

(2) A written report is produced regarding the status of the product, and the report is
available to everyone involved in the project, including management. (3) The review group
is responsible for the quality of information in the written report.

(Freedman and Weinberg, 1990)

Inspections are also characterized by individual preparation by all par- ticipants prior to
the meeting. Participants are told by the inspection team leader what their reviewer role is,
i.e., from what point of view to read the material. When performing inspections well, most
of the total defects to be discovered are found during preparation, though an effective
inspection meeting will usually uncover some significant additional defects.

Inspection: key elements and phases

Objectives:

to obtain defects and collect data; meeting does not examine alternative solutions;

to communicate important work product information.

Elements:

a planned, structured meeting requiring individual preparation by all participants;

a team of 3-6 people, led by an impartial moderator;

presenter is "reader" other than the producer.

Input:

document to be inspected;

related source documents;

general and "tailored" checklists.

Output:

inspection summary/report;

data on error types.

Key phases:

briefing/entry - individual preparation - inspection meeting - edit/ rework-exit/re-inspect -

collect data;

some versions of inspection include causal analysis meetings.

The presenter in an inspection meeting is ideally not the producer or author of the
document being inspected. Not only does this oblige someone other than the author to
look at the product thoroughly, but the meeting audience gets presented with a potentially
different interpretation of the material. The other people in the meeting are the inspectors,
representing different viewpoints.

Testers participating in any kind of review see the product differently than developers:
"How is this thing going to work? If I test it this way, what's going to happen?" As testers we
are working out what problems we're going to find when we test it. Our attention is drawn
to weak areas of which we might not otherwise have been aware. That is why it is so
important that testers are part of the inspection or review process - they bring a unique
view- point to the meeting.

Any work product that's important can and should be inspected - a project plan, an
engineering drawing, a user manual, a test plan. Appendix B includes a generic checklist
for use on any document.

Walkthroughs
Walkthroughs are less formal than inspections mainly because of the lack of preparation.
In walkthroughs the participants simply come to the meeting; the presenter prepares
(usually the author of the product), and there's no additional effort by the participants
prior to the meeting.

Walkthroughs: key elements

Objective:

to detect defects and to become familiar with the material. Elements:

a planned meeting where only the presenter must prepare;

a team of 2-7 people, led by the producer/author;

the presenter is usually the producer.

Input:

element under examination, objectives for the walkthrough, applicable standards.

Output: report.

Walkthroughs can cover more material than inspections and reviews because the
presenter is the producer, and the other participants do not have a heavy participating
work load. They therefore provide an opportunity for larger numbers of people to become
familiar with the material.

Occasionally, walkthroughs are used for purposes of communication rather than for
discovering defects. Software may be "inherited." We don't know exactly what's there so
we organize a walkthrough to go through it, page by page, with the key people in the room.
If defects are found, that's fine, but the main goal is to familiarize ourselves with the
product.

The disadvantage of walkthroughs is that a review tends to be less objective when the
presenter is the producer.

From the point of view of finding defects, preparation by participants in inspections and
formal reviews usually raise more penetrating issues.

Buddy checks

Any form of human testing, even undisciplined testing, is better than none, provided it is
performed by someone other than the author and its objective is to detect defects. There
may, for instance, be occasions when it is inappropriate or impossible to get material
inspected or formally reviewed.

However, simply giving a document to someone else and asking them to look at it closely
will turn up defects we might never find on our own. Some studies show that such desk
reviews can be extremely efficient, and they can be a good training ground for finding
errors in the product, not the person.

Getting leverage on verification

What (and how much) verification to do?

We have to ask ourselves:

How many defects are we finding as a result of these reviews?

How many defects are getting through reviews and getting found in later validation testing?

What percentage of them are left at the end of testing and only being found by customers?

Verification activities are implemented in an order where the size of the work product, its
level of detail, and the cost to verify it are increasing, while the potential payoff is
decreasing. This means that, if resources or schedule limitations preclude any of the
verification activities (and they almost always will), candidates for elimination should be
considered in the reverse order of their occurrence.

Requirements verification offers the biggest potential saving to software development

efforts. It can detect many deficiencies that can otherwise go undetected until late in the
development cycle, where correction is much more expensive because problems have
migrated to other phases. Furthermore, the requirements stage is often where more than
50% of the defects are actually introduced.

In general, formal reviews and inspections are recommended. They are more expensive in
the short term, but if done properly, the benefits will always outweigh the costs. They will
detect a significant percentage of the errors, and the inherent data collection method is a
useful metric for develop- ment performance. The success of these methods, and their
cost effectiveness, can be measured.

Because they are time consuming and require intense concentration, inspections typically
deal with relatively low volumes of material. Like all testing activities, verification of large
work products will not be exhaustive and will usually involve risk assessments and trade-
offs.

It can be a good idea to "mix and match" verification methods. There are times when it is
appropriate to say: "This is the part of the code that's critical to us; it's the heart of the
system, so we'll do a proper inspection on it. We're going to get several carefully chosen
people and go through every line of the specification." Sampling of important documents
can also be useful. Inspecting samples can provide a good estimate of the quality of the
document and the number of errors in the uninspected parts of it.

On the other hand we may say: "Here's another part of the code that's not so important. It
is not the heart of the system, and we don't want to invest in an inspection." In this case,
we may want to do a less formal review or desk check or an informal walkthrough or
variations on these. There are always trade-offs, and this is where risk analysis comes into
play (see Chapter 5).

Verification is almost always more effective overall than validation testing. It may find
defects that are nearly impossible to detect during validation testing. Most importantly, it
allows us to find and detect the defects at the earliest possible time.

In most organizations the distribution of verification/validation defects is 20/80, or even

less for verification. As a general strategy, we should be working toward a higher
proportion of verification, though this can entail a major cultural change. Try verification in
one new area, one that shows results fairly quickly and easily, to get people's support and
enthusiasm. Then do a little bit more, slowly shifting the percentages.

Checklists: the verification tool

An important tool for verification, especially in more formal forms of verification like
inspections, is the checklist. There are generic checklists that can be applied at a high
level and maintained for each type of inspection. In other words, there is a checklist for
requirements, a checklist for functional design specifications, a checklist for internal
design specifications, a checklist for test plans. We can develop our own checklists for
anything that we may want to

review.

Sample generic checklists

The following checklists are included in Appendix B:

Requirements verification checklist

Functional design verification checklist

Internal design verification checklist

Generic code verification checklist (1)

Generic code verification checklist (2) Code verification checklist for "C"

Code verification checklist for "COBOL" Generic document verification checklist

Development and testing often have different checklists. Testing check- lists tend to be
orientated towards reliability and usability of the product. Development checklists are
usually more focused on maintainability and things like guidelines for coding standards.

What is important is to make use of and build on generic checklists, but also to develop
our own checklists within our organization for specific purposes and specific projects.
These checklists should reflect our chosen focus and our particular present level of
maturity in verification testing.

Checklists are an important part of testware. To get maximum leverage on verification,

they should be carefully kept, improved, developed, updated - and someone has to take
responsibility for this. They are a vital tool for verification testing; they are an important
training device; they ensure continuity of the verification effort through different projects
and different personnel, and they provide a record of the organization's progress in
verification.

Verifying documents at different phases

Verifying requirements

The purpose of the requirements phase is to ensure that the users' needs are properly
understood before translating them into design. Requirements are difficult to develop
because it is hard to distinguish needs from wants. They will usually change in later
phases and have always been the weakest link in the software chain. What do we mean by
requirements and what kind of questions are we asking when we do requirements
verification?

What's the capability that's needed by the user? What are we trying to provide for the
customer? What do they want? The answer is a statement of requirement. The IEEE/ANSI
definition is:

A requirement is a condition of capability needed by a user to solve a problem or achieve

an objective.

A requirement is a condition of capability that must be met by or possessed by a system or

system component to satisfy a contract, standard, specification, or other formally
imposed documents.

Requirements may be expressed in formally composed documents or in an informal

communication that defines the users' needs. They may be explicit, or implicit, but they
are always there, and the purpose of this phase of our testing effort is to ensure that the
users' needs are properly understood before we go any further.

One difficulty is to keep requirements strictly separate from solutions to those

requirements. These two are frequently confused in documents produced at the early
stages of the development process. The other difficulty is to realize that, even in the best
of worlds, the requirements are going to change.

In the early days as testers we used to say, "We have to freeze the requirements. We have
to get agreement that the requirements will never change, and then make sure that they
never do." However, this is unrealistic because in real-world projects requirements do
change. What we have to learn is to handle and control the change through proper
verification of requirements and configuration management of the various versions (see
Chapter 6) and provide ourselves with a solid framework for later testing.

The properties of good requirements specifications

The following are the properties that all good requirements specifications should have:
visible

clear (unambiguous)

complete

consistent (conflicting requirements must be prioritized) • reasonable (achievable)

measurable (quantifiable, testable)

modifiable

traceable

dependent requirements identified.

The IEEE/ANSI guide to software requirements specifications is a very useful reference. It

helps customers identify what they want; it helps suppliers understand what they have to
provide, and it helps individuals involved with requirements to understand what they're
trying to accomplish.
Help from IEEE/ANSI

IEEE/ANSI Standard 830-1993, Recommended Practice for Software Requirements

Specifications, helps:

- software customers to accurately describe what they wish to obtain; software suppliers
to understand exactly what the customer wants; individuals to accomplish the following
goals:

- develop standard software requirements specifications outline for their own

organizations;

define the form and content of their specific software requirements specifications;

- develop additional local supporting items such as requirements quality checklists or a

requirements writers' handbook.

Frequently, the quality of a requirements document is a function of the producer, their

skills, whether they know how to write requirements, and whether they are familiar with
the standards. The quality of requirements is an important indicator of the level of maturity
of an organization. If there is no life cycle, if there is no agreement that there will be a
requirements document and there's no agreement on who writes it, what should be in it,
or even what are the frameworks for acceptable requirements, development will be
difficult and proper testing will be extremely difficult.

When we review requirements, we should be looking for basic functionality, but we need
more than that. We may, for instance, need some definitions. Requirements frequently
contain important terms and everybody assumes that we know what they mean.

Requirements checklist - sample items.

The following is an extract from a generic requirements verification checklist:

Precise, unambiguous, and clear

Each item is exact and not vague; there is a single interpretation; the

meaning of each item is understood; the specification is easy to read. Consistent

No item conflicts with another item in the specification.

Relevant

Each item is pertinent to the problem and its eventual solution.

Testable

During program development and acceptance testing, it will be possible to determine

whether the item has been satisfied.

Traceable

During program development and testing, it will be possible to trace each item through the
various stages of development.

A more complete requirements verification checklist is in Appendix B.

Requirements may make frequent reference to vital, essential issues like security,
usability, maintainability, and performance that are entirely non- specific or unexplained.
When we're reading requirements from a testability standpoint all this vagueness is
revealed for what it is. It's hard to test what constitutes "very good performance." What we
can do is come back and say, "Well what did you mean by 'very good'; is that two seconds
response time; does it mean 24 hours - or what?" The concept of testability is closely
linked to the concept of measurability. Generally, the more quantifiable the requirements,
the simpler it is to derive tests for the success of the system that results from them.
Testers can provide requirements writers with the framework for the kinds of items that
need to be there, like performance specifications, or usability metrics, or maintainability
standards.

The testers' viewpoint on requirements is to look for anything that can be a problem. Is it
unambiguous, complete, consistent, reasonable? Does it seem achievable? Is it
traceable? Is it measurable from a testing standpoint? Verifying requirements provides
fertile ground for development improvements as well as later testing. Asking "how can I
test this?" will often produce better solutions from product designers as well as ideas
about what and how to test.

Exercise on requirements verification

•Demonstrate the effectiveness of requirements verification by trying it out on the

requirements document in Appendix C, a simple requirements

specification for a reservation system. It can be verified with anything from a one-man
"buddy check" to a full-scale inspection. Remember that successful verification takes
time.

Use the requirements verification checklist in Appendix B. Add to the checklist any new
items that should be checked in future documents on the basis of this particular
verification.

. Record the time taken to do the exercise.

•Estimate the time/cost consequences if the errors found had been allowed to migrate to
later stages of development, or even to the user.

Solution: The consolidated notes on this document follows the exercise. It represents four
groups of testing practitioners reviewing this document for approximately 30 minutes.

The requirements have been inspected; the changes have been made; the defects have
been handled; the requirements have been signed off as approved by the review group. We
now have a good basis for test design, for change negotiations and for validation. As a
result of participating in require- ments verification, we have already learned something
about the product and can now plan for validation testing and how it's going to work. We're
already ahead of the game!

Verifying the functional design

Functional design is the process of translating user requirements into the set of external
(human) interfaces. The output of the process is the functional design specification, which
describes the product's behavior as seen by an observer external to the product. It should
describe everything the user can see and should avoid describing what the user cannot
see. It is eventually translated into an internal design as well as user manuals. It should
not include internal information, internal data structures, data diagrams or flow diagrams;
they belong in the internal design specification which is the next step in the process.

How is verifying functional design different than verifying the require-ments? If

requirements are the most important, functional design is the next most important, simply
because it is early in the process.

Functional design checklist - sample items

The following is an extract from a generic functional design verification checklist:

When a term is defined explicitly somewhere, try substituting that definition in place of the
term.

When a structure is described in words, try to sketch a picture of the structure being
described.

When a calculation is specified, work at least two examples by hand and give them as
examples in the specification.

When searching behind certainty statements, push the search back as many levels as are
needed to achieve the kind of certainty a computer will need. Watch for vague words, such
as some, sometimes, often, usually, ordinarily, customarily, most, or mostly.

A more complete functional design verification checklist is in Appendix B.

The goal in verifying a functional design is to determine how success- fully the user
requirements have been incorporated into the functional design. The concept of
traceability starts to operate here. We have requirements and we use the requirements
document again at this stage as a source document for verifying the functional design.
Every paragraph in the requirements should be reflected in the functional design
specifications. If it's not, maybe it was dropped completely (but where is this recorded?)
or maybe someone sim- ply forgot to implement it.

One of the most common failings of the functional design specifications is

incompleteness. Good inspectors or reviewers don't just read what's in front of them. We
have to constantly ask: "What's missing?" and keep asking ourselves what should have
been written on the page. Try to imagine, if you had been writing the document, what you
would have included before you read the section. How should this be described
functionally? How should it look to the end user? Pretending to be the designer writing the
document enables you to see some of the errors of omission. These are the most import-
ant errors to find.

The requirements document itself often has a number of sources, such as standards,
correspondence, minutes of meetings, etc. If so, that's where trace- ability starts. It is also
important to look out for unwarranted additions.
Exercise on verifying functional design specification

- Demonstrate the effectiveness of functional design verification by trying

it out on the functional design document in Appendix C, a functional design specification

for a sales system.

- Use the functional design verification checklist in Appendix B.

Solution: The consolidated notes on this document follows the exercise. It represents four
groups of testing practitioners reviewing this document for approximately 30 minutes.

Verifying the internal design

Internal design is the process of translating the functional specification into a detailed set
of data structures, data flows, and algorithms. The output of the process is the internal
design specification which shows how the product is to be built. Multiple internal design
specifications, representing successive levels of abstraction, may be produced. If
possible, each of them should be verified.

Internal design checklist - sample items

The following is an extract from a typical internal design verification checklist:

Does the design document contain a description of the procedure that was used to do
preliminary design or is there a reference to such a procedure?

Is there a model of the user interface to the computing system?

Is there a high-level functional model of the proposed computing system? Are the major
implementation alternatives and their evaluations represented in the document?

A more complete internal design verification checklist is in Appendix B.

There is a recommended practice for software design descriptions from IEEE/ANSI in the
software engineering standards documentation. It has recommendations for locating
information, formats, and ways of organizing the material.

Help from IEEE/ANSI

IEEE/ANSI Standard 1016-1987, IEEE Recommended Practice for Software Design

Descriptions (Reaff. 1993), specifies the necessary information content and recommends
an organization for software design descriptions. See Appendix A.

Internal design specifications are invaluable to have as a testing perspective. Seeing how
the product is going to be built and thinking how the whole system is going to come
together enables testers to design additional internals-based tests.

Reviewing internal design involves using checklists, tracing the path back to functional
design and back to requirements, and trying to see whether we agree with the algorithms
and how they're being put together. We're searching for defects, but we're also thinking,
"How would we test that?"

For example, if there's a table that's used in part of the internal design, testers can start
asking some questions which are pertinent for testing: "How big is the table? Why are
there 25 entries? How do we fill that table? How do we overflow that table? What happens
if nothing goes in that table?"

Such questions will provoke good testing ideas. As soon as we start to look at a
description of an internal document, the limits within the product become clearer. It
shows up boundary conditions; it will warn us about performance and possible failure
conditions and all kinds of other internal considerations. From a testing standpoint,
whether there are formal internal design specifications or whether we do reviews of them,
we can learn a lot about the product by getting this information and thinking about it from
a testing perspective.

Verifying the code

Coding is the process of translating the detailed design specification into a specific set of
code. The output of the process is the source code itself. This is often the place where
companies start when they begin doing walkthroughs and inspections. Sometimes it is the
most comfortable place to commence, which is all right as long as we realize it's not the
most efficient place to be. Once people are doing walkthroughs and inspections on code,
and getting comfortable with the process and learning how to exploit it, we can move them
toward reviewing the documents that exist long before there is any code. After all, if we're
coding to a poor specification and to the wrong requirements, we've already wasted a lot
of time!

Verifying the code involves the following activities:

(1) Comparing the code with internal design specifications.

(2) Examining the code against a language-specific checklist.

(3) Using a static analysis tool to check for compliance with the syntactic/content
requirements.

(4) Verifying the correspondence of terms in code with data dictionary and with internal
design specification.

(5) Searching for new boundary conditions, possible performance bottle- necks, and other
internal considerations which may form the basis for additional validation tests.

Some companies do a formal code review. Participants are given the code in advance;
they read through it, look for defects, come to a meeting, walk the code step by step and
testers are there helping to find defects and asking for clarification. On the other hand it
can be very informal. It can simply be a buddy check, where one person just looks at
another person's code and marks the errors in it. On a small scale it can work very
effectively.

Code checklist-sample items

The following are typical headings with a single example under each from a generic code
verification checklist:

Data reference errors

Is an unset or unitialized variable referenced?

Data declaration errors

Are there variables with similar names?

Computation errors

Is the target variable of an assignment smaller than the right-hand expression?

Comparison errors

Are the conversion rules for comparisons between data or variables of inconsistent type or
length handled?

Control flow errors

Is there a possibility of premature loop exit?

Interface errors

If the module has multiple entry points, is a parameter ever referenced that is not
associated with the current point of entry?

Input/output errors

Are there grammatical errors in program output text?

Portability

How is the data organized (e.g., packed structures)?

Complete generic and particular code verification checklists are in Appendix B.

We should always be thinking from a testing perspective. Whenever we look at a piece of

code, we will think of new tests that we could not have thought of by only reading the
requirement or the functional design specification. Often we can spot the error conditions
that will provoke failure when the code is first executed. Frequently, developers do straight
line checking which does not find these kinds of errors. They will give the code to testing,
but the first time the path is executed the whole system crashes.

If code verification is being done more formally, existing checklists in the organization can
be used, or a generic checklist can be used as a starting point (see Appendix B) for
building a customized version.

Getting the best from verification

The author

People who have their work reviewed in public see themselves in a hot seat, and naturally
tend to get defensive. It's important to work hard and consistently for the team spirit
attitude: "We are here to find defects in our work products, not to attack individuals for the
failures in their part of it." As authors, we should be aware that we are in a position that we
may find hard to handle at first, but we have everything to gain from having our work
reviewed in this way. Next time around when we're on the team, we should remember to
treat the author as we would like to be treated when our work is being reviewed.
As a member of the inspection team, avoid discussions of style. We are there to evaluate
the product, not its producer. It's tempting to get diverted into "I would have said it this
way," or "It would be much better if you wrote it this way instead of that way." Style issues
are not about the content of the information, but the way it has been written. There may,
however, be legitimate issues of clarity or definition of terms which ultimately affect the
content.

Whatever level of formality is being used in verification, be tactful, be reasonable, and be

sensitive to other people's egos. Do any preparation required for inspections properly, and
at the inspection meeting, raise issues - don't try to solve them.

If we can develop these attitudes successfully within the testing organization (and acting
as an example goes a very long way), we will be the instigators of a critical culture change.

The development team

Reviews and inspections can help with communication and with motivation, sometimes
through a sense of pride or even embarrassment. If people know their work product is
going to be inspected they tend to do a better job.

Feedback can be very positive. There are times in review meetings when
producers/authors get a sense of "that looks good, that's a good way to do it" from the
team, and it becomes, without being formally adopted, common good practice within the
organization. Feedback sharpens people's level of performance in a constructive way, and
in six months to a year, we'll get a reduction in defects, just because we know our material
is going to be under scrutiny the next time there is a review.

The other issue that is really important is communication. We don't meas- ure our success
in inspection purely by defects that are being found. Inspection is also about improving the
software development process. One of the things that contributes to the value of the
inspection is that people start communicating about things that they need to know to do
their jobs properly.

For example, being on an inspection team enables testers to find out basic information
about the product that they didn't have before. This communication value of inspection
should never be ignored.

Verification, especially at the early stages, can help with communication not only within
the development environment, but throughout the organization. If we are doing
requirements verification, for instance, ideally we need the marketing people, as well as
the test people and development people there, and maybe support people as well. It's a
golden opportunity to spread understanding and better communication to a wider group.

The inspection team

Inspection is difficult because there is no detailed cookbook method for any given type of
work product. Each type of work product requires different expertise and thought
processes. Critical thinking, which must often transcend even the most robust and well-
defined inspection checklists, is required.
Inspection also requires intense concentration and is very fatiguing if overused. It requires
the ability to detect omissions. Most people react only to what they see in front of them.
The best reviewers and inspectors must ask "What's missing?" or "What should be written
here that isn't?" The tough thing is to ask what should have been on the page but isn't
there. Good reviewers, especially good inspection team people, are worth their weight in
gold in organizations. After a while, everybody knows who they are, and they are constantly
in demand for any kind of verification activity.

Cost-effective verification

Verification is just as productive and important as validation testing, if not more so. It has
the potential to detect defects at the earliest possible stage, thereby minimizing the cost
of correction. But is the verification process itself cost effective? Do we save more than we
spend by finding errors early? The short answer is that it has been demonstrated again and
again that verification, although it isn't free, is cost effective.

We should therefore try to verify critical documents, or at least some parts of them. On a
large work product, we will almost certainly feel we can't do all the ideal complete code
reviews and formal specification and design reviews. We may want to use walkthroughs
for some parts, instead of having formal reviews, and we may want to do desk checks for
some items. These are all decisions based on the particular case.

We can increase the cost effectiveness of verification by having a good configuration

management system (see Chapter 6). Somebody must make sure we're not verifying the
wrong thing at the wrong time, and that what we verify is the same thing that we validation-
test and that we eventually ship.

three critical success factors for implementing

verification

Success factor 1: process ownership

If we are not already using verification or we need to improve the way we are doing it, the
process needs a champion. The champion can be anybody who really is interested in it,
who cares a lot about it, and who will take ownership for the process and will make it
happen. It could be someone in the development organization, who knows inspections are
a good thing. It could be a quality assurance person; it could be a process expert, or
someone in the process engineering group. It could be a full-time job or it could be a part-
time job. When companies get really big they hire people to do nothing but organize formal
verification processes. In smaller companies this is neither necessary nor possible. The
important thing is that somebody must become the champion of the method and work in
the medium and long term to gradually gain support for it.

Success factor 2: management support

Frequently, it is management that initiates the adoption of inspections. Whether or not

they are initiating the process, it is important that managers are well briefed on
inspections and their benefits. It is unreasonable to expect them to spend resources and
support the effort if they don't see the long-term gains. It can be difficult for anyone with
different pressures and different immediate problems to solve to understand why it's
worth putting resources into these early testing efforts.

You should promote support for inspections by getting out early results. If possible, collect
data on the errors found by verifying at the early stages and compare this to the estimated
costs if they had been allowed to migrate to the later stages. Get people who aren't so
enthusiastic about these methods to a meeting to discuss problems with a particular work
product, and then demonstrate with a small, live inspection how effective inspections are
in their own organization and on their own material. Show them it's possible to monitor the
cost effectiveness of the new process.

Success factor 3: training

Training in reviews and inspections is crucial, including specific training for practitioners
on how to perform reviews and inspections, including costs, benefits, and dealing with the
human and cultural issues. Training should also include performing inspections in a
workshop setting of real work products from the local environment. Everyone who is going
to be inspecting documents

should be trained. Where this is not possible, an experienced team with a good grasp of
the human issues involved can absorb a new member.

Help from SIRO

Software Inspection and Review Organization (SIRO)

This organization was formed to exchange new ideas and information about group-based
software examinations. It facilitates emerging inspection and review techniques, provides
a clearing house for resources, and surveys/reports on the current use of inspections and
reviews.

Contact:

SIRO

PO Box 61015

Sunnyvale, CA 94088-1015

Recommendations

Inspections are recommended first and foremost because they have proven to be the
most effective, most practical method of verification. Inspections are probably the oldest
methodology in the history of software development. They have been used successfully
for at least 25 years. So while inspection techniques have evolved over this time, you won't
be suggesting untried leading edge methods when you suggest implementing inspections
in your organization. They work. Getting the right people in a room to look at the right
material in a systematic, objective way is simply common sense and good basic
communication.

There's a real trade-off between verification and validation testing. In doing verification we
begin to ease the pain in validation testing. Most testing organizations experience "The
Chaos Zone" when they only do functional testing or system testing. At that point, all of the
code, of which testing has little or no prior knowledge, is thrown into the testing
department. All kinds of defects are found through hasty testing in unfamiliar territory.
Shipment dates are missed and pressure is created because testers are operating in an
un- familiar environment where there should be an opportunity to get acquainted with the
product and find many defects a lot earlier.

Get started by inspecting some key material. Typically, an organization will have all kinds
of projects all at different stages. Some projects may be so far along that it doesn't make
sense to implement verification on them.

It is usually better not to go for a "big bang" approach and implement formal inspections
on all documentation from a certain point in time. It's best to pick some high-risk material
with a high pay-off on a number of different projects first. Perhaps start by reviewing all
new requirement specifications or all new and changed code on critical projects. Build
support while measuring and tracking enthusiasm from peers. Then do inspections on a
wider range of documentation and demonstrate the results.

Show management or colleagues who don't know about inspections the results of the
requirements and functional design verification exercises in this chapter. Be sure to
include the time resources used and the estimated con- sequences of allowing the errors
found to migrate to later stages.