Unit VI : PRODUCT METRICS

Metrics for source code :

Halstead’s theory of “software science” [Hal77] proposed the fi rst analytical “laws”
for computer software. 14 Halstead assigned quantitative laws to the development
of computer software, using a set of primitive measures that may be derived after
code is generated or estimated once design is complete.

The measures are:

n1 = number of distinct operators that appear in a program
n2 = number of distinct operands that appear in a program
N1 = total number of operator occurrences
N2 = total number of operand occurrences
Halstead uses these primitive measures to develop expressions for the overall
program length, potential minimum volume for an algorithm, the actual volume
(number of bits required to specify a program), the program level (a measure of
software complexity), the language level (a constant for a given language), and
other features such as development effort, development time, and even the projected
number of faults in the software.

Halstead shows that length N can be estimated

N = n1 log2 n 1 + n2 log2 n2

and program volume may be defined

V = N log2 ( n1 + n2)

It should be noted that V will vary with programming language and represents the
volume of information (in bits) required to specify a [Link], a
minimum volume must exist for a particular algorithm. Halstead defines a volume
ratio L as the ratio of volume of the most compact form of a program to the volume
of the actual program. In actuality, L must always be less than 1. In terms of primitive
measures, the volume ratio may be expressed as :

L = 2 / n1 * n2 / N2.
Metrics for Testing :

The majority of metrics proposed for testing focus on the process of testing, not
the technical characteristics of the tests themselves. In general, testers must rely
on analysis, design, and code metrics to guide them in the design and execution
of test cases.

Architectural design metrics provide information on the ease or diffi culty associated
with integration testing and the need for specialized testing software (e.g., stubs and
drivers). Cyclomatic complexity (a component-level design metric) lies at the core
of basis path testing.
In addition, cyclomatic complexity can be used to target modules as candidates for
extensive unit testing. Modules with high cyclomatic complexity are more likely to
be error prone than modules whose cyclomatic complexity is lower. For this reason,
you should expend above average effort to uncover errors in such modules before
they are integrated in a system.

Testing effort can be estimated using metrics derived from Halstead measures. Using
the definitions for program volume V and program level PL,

Halstead effort e can be computed as

PL = 1 / [(n /2) * (N2 / n2)]

e = V / PL

The percentage of overall testing effort to be allocated to a module k can be estimated

using the following relationship:

Percentage of testing effort ( k ) :

e(k)
_____________
summation( i )
Metrics for Object-Oriented Testing

The OO design metrics provide an indication of design quality. They also provide a
general indication of the amount of testing effort required to exercise an OO system.
It suggests a broad array of design metrics that have a direct influence on the
“testability” of an OO system. The metrics consider aspects of encapsulation and
inheritance.

Lack of cohesion in methods (LCOM) : The higher the value of LCOM, the more
states must be tested to ensure that methods do not generate side effects.

Percent public and protected (PAP) : Public attributes are inherited from other
classes and therefore are visible to those classes. Protected attributes are accessible
to methods in subclasses. This metric indicates the percentage of class attributes that
are public or protected. High values for PAP increase the likelihood of side effects
among classes because public and protected attributes lead to high potential for
coupling. “Tests must be designed to ensure that such side effects are
Uncovered”.

Public access to data members (PAD) : This metric indicates the number of classes
(or methods) that can access another class’s attributes, a violation of encapsulation.
High values for PAD lead to the potential for side effects among classes. Tests must
be designed to ensure that such side effects are uncovered.

Number of root classes (NOR) : This metric is a count of the distinct class
hierarchies that are described in the design model. Test suites for each root class and
the corresponding class hierarchy must be developed. As NOR increases, testing
effort also increases.
Metrics For The Design :

It is inconceivable that the design of a new aircraft, a new computer chip, or a new
office building would be conducted without defi ning design measures, determining
metrics for various aspects of design quality, and using them as indicators to guide
the manner in which the design evolves. And yet, the design of complex software-
based systems often proceeds with virtually no measurement. The irony of this is
that design metrics for software are available, but the vast majority of software
engineers continue to be unaware of their existence.

Design metrics for computer software, like all other software metrics, are not perfect.
Debate continues over their efficacy and the manner in which they should be applied.
Many experts argue that further experimentation is required before design measures
can be used. And yet, design without measurement is an unacceptable alternative.

Architectural Design Metrics

Architectural design metrics focus on characteristics of the program architecture

with an emphasis on the architectural structure and the effectiveness of modules or
components within the architecture. These metrics are “black box” in the sense that
they do not require any knowledge of the inner workings of a particular software
component.
( Fan-In: How many components depend on this one.
→ High fan-in = possibly a reusable utility.

 Fan-Out: How many components this one depends on.

→ High fan-out = possibly tight coupling.)
Metrics For The Requirements Model
Technical work in software engineering begins with the creation of the requirements
model. It is at this stage that requirements are derived and a foundation for design is
established. Therefore, product metrics that provide insight into the quality of the
analysis model are desirable.

Although relatively few analysis and specification metrics have appeared in the
literature, it is possible to adapt metrics that are often used for project estimation and
apply them in this context. These metrics examine the requirements model with the
intent of predicting the “size” of the resultant system. Size is sometimes (but not
always) an indicator of design complexity and is almost always an indicator of
increased coding, integration, and testing effort.
Function-Based Metrics :

The function point (FP) metric can be used effectively as a means for measuring
the functionality delivered by a system. 4 Using historical data, the FP metric can
then be used to :
(1) estimate the cost or effort required to design, code, and test the software;
(2) predict the number of errors that will be encountered during testing;
(3) forecast the number of components and/or the number of projected
source lines in the implemented system.

Function points are derived using an empirical relationship based on countable

(direct) measures of software’s information domain and qualitative assessments
of software complexity. Information domain values are defined in the following
manner:
Number of external inputs (EIs) :

Each external input originates from a user or is transmitted from another application
and provides distinct application-oriented data or control information. Inputs are
often used to update internal logical fi les (ILFs). Inputs should be distinguished
from inquiries, which are counted separately.
Number of external outputs (EOs) :

Each external output is derived data within the application that provides information
to the user. In this context external output refers to reports, screens, error messages,
and the like. Individual data items within a report are not counted separately.
Number of external inquiries (EQs) :

An external inquiry is defined as an online input that results in the generation of

some immediate software response in the form of an online output (often retrieved
from an ILF).

Number of internal logical files (ILFs) :

Each internal logical file is a logical grouping of data that resides within the
application’s boundary and is maintained via external inputs.

Number of external interface files (EIFs) :

Each external interface file is a logical grouping of data that resides external to the
application but provides data that may be of use to the application.
A Framework For Product Metrics

Measurement assigns numbers or symbols to attributes of entities in the real world.

To accomplish this, a measurement model encompassing a consistent set of rules is
required.
Measures, Metrics, and Indicators
Within the software engineering context, a measure provides a quantitative
indication of the extent, amount, dimension, capacity, or size of some attribute of a
product or process. Measurement is the act of determining a measure.

When a single data point has been collected (e.g., the number of errors uncovered
within a single software component), a measure has been established. Measurement
occurs as the result of the collection of one or more data points (eg: a number of
component reviews and unit tests are investigated to collect measures of the number
of errors for each). A software metric relates the individual measures in some way
(e.g., the average number of errors found per review or the average number of errors
found per unit test).

A software engineer collects measures and develops metrics so that indicators will
be obtained. An indicator is a metric or combination of metrics that provides insight
into the software process, a software project, or the product itself. An indicator
provides insight that enables the project manager or software engineers to adjust the
process, the project, or the product to make things better.

Measurement Principles

The following principles [Let03b] are representative of many that can be proposed
for metrics characterization and validation:

• A metric should have desirable mathematical properties. That is, the metric’s
value should be in a meaningful range (e.g., 0 to 1, where 0 truly means absence, 1
indicates the maximum value, and 0.5 represents the “halfway point”). Also, a metric
that purports to be on a rational scale should not be composed of components that
are only measured on an ordinal scale.

• When a metric represents a software characteristic that increases when positive

traits occur or decreases when undesirable traits are encountered, the value of the
metric should increase or decrease in the same manner.
• Each metric should be validated empirically in a wide variety of contexts before
being published or used to make decisions. A metric should measure the factor of
interest, independently of other factors. It should “scale up” to large systems and
work in a variety of programming languages and system domains.

Although formulation, characterization, and validation are critical, collection and

analysis are the activities that drive the measurement process. Roche [Roc94]
suggests the following guidelines for these activities: (1) whenever possible, data
collection and analysis should be automated; (2) valid statistical techniques should
be applied to establish relationships between internal product attributes and external
quality characteristics (e.g., whether the level of architectural complexity
correlates with the number of defects reported in production use); and (3)
interpretative guidelines and recommendations should be established for each
metric.