Arba Minch Institute of Technology

Faculty of Computing and Software Engineering

SoftwareMetrics
Chapter 3
A GOAL-BASED FRAMEWORK FOR SOFTWARE
MEASUREMENT

Compiled By: Aliazar D.

BSc. & MSc. in Software Engineering
Ph.D. Candidate in Computing & IT
Email: [Link]@[Link]
 CLASSIFYING SOFTWARE MEASURES

• The framework for software measurement is based on

three principles:
1. Classifying the entities to be examined.
2. Determining relevant measurement goals.
3. Identifying the level of maturity that the
organization has reached.

2
 ..Conti

• The first obligation of any software measurement activity is

identifying the entities and attributes that we want to measure.
• Software entities can be classified as follows:
 Processes: Collection of software-related activities.
 Products: Artifacts, deliverables or documents that
result from a process activity.
 Resources: Entities required by a process activity.

3
 ..Conti
• Within each (process, product, resources) class of entity, we
distinguish between internal and external attributes of a
product, process, or resource:

- Internal Attributes: are those that can be measured by

examining the product, process, or resource on its own, separate
from its behavior.
- External Attributes: are those that can be measured only with
respect to how the product, process or resource relates to the
environment.

• Internal software attributes (code structure, complexity) describe

the product itself, whereas external attributes (usability,
reliability, performance) describe how the software behaves in its
environment 4
 ..Conti
• Internal Attributes: • External Attributes:
 Size, Effort, Cost  Usability
 Code Complexity  Integrity
 Functionality  Efficiency
 Modularity  Testability
 Redundancy  Reusability
 Syntactic Correctness  Portability
 Reuse  Interoperability

• One of the goals of software measurement research is to

identify the relationships between internal and external
attributes, as well as to find new and useful methods for
directly measuring the attributes of interest.
5
..Conti

76
 Processes
• We often have questions about our software-development
activities and processes that measurement can help us to
answer.
• We want to know how long it takes for a process to complete,
how much it will cost, whether it is effective or efficient, and
how it compares with other processes that we could have
chosen.
• Following are some of the internal attributes that can be
measured directly for a process:
 The duration of the process or one of its activities.
 The effort associated with the process or one of its
activities.
 The number of incidents of a specified type arising during
the process or one of its activities.
7
 ..Conti
EXAMPLE-1: During formal testing, we can use the indirect measure
Cost / Number of errors
As a measure of the average cost of each error found during the process.

• EXAMPLE-2: AT&T developers wanted to know the effectiveness of their

software inspections. In particular, managers needed to evacuate the cost of
inspections against benefits received. To do this, they measured the average
amount of effort expended per thousand lines of code reviewed. This
information combined with measures of the number of faults discovered
during the inspections, allowed managers to perform a cost-benefit analysis.
EXAMPLE-3: The testing process may be composed of unit testing,
integration testing, system testing, and acceptance testing. Each
component process can be measured to determine how effectively it
contributes to overall testing. We can track the number of errors
identified in each subprocess, along with the duration and cost of
identifying each error, to see if each subprocess is cost-effective.

8
 Products
• Products are not restricted to the items that
management is committed to deliver to the
customer. Any artifact or document produced
during the software life cycle can be measured and
assessed.
• For example developers often build prototypes for
examination only, so that they can understand
requirements or evaluate possible designs; these
prototypes may be measured in some way.
• External product attributes depend on both
product behavior and environment, each attribute
measure should take these characteristics into
account.
9
 ..Conti
• Internal product attributes are sometimes easy
to measure.
• We can determine the size of a product by
measuring the number of pages it fills or the
number of words it contains. Since the products
are concrete, we have a better understanding of
attributes like size, effort, and cost.
• Other internal product attributes are more
difficult to measure, because opinions differ as
to what they mean and how to measured them.

12
10
 ..Conti
• Example-1, The complexity of codes and no consensus about
what best measures it.
• Example-2, If we are interested in measuring the reliability of
code, we must consider the machine and system configuration
on which the program is run as well as the mode of
operational usage.
• Example-3, Someone who uses a word-processing package
only to type letters may find its reliability to be different from
someone who uses the same package to merge tables and link
to spreadsheets.

11
 Importance Of Internal Attributes

• Many software engineering methods proposed and

developed in the last 40 years provide rules, tools,
and heuristics for producing software products.

• It is claimed that this structure makes them easier

to understand, and tests.
• It is assumed that good internal structure leads
to a good external quality.

• This connection has rarely been established.

12
 Resources
• The resources that we are likely to measure include any input
for software production.
• Resources include personnel, materials, tools and methods.
• Resources are measured to determine their magnitude, cost
and quality.
• Cost is often measured across all types of resources, so that
managers can see how the cost of inputs affects the cost of
the outputs.
• Resource measure combines a process measure (input) with
a product measure (output).

Productivity = Amount of output / Effort input

13
 Change and Evolution
• Software that is used is continually revised. New
features are added and errors are fixed.
• To understand and manage evolving software, we
need to measure attributes of products, processes, and
resources at various points in time.
• Processes change over time.
• Measurement can help us to understand and manage
evolving systems by allowing us to track the changes
in attributes over time.

14
 Determining What To Measure

• Measurement is useful only if it helps you to understand the

underlying process or one of its resultant products.
• The improvement in the process or products can be
performed only when the project has clearly defined goals for
processes and products.
• A clear understanding of goals can be used to generate
suggested metrics for a given project in the context of a
process maturity framework.

15
 Goal-Question-Metric Paradigm
• Many metrics programs begin by measuring what is convenient
or easy to measure, rather than by measuring what is needed.
• Such programs often fail because the resulting data are not useful
to the developers and maintainers of the software.
• A measurement program can be more successful if it is designed
with the goals of the project in mind.
• GQM (Goal-Question-Metric): A structured approach that
defines a measurement goal, refines it into questions, and
specifies metrics to answer those questions.
• So, GQM approach provides a framework involving the
following three steps:
Listing the major goals of the development or
maintenance project.
Deriving the questions from each goal that must be
answered to determine if the goals are being met.
Decide what must be measured in order16to be able to
answer the questions adequately.
 ..Conti

• Suppose your overall goal is to evaluate the

effectiveness of using a coding standard.
• The figure illustrates how several metrics
might be generated from your goal.

17
 ..Conti

• AT&T used GQM to help determine which metrics were appropriate for
assessing their inspection process (Barnard and Price 1994). Their goals, with
the questions and metrics derived are shown in the fig above.

 AT&T: American multinational telecommunications holding company.

18
 ..Conti
• In general, typical goals are expressed in terms of
productivity, quality, risk, and customer satisfaction
and the like, coupled with verbs expressing the need
to assess, evaluate, improve, or understand.
• Goals and questions must be audience-specific; a
productivity goal for a project manager may differ
from that of a department manager or corporate
director.
• To aid in generating the goals, questions, and
metrics, Basili and Rombach provided a series of
templates.

19
 ..Conti
• Templates for goal definition (Basili and Rombach 1988):

 Purpose: To (characterize, evaluate, predict, motivate, etc.) the (process,

product, model, metric, etc.) in order to (understand, assess, manage,
engineer, learn, improve, etc.) it.
• Example: To evaluate the maintenance process in order to
improve it.
 Perspective: Examine the (cost, effectiveness, correctness, defects,
changes, product measures, etc.) from the viewpoint of the (developer,
manager, customer, etc.)
• Example: Examine the cost from the viewpoint of the manager.
 Environment: The environment consists of the following: process
factors, people factors, problem factors, methods, tools, constraints, etc.
• Example: The maintenance staff consists of poorly motivated
programmers who have limited access to tools.

20
 Measurement and Process Improvement

• In the software industry, a common goal is process

improvement.
• Software processes vary from chaotic to well-defined.
• A mature process enhances the likelihood of delivering
reliable, adaptable software on time and within budget.

• Measurement is crucial for evaluating and

improving the relationships between processes,
products, resources, methods, and technologies in
software development.

21
 ..Conti

 The Software Engineering Institute’s (SEI’s) Capability

Maturity Model Integration (CMMI) for Development,
from the perspective of GQM (CMMI Product Team
2010) provides an ordinal ranking of development
organizations.
 The SEI differentiates between each level based on the
key process activities taking place. Which is from initial
(the least predictable and controllable, and least
understood) to optimizing (the most predictable and
controllable).
 This distinction is outlined by the CMMI Product Team,
providing a clear understanding of the specific activities
characterizing each maturity level.
22
 ..Conti
1. Initial: Level 1 processes are ad hoc and “success depends on the
competence and heroics of the people in the organization.”
2. Managed: Level 2 processes are planned; “the projects employ
skilled people … have adequate resources … involve relevant
stakeholders; are monitored, controlled, and reviewed … .”
3. Defined: Level 3 “processes are well characterized and
understood, and are described in standards procedures, tools, and
methods.”
4. Quantitatively managed: A Level 4 “organization and projects
establish quantitative objectives for quality and process
performance and use them as criteria in managing projects.”
5. Optimizing: A Level 5 “organization continually improves its
processes based on a quantitative understanding of its business
objectives and performance needs”

23
Combining GQM with Process Maturity

• Combining the Goal-Question-Metric (GQM) paradigm with Process

Maturity Models (such as CMMI) provides a structured, data-driven
approach to assess an organization's current process capabilities and
map them to maturity levels.
• maturity models define "what" good processes look like (level1-5),
GQM provides the operational "how" to measure, analyze, and justify
that maturity level

24
 ..Conti

• Combining these approaches transforms maturity assessments from

subjective audits into objective, evidence-based evaluations.
• Define Goals (GQM Level 1): Establish the organizational goal, such as
achieving CMMI Level 3 (Defined) or improving a specific process area like
"Project Planning".
• Formulate Questions (GQM Level 2): Break down the goal into questions
based on maturity model criteria.
- Example: "Do we have standardized, documented processes for project
planning?".
• Identify Metrics (GQM Level 3): Define quantitative metrics to answer
these questions
-e.g., percentage of projects using standardized templates, average variance in
schedule
• Evaluate & Score: Collect data to answer the questions, mapping results to
maturity level definitions (Initial Optimizing)

25
 ..Conti
Identifying the Level of Maturity
Using GQM, maturity is identified by evaluating the sophistication of the
metrics used and the consistency of the processes across the organization.
• Level 1 - Initial (Ad hoc/Reactive):
- GQM results indicate a lack of consistent data; metrics are nonexistent or
chaotic.
• Level 2 - Managed (Project level):
- GQM is applied at the project level. Metrics are used to control projects, but
not standardized across the organization.
• Level 3 - Defined (Organizational standard):
-GQM is applied systematically across the organization. Metrics confirm
adherence to a standardized, proactive process.
• Level 4 - Quantitatively Managed (Data-driven):
- Metrics are statistically analyzed to make processes predictable, often using
advanced GQM dashboards.
• Level 5 - Optimizing (Continuous improvement):
-Metrics are used to proactively identify and implement process improvements

26
 Software Measurement Validation
• Even when you know which entity and attribute you want to
assess, there are many measures from which to choose.
• Finding the best measure for your purpose can be difficult, as
candidates measure or predict the same attribute (such as cost,
size or complexity) in very different ways.
• So it is not surprising when managers are confused by
measurement; they see different measures for the same thing,
and sometimes the implications of one measure lead to a
management decision opposite to the implications of another!
• One of the roots of this confusion is the lack of
software measurement validation.
• That is, we do not always stop to ensure that the
measures we use actually capture the attribute
information we seek.
27
 ..Conti
• In a software development project, conflicting implications can
arise when using measures like Lines of Code (LOC) and
Function Points (FP) to assess progress.
• A higher LOC may be interpreted as increased complexity,
favoring a perspective on intricate coding efforts, while a higher
FP count emphasizes functionality and efficiency, potentially
advocating for streamlined code.
• Suppose a development team faces a choice between a code-
intensive approach (higher LOC) and an efficient, modular
approach (lower LOC but higher FP) for implementing a feature.
• The management decision may be conflicted, as praising
complexity may clash with commending efficiency, highlighting
the challenge of reconciling divergent measures in decision-
making.

28
 ..Conti
• The validation approach depends on distinguishing
measurement from prediction.
Measures or measurement systems are used to assess
an existing entity by numerically characterizing one or
more of its attribute.
Prediction systems are used to predict some attribute of
a future entity, involving a mathematical model with
associated prediction procedures.
• Informally, we say that a measure is “valid” if it accurately
characterizes the attribute it claims to measure. However, a
prediction system is “valid” if it makes accurate predictions.
So not only are measures different from prediction systems,
but the notion of validation is different for each. 29
 ..Conti

• Validating Software Measures: is the process of ensuring that

the measure is a proper numerical characterization of the claimed
attribute by showing that the representation condition is satisfied.
• The formal requirement for a validating measure involves
demonstrating that it characterizes the stated attribute in the
sense of measurement theory.
• Validating a prediction system in a given environment is the
process of establishing the accuracy of the prediction system
by empirical means; that is, by comparing model performance
with known data in the given environment.

30
 ..Conti
• Validating a prediction system involves experimentation
and hypothesis testing.
• To validate the prediction system formally you must first
decide how stochastic it is, and then compare performance
of the prediction system with known data points.

• Software engineering community has always been aware of

the need for validation.
• Thus, a measure must be viewed in the context in which it
will be used.
• Validation must take into account the measurement's
purpose; measure X may be valid for some uses but not for
others.
31
Thank You!! 32
Quiz(5%)

• List the main points of today’s lesson?(1 pt)

• List the 3 steps of GQM (Goal-Question-Metric)?(2 pt)
• List the levels of Capability Maturity Model?(2 pt)

33