Software Metrics
What is Measurement?
Measurement: Definition /1
Measurement is the process by which numbers or
symbols are assigned to attributes of entities
(objects) in the real world in such a way as to
ascribe them according to defined rules.
attribute1 value11 , value12 , ...
attribute2
Object
value21 , value 22 , ...
... ...
attribute valuen1 , value n2 , ...
n
Measurement: Definition /2
Metrics are standards (i.e., commonly
accepted scales) that define measurable
attributes of entities, their units and their
scopes.
Measure is a relation between an attribute
and a measurement scale.
Entity
An entity in software measurement can be
any of the following:
Processes: any activity related to software
development and/or maintenance (e.g., requirements
engineering, testing) – these can be at different
levels of granularity
Products: any artifact produced or changed during
software development and/or maintenance (e.g.,
source code, software design documents)
Resources: people, hardware or software needed for
the processes
Attribute
An attribute is a feature or property of an entity
e.g., blood pressure of a person, cost of a journey, duration of
the software specification process
There are two general types of attributes:
Internal attributes can be measured only
based on the entity itself,
Code e.g., entity: code, internal attribute: size,
modularity, coupling
External attributes can be measured only
with respect to how the entity relates to its
environment
e.g., entity: code, external attribute: reliability,
maintainability
Measurement Example
Entity Attribute 4
3
Requirements Size, Reuse, Redundancy 2
1
0
Specification Size, Reuse, Redundancy
A B
Design Size, Reuse, Modularity,
Cohesion, Coupling
Code Size, Reuse, Modularity,
Cohesion, Coupling, Complexity
Test Cases Size, Coverage
Measurement : How to
In order to make entities measurable:
What entities (objects) should be selected?
What attributes should be selected?
What values should be assigned to the attributes?
What shall be the rules (relationships) qualified to
the attributes and their entities?
Note: assigned values and/or certified rules can be
either quantitative or qualitative.
Example 1: Pressure Tank /1
Example 1: Pressure Tank /2
Entities (Objects):
Tanks (T1, T4), Valves (CV1, CV6), Pipes
Attributes:
Level of liquid, pressure of liquids, flow of material
Values:
Tank full, Tank empty, etc.
Rules:
Relations among level, pressure and flow of material.
Example 2: Code
Entity: Code
Attribute: Size
Possible measures:
NCSLOC (Not Commented Source Lines of
Code Code)
#Statements
#Modules
#Procedures
#Classes
#Methods
…
Example 3: Availability
Entity: Availability
Attributes: system uptime, system downtime
Values: time in seconds
Relations:
Availability = uptime / (uptime + downtime)
22
Software Metrics Challenges
Measuring physical entities:
entity attribute unit value
Human Height cm 178
Measuring non-physical entities:
entity attribute unit value
Human IQ IQ index 89
SE metrics are mostly non-physical
Reliability, maturity, portability, flexibility,
maintainability, etc. and relations are unknown
Misleading Metrics!
Fact (1): Knowledge is power
Fact (2): Time is money
Relation (rule) : power = work / time
Substituting “power” and “time”
Knowledge = work / money
As knowledge approaches zero money
approaches infinity regardless of the amount
of work done!
What went wrong
Conclusion: here?
The less you know, the more you make! - Counter intuitive
- Needs validation
What is
Software Measurement?
What Is Software Measurement?
Software metrics are measures that are used
to quantify software, software development
resources, and/or the software development
process.
This includes items which are directly
measurable, such as lines of code, as well as
items which are calculated from
measurements, such as software quality.
Measurement in SE
Measurement in SE is selecting, measuring
and putting together many different attributes
of the software, and adding our subjective
interpretations in order to get a whole picture
of the software.
This is not a
minor task!
300+ metrics have
been defined.
Measurement in SE /1
Before a measurement project can be planned
Objectives and scope should be established
Alternative solutions should be considered
Technical and management constraints should be
identified.
This information is required to estimate costs,
project tasks, and a project schedule.
Measurement in SE /2
In order to manage software measurement project
one must understand and plan:
The goal and scope of work
Risks
Resources required
Tasks to be accomplished
Milestones to be tracked
Total costs of the project
Schedule to be followed
Measurement in SE /3
Software metrics help us understand the technical
process that is used to develop a software product.
The process is measured to be improved.
The product is measured to increase its quality.
But …
Measuring software projects is controversial.
It is not yet clear which are the appropriate metrics
for a software project or whether people, processes,
or products can be compared using metrics.
Scope of Software Metrics
Cost and effort estimation
Productivity measures and models
Data collection
Quality models and measures
Reliability models
Performance evaluation and models
Structural and complexity metrics
Management by metrics
Evaluation of methods and tools
Example 1
We are going to buy a new colour laser printer for our
department. We have borrowed the printer for the test.
Maker’s data shows that we need to change the toner every
10,000 pages. We would like to have only one failure during
the period.
During test period, we observe that failures occur at 4,000
pages, 6,000 pages, 10,000 pages, 11,000 pages, 12,000
pages and 15,000 pages of output.
What can we conclude about this printer?
Example 1 (cont’d)
Failure intensity
objective:
λF = 1/10000 pages
Failure number
Using reliability
demonstration
chart we can
conclude that the
printer must be
rejected. Normalized measurement unit
Example 2
We are going to initiate a new game and video on-demand
download service. The service is provided to the customers
who own PCs and register with the service.
The customers must use specialized software to download
games or videos from the server. The failure intensity of the
software is 1 failure per 100 CPU hr. On average, the
specialized software system runs 20 CPU hr per week on
each client machine and there are 800 customers to be
serviced.
We would like to provide the customers with an on-site
repair service. Each serviceperson can make 4 service calls
per day. Service personnel are available 5 working days per
week.
How many service personnel do we need to hire?
Example 2 (cont’d)
How many service personnel do we need?
Using the value for failure intensity, each system
experiences 0.2 failure per 20 hours of operation or
0.2 failure per week, on average.
The total failures for 800 customers is 160 per week
or 32 per day.
Each serviceperson can visit 4 sites per day,
therefore, the number of required personnel is
32 / 4 = 8.
Overview:
Scope of Software Metrics
Process, Product and Resources
Scope of Software Metrics /1
Cost and effort estimation
Software cost estimation is the process of predicting the
amount of effort required to build a software system.
Estimates for project cost and time requirements are
derived during the planning stage of a project.
Models used to estimate cost can be categorized as either
cost models (e.g., Constructive Cost Model COCOMO).
Experience is often the only guide used to derive these
estimates, but it may be insufficient if the project breaks
new ground.
Many models are available as automated tools.
Scope of Software Metrics /2
Productivity models and measures
Definition: The rate of output per unit of input.
Productivity = size / effort
Productivity = LOC / person-month
Productivity model based on decomposition to measurable
attributes:
measurabl
e
far@[Link]
Scope of Software Metrics /3
Data collection
Very critical and very hard step.
What data should be collected?
How it should be collected?
Is collected data reproducible?
Example: software failure data collection
1) Time of failure
2) Time interval between failures
3) Cumulative failure up to a given time
4) Failures experienced in a time interval
Scope of Software Metrics /4
Quality models and measures
Software quality measurement (Rubey and Hartwick 1968~)
McCall’s quality factors (1977~), ISO 9126
Scope of Software Metrics /5
Reliability models
Plot the change of failure intensity () against time.
Many models are proposed. The most famous ones
are basic exponential model and logarithmic
Poisson model.
The basic exponential model assumes finite failures
in infinite time; the logarithmic Poisson model
assumes infinite failures.
Automated tools such as CASRE are available.
Reliability Models
Failure intensity ()
versus execution
time (τ)
0
(B) 0 e 0
0
(P)
0 1
Scope of Software Metrics /6
Performance evaluation and models
Using externally observable performance
characteristics such as response time and
completion rate (Ferrari 1978~)
Efficiency of algorithms (Garey 1979~)
Scope of Software Metrics /7
Structural and complexity
metrics
Control-flow structure
Data-flow structure
Data structure
Information flow attributes
Complexity metrics (1979~) V(F) = 5
Cyclomatic complexity (McCabe 1989)
defining number of independent paths in
execution of a program
Scope of Software Metrics /8
Management by metrics
Metrics for project control (1980~)
Metrics related to specification quality
Metrics for the design model
Metrics for documentation
Checking and testing metrics
Resource metrics
Change metrics
How to Implement?
The eight steps required to implement a software
measurement program are:
Document the software development process
State the goals
Define metrics required to reach goals GQM
Identify data to collect
Define data collection procedures
Assemble a metrics toolset
Create a metrics database
Define the feedback mechanism
Who Benefits From Measurement
Managers
What does each process cost?
How productive is the staff?
How good is the code being developed?
Will the user be satisfied with the product?
How can we improve?
Engineers
Are the requirements testable?
Have we found all the failures?
Have we met our product or process goals?
What can we predict about our software product in the
future?
Exercise
Suppose that you are asked to study various
software development tools and recommend the
best three to your company. The following table
shows a list of available development tools.
Exercise (cont’d)
Tool Name/Vendor Languages Platforms Features
Supported
Bean Machine Java Windows, OS2, Unix Best: Visual applet and JavaBean
IBM generation
CodeWarrior Pro Java, C, C++, Unix, Windows, Mac Best: if you need to support Unix,
Metrowerks Pascal Windows, and Mac platforms
Java Workshop Java Solaris, Windows Better: Written 100% in Java; tools
Sun Microsystems based on a web browser metaphor
JBuilder Imprise Java Windows, AS400 Better: database support
Visual Cafe for Java Java Windows Good: multithreaded debugger
Symantec
VisualAge Java Unix, Windows Good: includes incremental compiler
IBM and automatic version control
Visual J++ Java Windows Fair: All the bells and whistlesfor
Microsoft Windows
Exercise (cont’d)
What are the entities, attributes and their
values in your model?
Entity Attribute Value
Development Language Java, C, C++, Pascal
Tool supported
Platform Win, Unix, Mac,
OS2, AS400
Feature Fair, Good, Better,
Best
Conclusion
Without measurements there is no way to determine if the
process/product are improving.
Metrics allow the establishment of meaningful goals for
improvement. A baseline from which improvements can
be measured can be established.
Metrics allow us to identify the causes of defects which have
major effect on software development.
When metrics are applied to a product they help identify:
which user requirements are likely to change
which modules are most error prone
how much testing should be planned for each module