MODULE 5
SOFTWARE QUALITY
Syllabus
Handouts for Lecture-26
INTRODUCTION
Quality is generally agreed to be 'a good thing', in practice what is meant by The 'quality ‘of a
system can be vague. We need to define precisely what qualities we require of a system.
Whether a system meets our quality requirements needs measurement.
THE PLACE OF SOFTWARE QUALITY IN PROJECT PLANNING
Quality will be of concern at all stages of project planning and execution but will be of
particular interest at the following points in the Step Wise framework as in Figure below
Step 1: identify project scope and objectives Some objectives could relate to the qualities of
the application to be delivered.
Step 2: Identify project infrastructure Within this step, identify installation standards and
procedures. Some of these will almost certainly be about quality.
Step 3: Analyze project characteristics the application to be implemented is examined to see if
it has any special quality requirements.
Step 4: Identify the products and activities of the project at this point that the entry, exit, and
process requirements are identified for each activity.
Step B: Review and publicize the plan At this stage the overall quality aspects of the project
plan are reviewed.
Figure The place of software quality in Step Wise
THE IMPORTANCE OF SOFTWARE QUALITY
Increasing the criticality of the software: The final customer or user is anxious about the
general quality of the software, especially its reliability. This is increasingly so as organizations
rely more on their computer systems and software is used in more safety-critical applications,
for example, to control aircraft.
The intangibility of software can make it difficult to l<now that a project task was completed
satisfactorily. Task outcomes can be made tangible by demanding that the developer produce
deliverables can be examined for quality.
Accumulating errors during software development As computer system development
comprises steps where the output from the step is the input to the next, the errors in the later
deliverables will be added to those in the earlier steps, leading to an accumulating detrimental
effect.
DEFINING SOFTWARE QUALITY
Some qualities of a software product reflect the external view of software held by users, as in the case
of usability. These external qualities have to be mapped to internal factors of which the developers
would be aware. It could be argued, for example, that well-structured code is likely to have fewer errors
and thus improve reliability. Defining quality is not enough. If we are to judge whether a system meets
our requirements, we need to be able to measure its qualities.
A good measure must relate the number of units to the maximum possible. For example, the maximum
number of faults in a program is standard relative to the size of the program, so a measure of faults per
thousand systems and lines of code is more helpful than total faults in a program.
Trying to find measures for a particular quality helps to clarify what that quality really is. What
is being asked, in effect, is, "How do we know when we have been successful?"
The measures may be direct, where we can measure the quality directly, or indirect, where the
thing being measured is not the quality itself but an indicator that the quality is present. For
example, the number of inquiries by users received by a help desk about how to operate a
particular software application might be an indirect measurement of its usability
When project managers identify quality measurements, they effectively set targets for project
team members, so care has to be taken that an improvement in the measured quality is always
meaningful. For example, the number of errors found in program inspections could be counted,
on the grounds that the more thorough the inspection process, the more errors will be
discovered.
When there is concern about the need for a specific quality characteristic in a software product,
a quality specification with the following minimum details should be drafted:
• Definition/Description: Clear definition of the quality characteristic.
• Scale: The unit of measurement used.
• Test: Practical test to measure the extent to which the quality attribute exists.
• Minimum Acceptable: The lowest value that might be acceptable if other characteristics
compensate, below which the product would be rejected outright.
• Target Range: The planned range of values for the quality measurement.
• Current Value: The current value of the quality characteristic.
SOFTWARE QUALITY MODELS
Quality models provide a hierarchical characterization of software based on a set of
characteristics. The bottom level of this hierarchy is quantitatively measurable, allowing for an
assessment of software quality. Among the well-established models are McCall's, Dromey’s,
and Boehm’s. Due to the lack of standardization among numerous quality models, the ISO
9126 model was developed to unify these approaches. In this section, we briefly discuss
Garvin's, McCall’s, Dromey’s, and Boehm’s models, while ISO 9126 will be covered in the
next section.
Garvin, a professor at Harvard Business School, defined eight dimensions of quality that users
typically consider when judging a product's quality in his book "Quality Management":
1. Performance: How well the software performs its intended functions.
2. Reliability: The software's ability to maintain performance under stated conditions for
a specified period.
3. Conformance: How well the software conforms to predefined standards and
specifications.
4. Durability: The software's expected operational life under normal conditions of use.
5. Serviceability: The ease and speed with which the software can be repaired or
maintained.
6. Aesthetics: The visual appeal and design of the software.
7. Perceived Quality: Users' subjective opinions and feelings about the software's quality.
McCall’ Model: McCall defined the quality of a software in terms of three broad
parameters: its operational characteristics, how easy it is to fix defects and how easy it is
to part it to different platforms. These three high-level quality attributes are defined based
on the following eleven attributes of the software:
Correctness: The extent to which a software product satisfies its specifications. Reliability:
The probability of the software product working satisfactorily over a given duration.
Efficiency: The amount of computing resources required to perform the required functions.
Integrity: The extent to which the data of the software product remain valid.
Usability: The effort required to operate the software product.
Maintainability: The ease with which it is possible to locate and fix bugs in the software
product.
Flexibility: The effort required to adapt the software product to changing requirements.
Testability: The effort required to test a software product to ensure that it performs its intended
function.
Portability: The effort required to transfer the software product from one hardware or software
system environment to another.
Reusability: The extent to which a software can be reused in other applications.
Interoperability: The effort required to integrate the software with other software.
Dromey’s model: Dromey proposed that software product quality depends on four major high-
level properties of the software: Correctness, internal characteristics, contextual characteristics
and certain descriptive properties. Each of these high-level properties of a software product,
in turn depends on several lower-level quality attributes. Dromey’s hierarchical quality model
is shown in Figure below.
Boehm’s Model: Boehm’s suggested that the quality of a software can be defined based on
these high-level characteristics that are important for the users of the software. These three
high-level characteristics are the following:
As-is -utility: How well (easily, reliably and efficiently) can it be used?
Maintainability: How easy is to understand, modify and then retest the software?
Portability: How difficult would it be to make the software in a changed
environment?
Boehm’s expressed these high-level product quality attributes in terms of several
measurable product attributes. Boehm’s hierarchical quality model is shown in Figure below
1. Define software quality
2. Define planning
3. Define reliability
4. What is lower level planning
5. What is necessity of quality models
6. Define augmentability
7. What are software attributes
Handouts for Lecture-27
PRODUCT AND PROCESS METRICS
Users assess the quality of a software product based on its external attributes, while developers
assess its quality during development based on various internal attributes. During development,
developers can ensure the quality of a software product by measuring relevant internal
attributes. These internal attributes may measure aspects of the product (product metrics) or
the development process (process metrics). Let's clarify the basic differences between product
and process metrics.
Product metrics: these measure the characteristics of the product being developed. For
example, metrics like lines of code (loc) and function points measure size, person-months (pm)
measure effort, and development time is measured in months.
Process metrics: these measure how well the development process is performing. Examples
include the effectiveness of reviews, average defects found per hour during inspections,
average time to correct defects, productivity metrics, average failures detected per loc during
testing, and the number of latent defects per line of code in the developed product.
PRODUCT VERSUS PROCESS QUALITY MANAGEMENT
In PRINCE2 project management, the focus on product-based planning is practical but
measuring product qualities during development is challenging. It's easier to assess these
qualities in finished applications than in intermediate stages. Alternatively, evaluating the
quality of development processes ensures effective practices to enhance overall product
quality.
The system development process involves interconnected activities where outputs from one
activity become inputs for the next Errors, can enter at any stage, caused by process defects or
information inaccuracies passing between stages. Addressing errors early is crucial as they
become costlier to correct in later stages, potentially adding more rework. For instance, an error
found in testing may require rework across all preceding stages, where changes are harder to
accommodate as development progresses.
Errors should therefore be eradicated by careful examination of the deliverables of each step
before they are passed on. One way of doing this is by having the following process
requirements for each step
Entry requirements, which must be in place before an activity can start, include examples such
as having a comprehensive set of test data and expected results prepared and approved before
program testing can commence.
Implementation requirements, which specify how a process is to be carried out, could stipulate
that during the testing phase, whenever an error is found and corrected, all test runs must be
repeated, including those were previously found to run correctly.
Exit requirements are conditions that must be fulfilled before an activity is considered
completed. For example, in the testing phase, for it to be recognized as completed, all tests
must have been successfully executed with no outstanding errors.
Handouts for Lecture-28
SOFTWARE PROJECT ESTIMATION
Software cost and effort estimation will never be an exact science. Too many variables human,
technical, environmental, political—can affect the ultimate cost of software and effort applied
to develop it.
To achieve reliable cost and effort estimates, a number of options arise:
1. Delay estimation until late in the project (obviously, we can achieve 100 percent accurate
estimates after the project is complete!).
2. Base estimates on similar projects that have already been completed.
3. Use relatively simple decomposition techniques to generate project cost and effort estimates.
4. Use one or more empirical models for software cost and effort estimation.
The first option, however attractive, is not practical. Cost estimates must be provided up-front.
However, you should recognize that the longer you wait, the more you know, and the more you
know, the less likely you are to make serious errors in your estimates.
The second option can work reasonably well, if the current project is quite similar to past efforts
and other project influences (e.g., the customer, business conditions, the software engineering
environment, deadlines) are roughly [Link] remaining options are viable approaches
to software project estimation.
Empirical estimation models can be used to complement decomposition techniques and offer
a potentially valuable estimation approach in their own right. A model is based on experience
(historical data) and takes the form
d = f (vi)
where d is one of a number of estimated values (e.g., effort, cost, project duration) and vi are
selected independent parameters
DECOMPOSITION TECHNIQUES
Software project estimation is a form of problem-solving, and in most cases, the problem to be
solved is too complex to be considered in one piece. For this reason, you should decompose
the problem, recharacterizing it as a set of smaller (and hopefully, more manageable) problems.
Software Sizing:
The accuracy of a software project estimate is predicated on several things:
(1) the degree to which you have properly estimated the size of the product to be built;
(2) the ability to translate the size estimate into human effort, calendar time, and dollars
(3) the degree to which the project plan reflects the abilities of the software team; and
(4) the stability of product requirements and the environment that supports the software
engineering error.
Putnam and Myers [Put92] suggest four different approaches to the sizing problem:
• “Fuzzy logic” sizing. This approach uses the approximate reasoning techniques that are the
cornerstone of fuzzy logic. To apply this approach, the planner must identify the type of
application, establish its magnitude on a qualitative scale, and then refine the magnitude within
the original range.
• Function point sizing. The planner develops estimates of the information domain
characteristics
• Standard component sizing. Software is composed of several different “standard components”
that are generic to a particular application area.
• Change sizing. This approach is used when a project encompasses the use of existing software
that must be modified in some way as part of a project. The planner estimates the number and
type of modifications that must be accomplished.
Problem-Based Estimation:
lines of code (LOC )and function points (FP) were described as measures from which
productivity metrics can be computed. LOC and FP data are used in two ways during software
project estimation: (1) as estimation variables to “size” each element of the software and (2) as
baseline metrics collected from past projects and used in conjunction with estimation variables
to develop cost and effort projections
LOC and FP estimation are distinct estimation techniques. Yet both have a number of
characteristics in common. You begin with a bounded statement of software scope and from
this statement attempt to decompose the statement of scope into problem functions that can
each be estimated individually. LOC or FP (the estimation variable) is then estimated for each
function. Alternatively, you may choose another component for sizing, such as classes or
objects, changes, or business processes affected.
Baseline productivity metrics (e.g., LOC/pm or FP/pm6 ) are then applied to the appropriate
estimation variable, and cost or effort for the function is derived. Function estimates are
combined to produce an overall estimate for the entire project.
The LOC and FP estimation techniques differ in the level of detail required for decomposition
and the target of the partitioning. When LOC is used as the estimation variable, decomposition
is absolutely essential and is often taken to considerable levels of detail. The greater the degree
of partitioning, the more likely reasonably accurate estimates of LOC can be developed.
For FP estimates, decomposition works differently. Rather than focusing on function, each of
the information domain characteristics—inputs, outputs, data files, inquiries, and external
interfaces
A three-point or expected value can then be computed. The expected value for the estimation
variable (size) S can be computed as a weighted average of the optimistic (sopt), most likely
(sm), and pessimistic (spess) estimates. For example,
gives heaviest credence to the “most likely” estimate and follows a beta probability
distribution. We assume that there is a very small probability the actual size result will fall
outside the optimistic or pessimistic values.
Once the expected value for the estimation variable has been determined, historical LOC or FP
productivity data are applied. Are the estimates correct? The only reasonable answer to this
question is: “You can’t be sure.”
An Example of LOC-Based Estimation
As an example of LOC and FP problem-based estimation techniques, I consider a software
package to be developed for a computer-aided design application for mechanical components.
The software is to execute on an engineering workstation and must interface with various
computer graphics peripherals including a mouse, digitizer, high-resolution color display, and
laser printer. A preliminary statement of software scope can be developed:
The mechanical CAD software will accept two- and three-dimensional geometric data from an
engineer. The engineer will interact and control the CAD system through a user interface that
will exhibit characteristics of good human/machine interface design. All geometric data and
other supporting information will be maintained in a CAD database. Design analysis modules
will be developed to produce the required output, which will be displayed on a variety of
graphics devices. The software will be designed to control and interact with peripheral devices
that include a mouse, digitizer, laser printer, and plotter.
This statement of scope is preliminary—it is not bounded. Every sentence would have to be
expanded to provide concrete detail and quantitative bounding. For example, before estimation
can begin, the planner must determine what “characteristics of good human/machine interface
design” means or what the size and sophistication of the “CAD database” are to be.
By summing vertically in the estimated LOC column, an estimate of 33,200 lines of code is
established for the CAD system. A review of historical data indicates that the organizational
average productivity for systems of this type is 620 LOC/pm. Based on a burdened labor rate
of $8000 per month, the cost per line of code is approximately $13. Based on the LOC estimate
and the historical productivity data, the total estimated project cost is $431,000 and the
estimated effort is 54 person-months
An Example of FP-Based Estimation:
Decomposition for FP-based estimation focuses on information domain values rather than
software functions. Referring to the table presented in Figure, you would estimate inputs,
outputs, inquiries, files, and external interfaces for the CAD software.
Each of the complexity weighting factors is estimated, and the value adjustment factor is
computed as
Process-Based Estimation:
The most common technique for estimating a project is to base the estimate on the process that
will be used. That is, the process is decomposed into a relatively small set of tasks and the
effort required to accomplish each task is estimated.
Like the problem-based techniques, process-based estimation begins with a delineation of
software functions obtained from the project scope. A series of framework activities must be
performed for each function. Functions and related framework activities8 may be represented
as part of a table similar to the one presented in Figure
Costs and effort for each function and framework activity are computed as the last step. If
process-based estimation is performed independently of LOC or FP estimation, we now have
two or three estimates for cost and effort that may be compared and reconciled
An Example of Process-Based Estimation
The engineering and construction release activities are subdivided into the major software
engineering tasks shown. Gross estimates of effort are provided for customer communication,
planning, and risk analysis. These are noted in the total row at the bottom of the table.
Horizontal and vertical totals provide an indication of estimated effort required for analysis,
design, code, and test.
Based on an average burdened labor rate of $8000 per month, the total estimated project cost
is $368,000 and the estimated effort is 46 person-months. If desired, labor rates could be
associated with each framework activity or software engineering task and computed separately.
Estimation with Use Cases:
Developing an estimation approach with use cases is problematic for the following reasons:
• Use cases are described using many different formats and styles—there is no standard form.
• Use cases represent an external view (the user’s view) of the software and can therefore be
written at many different levels of abstraction.
• Use cases do not address the complexity of the functions and features that are described.
• Use cases can describe complex behavior (e.g., interactions) that involve many functions and
features.
a number of investigators have considered use cases as an estimation input no proven
estimation method has emerged to date.9 Smith [Smi99] suggests that use cases can be used
for estimation, but only if they are considered within the context of the “structural hierarchy”
Smith argues that any level of this structural hierarchy can be described by no more than 10
use cases
To illustrate how this computation might be made, consider the following relationship:
An Example of Use-Case–Based Estimation
The CAD software is composed of three subsystem groups: user interface subsystem (includes
UICF), engineering subsystem group (includes the 2DGA, 3DGA, and DAM subsystems), and
infrastructure subsystem group (includes CGDF and PCF subsystems). Six use cases describe
the user interface subsystem. Each use case is described by no more than 10 scenarios and has
an average length of six pages. The engineering subsystem group is described by 10 use cases
(these are considered to be at a higher level of the structural hierarchy). Each of these use cases
has no more than 20 scenarios associated with it and has an average length of eight pages.
Finally, the infrastructure subsystem group is described by five use cases with an average of
only six scenarios and an average length of five pages.
Using the relationship noted in Expression LOC estimate with n 30 percent, the table shown in
Figure is developed. Considering the first row of the table, historical data indicate that UI
software requires an average of 800 LOC per use case when the use case has no more than 12
scenarios and is described in less than five pages. Hence the LOC estimate for the user interface
subsystem is computed using expression. Using the same approach, estimates are made for
both the engineering and infrastructure subsystem groups. Figure 26.5 summarizes the
estimates and indicates that the overall size of the CAD is estimated at 42,500 LOC. Using 620
LOC/pm as the average productivity for systems of this type and a burdened labor rate of $8000
per month, the cost per line of code is approximately $13. Based on the use-case estimate and
the historical productivity data, the total estimated project cost is $552,000 and the estimated
effort is 68 person months.
Reconciling Estimates:
The estimation techniques discussed in the preceding sections result in multiple estimates that
must be reconciled to produce a single estimate of effort, project duration, or cost.
The total estimated effort for the CAD software ranges from a low of 46 person months (derived
using a process-based estimation approach) to a high of 68 person months (derived with use-
case estimation). The average estimate (using all four approaches) is 56 person-months. The
variation from the average estimate is approximately 18 percent on the low side and 21 percent
on the high side
What happens when agreement between estimates is poor? The answer to this question requires
a reevaluation of information used to make the estimates. Widely divergent estimates can often
be traced to one of two causes:
(1) the scope of the project is not adequately understood or has been misinterpreted by the
planner,
(2) productivity data used for problem-based estimation techniques is inappropriate for the
application, obsolete or has been misapplied.
EMPIRICAL ESTIMATION MODELS
An estimation model for computer software uses empirically derived formulas to predict effort
as a function of LOC or FP The empirical data that support most estimation models are derived
from a limited sample of projects. For this reason, no estimation model is appropriate for all
classes of software and in all development environments.
An estimation model should be calibrated to reflect local conditions. The model should be
tested by applying data collected from completed projects, plugging the data into the model,
and then comparing actual to predicted results. If agreement is poor, the model must be tuned
and retested before it can be used.
The Structure of Estimation Models
A typical estimation model is derived using regression analysis on data collected from past
software projects.
where A, B, and C are empirically derived constants, E is an effort in person-months, and ev is
the estimation variable. most estimation models have some form of project adjustment
component that enables E to be adjusted by other project characteristics Among the many LOC-
oriented estimation models proposed in the literature are.
The COCOMO II Model
Barry Boehm [Boe81] introduced a hierarchy of software estimation models bearing the name
COCOMO, for COnstructive COst MOdel. The original COCOMO model became one of the
most widely used and discussed software cost estimation models in the industry. It has evolved
into a more comprehensive estimation model, called COCOMO II [Boe00]. Like its
predecessor, COCOMO II is actually a hierarchy of estimation models that address the
following areas:
Application composition model. Used during the early stages of software engineering, when
prototyping of user interfaces, consideration of software and system interaction, assessment of
performance, and evaluation of technology maturity are paramount.
• Early design stage model. Used once requirements have been stabilized and basic software
architecture has been established.
• Post-architecture-stage model. Used during the construction of the software.
Like all estimation models for software, the COCOMO II models require sizing information.
Three different sizing options are available as part of the model hierarchy: object points,
function points, and lines of source code.
The COCOMO II application composition model uses object points
Like function points, the object point is an indirect software measure that is computed using
counts of the number of
(1) screens (at the user interface),
(2) reports, and
(3) components likely to be required to build the application.
Each object instance (e.g., a screen or report) is classified into one of three complexity levels
simple, medium, or difficult) using criteria suggested by Boehm.
Once complexity is determined, the number of screens, reports, and components are weighted
according to the table illustrated in Figure
When component-based development or general software reuse is to be applied, the percent of
reuse (%reuse) is estimated and the object point count is adjusted:
Once the productivity rate has been determined, an estimate of project effort is computed using
In more advanced COCOMO II models,12 a variety of scale factors, cost drivers, and
adjustment procedures are required.
The Software Equation
The software equation [Put92] is a dynamic multivariable model that assumes a specific
distribution of effort over the life of a software development project. The model has been
derived from productivity data collected for over 4000 contemporary software projects.
Where
E effort in person-months or person-years
t project duration in months or years
B “special skills factor”13
p “productivity parameter” that reflects: overall process maturity and management
practices, the extent to which good software engineering practices are used, the level of
programming languages used, the state of the software environment, the skills and experience
of the software team, and the complexity of the application.
the software equation has two independent parameters:
(1) an estimate of size (in LOC) and
(2) an indication of project duration in calendar months or years.
To simplify the estimation process and use a more common form for their estimation model,
Putnam and Myers [Put92] suggest a set of equations derived from the software equation.
Minimum development time is defined as.