Software Defects
Lecture 3
Lecture Outline
Concept of Correctness
Defects: Errors, Faults & Failures
Relationship
Types of defects
Correctness
Correctness is typically the most important aspect of quality for situations
where daily life or business depends on the software.
Therefore, we adopt the correctness-centered view of quality.
We will focus on correctness-related quality attributes and related ways to
ensure and demonstrate quality.
Correctness & Defects
High-quality with a software system is an indication that few, if any, software
problems, are expected to occur during its operations.
When problems occur, the negative impact is expected.
Definitions
Key to the correctness aspect of software quality is the concept of defect:
Error
Fault
Failure
“Defect” generally refers to some problem with the software, either with its
external behavior or with its internal characteristics.
Error, Fault & Failure
The IEEE Standard 610.12 (IEEE, 1990) defines the following terms related to
defects:
Error: A human action that produces an incorrect result.
Fault: An incorrect step, process or data definition in a computer program.
Failure: The inability of a system or component to perform its required
functions within specified performance requirements.
Error, Fault & Failure
Error refers to a missing or incorrect human action resulting in certain
fault(s) being injected into a software.
Fault refers to an underlying condition within a software that causes certain
failure(s) to occur.
Failure refers to a behavioral deviation from the user requirement or the
product specification.
Error, Fault & Failure
We also extend errors to include error sources, or the root causes for the
missing or incorrect actions, such as human misconceptions,
misunderstandings etc.
Errors, Faults and Failures are collectively referred to as defects in literature.
Bugs
Software problems or defects, are also commonly referred to as “bugs”.
The concepts of error (including error source), fault and failure can be placed
into the context of software artifact, software development activities and
operational usage.
Concepts & Relations Illustrated
The software system as represented by its artifacts is depicted in the middle
box.
The artifacts include mainly software code and sometime other artifacts such
as designs, specifications, requirement documents etc.
The faults scattered among these artifacts are depicted as circled entities
within the middle box.
Concepts & Relations Illustrated
The input to the software development activities, depicted in the left box,
include conceptual models and information, developers with certain
knowledge and experience, reusable software components etc.
Various error sources are also depicted as circled entities within this left box.
Concepts & Relations Illustrated
Usage scenarios and execution results, depicted in the right box, describe the
input to software execution, its expected dynamic behavior and output and the
overall results.
A subset of these behavior patterns or results can be classified as failures
when they deviate from the expected behavior and is depicted as the
collection of circled failure instances.
Concepts & Relations Illustrated
A causal relation exists among these three aspects of defects:
Errors Faults Failures
Errors may cause faults to be injected into the software, and faults may
cause failures when the software is executed.
Concepts & Relations Illustrated
However, this relationship is not necessarily 1-to-1:
An error source, such as e5, may not cause any fault injection, and a fault,
such as f4, may not cause any failure, under the given scenarios or
circumstances.
Such faults are typically called dormant or latent faults, which may still
cause problems under a different set of scenarios or circumstances.
Functional defects
Functional defects are the errors identified in case the behavior of software is
not compliant with the functional requirements.
Such types of defects are discovered via functional testing.
Example: A functional defect was found in an ecommerce website’s search
engine. It didn’t return any results when a user typed in a product ID, while it
was stated in the requirements that the search could be conducted by both a
product’s name and ID.
Performance defects
Performance defects are those bound to software’s speed, stability, response
time, and resource consumption, and are discovered during performance
testing.
Example: A system’s response time being X times longer than that stated in
the requirements.
Usability defects
Usability defects make an application inconvenient to use and, thus, hamper a
user’s experience with software.
Example: A content layout that is difficult to scan or navigate and an overly
complex signup procedure.
Compatibility defects
An application with compatibility errors doesn’t show consistent performance
on particular types of hardware, operating systems, browsers and devices or
when integrated with certain software or operating under certain network
configurations.
Compatibility testing is carried out in order to discover such issues.
Example: A mobile app for car insurance claim estimation, failed to behave
according to the requirements on Android 8.0 and 8.1. The defects were
related to the changes in font size, content alignment and scroll bar.
Security defects
Security defects are the weaknesses allowing for a potential security attack.
Security testing is performed to identify the security defects.
Example: Encryption errors, buffer overflows, weak authentication.
Software defects by severity
Critical defects usually block an entire system’s or module’s functionality,
and testing cannot proceed further without such a defect being fixed.
Example: An application is returning a server error message after a login attempt.
High-severity defects affect key functionality of an application and the app
behaves in a way that is strongly different from the one stated in the
requirements.
Example: An email service provider does not allow adding more than one email address
to the recipient field.
Software defects by severity
Medium-severity defects are identified in case a minor function does not
behave in a way stated in the requirements.
Example: A broken link in an application’s Terms and Requirements section.
Low-severity defects are primarily related to an application’s UI.
Example: A slightly different size or color of a button.
Software defects by Priority
Urgent defects need to be fixed within 24 hours after being reported. Defects
with a critical severity status fall in this category. However, low-severity
defects may be classified as high-priority as well.
Example: A typo in a company’s name on an application’s home page doesn’t
have technical impact on software, but has a major business impact.
Software defects by Priority
High-priority defects are the errors that must be fixed in an upcoming
release in order to meet the exit criteria (the criteria or requirements which
must be met to complete a specific task or process).
Example: An application is failing to navigate a user from the login page to
the home page even though a user has entered valid login data.
Software defects by Priority
Medium-priority defects are the errors that may be fixed after an upcoming
release or in the subsequent release.
Example: An application returning the expected result, which however,
formats incorrectly in a particular browser.
Software defects by Priority
Low-priority defects are the errors that do not need to be fixed in order to
meet the exit criteria but require fixing before an application becomes
generally available.
Example: Typos, alignment, element size, and other cosmetic UI issues.
Cosmetic Bugs
• Bugs found in the GUI of the application are called as cosmetic bugs. These
bugs do not affect the processing of the application, but can have great impact
in some contexts.
• Example: spelling mistake, grammar mistake, font size, colours, tab order,
text overlapping etc.
So, What Is Software Quality?
Software quality may include many different attributes and may be defined and
perceived differently based on people’s different roles and responsibilities.
We adopt the correctness-centered view of quality, that is, high quality means none
or few problems of limited damage to customers.
These problems are encountered by software users and caused by internal software
defects.