1
Software Quality engineering
(SQE)
CHAPTER 22: SOFTWARE TESTING STRATEGIES
2
What is Software Quality?
Software quality is:
The degree to which a system, component, or process meets specified
requirements.
The degree to which a system, component, or process meets customer or
user needs or expectations.
3
What is Software
testing?
4
Software Testing
Software Testing: A set of activities performed with the
objective of identifying failures in a software and to evaluate
its level of quality, to obtain user satisfaction.
As per ANSI/IEEE 1059, Testing in Software Engineering
isa process of evaluating a software product to find whether the
current software product meets the required conditions or not.
Thetesting process involves evaluating the features of the software
product for requirements in terms of any missing requirements,
bugs or errors, security, reliability and performance.
5
Software Testing
Software Testing is defined as an activity to:
1. check whether the actual results match the expected results.
2. ensure that the software system is defect free.
Software testing also helps to identify errors, gaps or missing
requirements in contrary to the actual requirements.
It can be either done manually or using automated tools. Some
prefer saying Software testing as a White Box and Black Box Testing.
In simple terms, Software Testing means Verification & Validation of
Application Under Test (AUT).
6
Error vs. FAULT vs. failure
Error: The Problem in code leads to errors, which means that a mistake
can occur due to the developer's coding error as the developer
misunderstood the requirement or the requirement was not defined
correctly. The developers use the term error.
The test engineers use the terminology Bug.
Fault (Defect): When the application is not working as per the
requirement is known as defects. It is specified as the deviation from
the actual and expected result of the application or software. A fault
(defect) is introduced into the software as the result of an error.
7
Error vs. FAULT vs. failure
Failure: Many defects lead to the software's failure, which means that there exists a
deadly issue in software/ application or in its module, which makes the system
unresponsive or broken.
In other words, we can say that if an end-user detects an issue in the product, then
that particular issue is called a failure.
Possibilities are there one defect that might lead to one failure or several failures.
For example, in a bank application if the Amount Transfer module is not working for end-
users when the end-user tries to transfer money, submit button is not working. Hence, this is
a failure.
8
Do all software faults end with software failures?
Suppose a program has this code:
if temperature > 100: trigger_alarm()
But it should be:
if temperature >= 100: trigger_alarm()
This is a fault.
If the temperature never equals exactly 100 during testing or real
use, the program will never fail — the fault remains hidden.
Relationships between Software 9
errors, faults and failures
Error vs. FAULT vs. failure 10
11
Why is Software
Testing
Important?
12
Testing is important because software bugs could be
expensive or even dangerous. Software bugs can
potentially cause financial and human loss, and history
is full of such examples.
1. Nissan cars have to recall over 1 million cars from the market
due to software failure in the airbag sensory detectors. There
has been reported two accident due to this software failure.
2. Starbucks was forced to close about 60 percent of stores in the
U.S and Canada due to software failure in its POS system. At
one point, the store served coffee for free as they were unable
to process the transaction.
3. China Airlines Airbus A300 crashed due to a software bug on
April 26, 1994, killing 264 innocent lives.
4. In may of 1996, a software bug caused the bank accounts of
823 customers of a major U.S. bank to be credited with 920
million US dollars.
13
Defect detection in Software Products
vs. Industrial products
14
Classification of the causes of software errors
The causes of software errors can be further classified as follows according to
the stages of the software development process in which they occur.
1. Faulty definition of requirements
2. Client–developer communication failures
3. Deviations from software requirements
4. Logical design errors
5. Coding Errors
6. Non-Compliance with documentation and coding instructions
7. Shortcomings of testing process
8. Documentation Errors
15
Classification of the causes of software errors
1. Faulty definition of requirements
Erroneous definition of requirements
Absence of vital requirement
Incomplete definition of Requirement
Inclusion of unnecessary requirements
16
Classification of the causes of software errors
2. Client–developer communication failures
Misunderstanding of the
Client’s instructions as stated in the requirement document
Client’s requirements changes presented to the developer in
written form
Client’s requirements changes presented orally to the
developer
Lack of attention to client messages referring to requirements
changes
17
Classification of the causes of software errors
3. Deviations from software requirements
Developers may deliberately deviate from the
documented requirements
The developer reuses software modules taken from an
earlier project without sufficient analysis of the
changes
Due to time or budget pressures, the developer
decides to omit part of the required functions
18
Classification of the causes of software errors
4. Logical design errors
Algorithm mistakes
Erroneous definition of boundary conditions
Omission of required software system states
5. Coding Errors
Misunderstanding the design documentation
Linguistic errors in the programming languages
Errors in the application of tools
Errors in data selection
19
Classification of the causes of software errors
6. Non-Compliance with documentation and coding
instructions
Following team members will suffer from the non-
compliance
Team members who need to coordinate their own code with
code modules developed by non-complying team members
Individuals replacing the non-complying team member
Test team will find it difficult to test the module
Team members required to correct the errors will find it difficult
to correct errors
For maintenance team, it will be difficult to remove bugs
20
7. Shortcomings of testing process
Incomplete test plans leave untreated
portions of the software
Failuresto document and report
detected errors and faults
Incomplete correction of detected errors
due to negligence or time pressures.
21
8. Documentation Errors
Errors in design document
Omission of software functions.
Errors in user manual
Errors in the explanations and instructions given to
users
Listing of non-existing software functions
22
What does a tester do?
On any typical work day, a tester is busy in
following tasks:
understanding requirement documents,
creating test cases,
executing test cases,
reporting and re-testing bugs,
attending review meetings and other team
building activities.