College of Computer & Cyber Sciences

Department of Computer Science

Program of SW Engineering

SE 463 Software Testing and Quality Assurance

Chapter 2: Testing Throughout the SW Life Cycle

Dr. Ftoon Kedwan

Chapter Outline
2.1 SDLC Models
2.1.1 Sequential Development Models
2.1.2 Iterative & Incremental Development Models
2.1.3 SDLC Models In Context

2.2 Test Levels

2.2.1 Component
2.2.1 Integration
2.2.1 System
2.2.1 Acceptance

2.3 Test Types

2.3.1 Functional
2.3.1 Non-functional
2.3.1 Structural
2.3.1 Related to Changes

2.4 Maintenance Testing

 2.1 SDLC Models

• Testing doesn't exist in isolation.

• Test activities are highly related to SW development activities.

• It will define the what, where, & when of our planned testing, influence
regression testing, & largely determine which test techniques to use.

• The way testing is organized must fit the development life cycle or it will fail
to deliver its benefit.

• If time to market is the key driver, then the testing must be fast & efficient.

• If a fully documented SDLC, with an audit trail of evidence, is required, the

testing must be fully documented.

• In every SDLC cycle, a part of testing is focused on verification testing & a

part is focused on validation testing.
2.1 SDLC Models, cont.
2.1.1 Sequential development models

• The waterfall model was 1 of the earliest models to be designed.

• The waterfall model has a natural timeline where tasks are executed in a
sequential fashion.

• In this model testing tends to happen towards the end of the project life cycle.

• So, defects are detected close to the live implementation date.

• No feedback passed backwards up the waterfall.

 2.1 SDLC Models, cont.
2.1.1 Sequential development models:
(waterfall)
 2.1 SDLC Models, cont.
2.1.1 Sequential development models (V-model)

• The V-model was developed to address some of the problems experienced using the
traditional waterfall approach.

• Defects were being found too late in the life cycle, as testing wasn't involved until the end
of the project.

• The V-model provides guidance that testing needs to begin as early as possible in the life
cycle, which is 1 of the fundamental principles of structured testing.

• The V-model is a model that illustrates how testing activities (verification & validation)
can be integrated into each phase of the life cycle.

• In V-model, validation testing takes place especially during the early stages, e.g., reviewing
the user requirements, & late in the life cycle, e.g., during user acceptance testing.
2.1 SDLC Models, cont.
2.1.1 Sequential
development models
(V-model)
2.1 SDLC Models, cont.
2.1.1 Sequential development models (V-model)

Figure 2.2 Adapted from Foundation of SW testing ISTQB Certification

2.1 SDLC Models, cont.
2.1.2 Iterative & incremental development
models

• Testing in Iterative & incremental

development:
o Rational Unified Process
o Scrum
o Kanban
o Spiral (or prototyping)
o Agile development

•Note: common issues with testing Iterative

& incremental models include:
o More regression testing.
o Defects outside the scope of the iteration
or increment.
o Less thorough testing.
 2.2 Test levels
•A thorough understanding & definition of the various test levels will identify missing areas & prevent overlap &
repetition.
o Unit (component) testing: tests each individual component (often a program) to ensure it is as defect-free as
possible.
• e.g., modules, programs, objects, classes, etc. that are separately testable.

o Integration testing: occurs between unit & system testing to test functionally grouped components.
• Tests interfaces between components, interactions to different parts of a system such as an operating system, file system &
hardware or interfaces between systems.

o System testing: tests the entire system as 1 entity.

• Concerned with the behavior of the whole system/product as defined by the scope of a development project or product.
• The main focus of system testing is verification against specified requirements.
 Alpha testing
 Beta testing

o User acceptance testing is an independent test performed by end users prior to accepting the delivered system.
• Validation testing with respect to user needs, requirements, & business processes conducted to determine whether or not
to accept the system.
• After performing a test, the users sign off if it was successful.
 2.3 Test types:
The Targets of Testing
• A test type is focused on a particular test objective, which could test:
o A function to be performed by the component or system.
o A non-functional quality characteristic, such as reliability or usability.
o The structure or architecture of the component or system.
o Or related to changes, i.e., confirming that defects have been fixed
(confirmation testing, or re-testing) & looking for unintended changes
(regression testing).

• Depending on its objectives, testing will be organized differently.

o e.g., component testing aimed at performance would be quite different
to component testing aimed at achieving decision coverage.

• In summary, testing types are:

1: Testing of function (functional testing / black box)
2: Testing of SW product characteristics (non-functional testing)
3: Testing of SW structure/architecture (structural testing / white box)
4: Testing related to changes (confirmation & regression testing)
 2.3 Test types:
The Targets of Testing, cont.
1: Testing of function (functional testing/ black box)
• The function of a system (or component) is what it
does. This is typically described in a requirements
specification, a functional specification, or in use cases.
• There may be some functions that are 'assumed' to be
provided that aren't documented that are also part of
the requirement for a system, though it is difficult to
test against undocumented & implicit requirements.
• Functional tests are based on these functions, described
in documents or understood by the testers & may be
performed at all test levels.
• e.g., test for components may be based on a
component specification.
• Functional testing considers the specified behavior & is
often also referred to as black-box testing.
• This isn't entirely true, since black-box testing also
includes non-functional testing.
• The techniques used for functional testing are often
specification-based, but experienced-based techniques
can also be used.
Functionality

2.3 Test types:

• Suitability
• Accuracy
• Security
• Interoperability
• Compliance The Targets of Testing, cont.
Reliability
• Maturity (robustness)
• Fault-tolerance
2: Testing of SW product characteristics
• Recoverability
(non-functional testing)
Usability The ISO 9126 standard defines 6 quality
• Understandability
• Learnability
characteristics & the subdivision of each
• Operability
• Attractiveness
quality characteristic into several sub-
characteristics.
Efficiency
• Time Behavior (Performance)
• Resource Utilization

Maintainability
• Analyzability
• changeability
• Stability
• Testability

Portability
• Adaptability
• Installability
• co-existence
• Replaceability
 2.3 Test types:
The Targets of Testing, cont.

3: Testing of SW structure/architecture (structural testing)

• Structural testing is often referred to as 'white-box' or
'glass-box' because we are interested in what is
happening 'inside the box’.
• Structural testing is most often used as a way of
measuring the thoroughness of testing through the
coverage of a set of structural elements or coverage
items.
• It can occur at any test level, although it is true to say
that it tends to be mostly applied at component &
integration testing.
• At component integration level, it may be based on the
architecture of the system, such as a calling hierarchy.
• At component level, & to a lesser extent at component
integration testing, there is good tool support to
measure code coverage.
 2.3 Test types:
the targets of testing,
cont.

4: Testing related to changes (confirmation &

regression testing)
• Specific types of tests relating to changes,
even though they may include all of the
previous test types.
a: Confirmation testing (re-testing)
b: Regression testing
 2.3 Test types:
The Targets of Testing, cont.

a: Confirmation testing (re-testing)

• When a test fails & the defect fixed, we will need to execute the test again
to confirm that the defect has indeed been fixed.
• When doing confirmation testing, it is important to ensure that the test is
executed in exactly the same way as it was the first time, using the same
inputs, data & environment.
• If the test now passes, we know that 1 part of the SW is correct - where
the defect was. But this isn't enough.
• The fix may have introduced or uncovered a different defect elsewhere in
the SW.
• The way to detect these 'unexpected side-effects' of fixes is to do
regression testing.
 2.3 Test types:
The Targets of Testing, cont.

b: Regression testing
• The purpose of regression testing is to verify that modifications in the SW or the
environment haven't caused unintended adverse side effects & that the system still meets
its requirements.
• It is common for organizations to have what is usually called a regression test suite or
regression test pack. ( at every level of testing: component, integration .. etc.).
• A regression test suite would be executed every time a new version of SW is produced &
this makes them ideal candidates for automation.
• If the regression test suite is very large, select a subset for execution.
• Regression tests are executed whenever the SW changes, either as a result of fixes or new
or changed functionality.
• A good idea is to execute them when some aspect of the environment changes
• e.g., when a new version of a database management system is introduced,
• or a new version of a source code compiler is used.
 2.4 Maintenance testing (cont.)
Triggers for maintenance testing
• Maintenance testing is done on an existing operational system.

• Usually, maintenance testing will consist of 2 parts:

o Testing the changes.
o Regression tests to show that the rest of the system hasn’t been affected by
the maintenance work.

• It is triggered by:
o Modifications. (planned or ad hoc)
o Migration. (e.g., from 1 platform to another)
o Retirement of the system.3

• Maintenance testing for the retirement of a system may include the

testing of data migration or archiving, if long data-retention periods are
required.

• Testing of restore or retrieve procedures after archiving may also be

needed.

• There is no point in trying to save & preserve some thing that you can no
longer access.