Chapter One
Software Quality Assurance Concepts
Software Testing and Quality Assurance
• To deliver software products that meet high standards of quality and
reliability, software testing and quality assurance are being used as
integral components of the software development process.
• But they are not the same concepts.
• Software testing:
• Involves the systematic process of evaluating a software
application to identify defects or inconsistencies, ensuring
that it functions as intended and meets user
requirements.
• Testers design test cases, execute them, and report any
discovered defects, ultimately striving to enhance the
software’s robustness.
• Quality assurance:
• Is a comprehensive approach that goes beyond testing.
• It encompasses the establishment of :
• quality standards,
• process improvement initiatives, and
• adherence to best practices throughout the development lifecycle.
• Quality assurance aims to prevent defects from occurring in the
first place, fostering a culture of continuous improvement and
ensuring that software development processes are efficient,
effective, and aligned with organizational goals.
Difference between Software testing and quality
assurance
Criteria Software Testing Quality Assurance
Objective • The primary objective of • Quality assurance is a broader
software testing is to identify process that focuses on ensuring
defects or bugs in the software. that the entire software
• Testing is a dynamic process development process is carried out
where testers actively execute correctly.
the software to find issues such • It aims to prevent defects from
as functionality errors, occurring in the first place by
performance problems, security implementing best practices,
vulnerabilities, and usability processes, and standards
issues
Criteria Software Testing Quality Assurance
Timing • Testing is typically carried out • Quality assurance starts at the
after the software has been beginning of the software
developed. It is a validation development process and
process that confirms whether continues throughout the entire
the software meets the lifecycle.
specified requirements. • It encompasses activities like
• process audits,
• standards enforcement, and
• training to improve the
development process itself.
Criteria Software Testing Quality Assurance
Responsibility • Testers are responsible for • Quality assurance is the
conducting testing activities. responsibility of the entire
• They execute test cases, report development team and often
defects, and ensure that the involves quality assurance
software functions as expected. professionals, process experts, and
project managers.
• It requires a collaborative effort to
improve processes and maintain
quality standards.
Criteria Software Testing Quality Assurance
Deliverables • The primary deliverables • Quality assurance delivers
of testing are test plans, process documentation,
test cases, test reports, standards, guidelines, and
and defect logs. process improvement
• These artifacts provide recommendations.
information about the • It ensures that the
software’s quality and development process itself
reliability. is robust and efficient.
Criteria Software Testing Quality Assurance
Long-term vs.
• Testing is more • Quality assurance has a
Short-term
focused on short-term long-term focus, aiming
goals, such as to improve processes
identifying and fixing and prevent defects in
defects in the current future software
software release. projects.
Principles of software quality assurance
Implementation of software quality Assurance
• To implement SQA effectively, it is essential to follow a
structured approach.
•: • Clearly define the quality standards that your software
product must meet.
• This includes defining requirements, acceptance criteria,
Define quality
standards and performance metrics.
• These standards should be agreed upon by all
stakeholders, including the development team,
management, and customers.
• Develop a plan for the SQA activities that will be
performed throughout the software development life
cycle.
Plan SQA
activities: • This plan should include reviews, testing, and
documentation activities.
• It should also specify who will be responsible for each
activity and when it will be performed.
• Conduct reviews of software artifacts such as
requirements, design documents, and code.
• These reviews should be conducted by a team of
Conduct
reviews: experts who are not directly involved in the
development process.
• This will help identify defects early in the development
process and reduce the cost of fixing them later.
• Perform different types of testing such as unit
testing, integration testing, system testing, and
Perform
testing: acceptance testing.
• Use automated testing tools to increase efficiency
and reduce the risk of human error.
• Monitor and measure the quality of the software
product throughout the development process.
Monitor and
measure: • This includes tracking defects, analyzing metrics such
as code coverage and defect density, and conducting
root cause analysis.
• Continuously improve the SQA process by analyzing
the results of the monitoring and measuring
Improve
continuously: activities.
• Use this data to identify areas for improvement and
implement changes to the SQA process.
What is validation and verification(V & V)?
• Verification refers to the set of tasks that ensure that software
correctly implements a specific function.
• Validation refers to a different set of tasks that ensure that the
software that has been built is traceable to customer requirements.
Verification: “Are we building the product, right?”
Validation: “Are we building the right product?”
• In verification one may note that a product can be an intermediate product, such as
• requirement specification,
• design specification,
• code,
• user manual,
• or even the final product.
• Activities that check the correctness of a development phase are called verification
activities
• Validation activities focus on the final product, which is
extensively tested from the customer point of view.
• Validation establishes whether the product meets overall
expectations of the users
• The verification process establishes the correspondence of an
implementation phase of the software development process with its
specification, whereas validation establishes the correspondence
between a system and users’ expectations.
Summery of V&V
Validation Verification
Aim at confirming that one is building the correct Aims at confirming that one is building the
product product correctly
validation is performed toward the end of system Verification activities review interim work
development to determine if the entire system products, such as requirements specification,
meets the customer’s needs and expectations. design, code, and user manual, during a project
life cycle to ensure their quality. The quality
attributes sought by verification activities
are consistency, completeness, and correctness at
each major stage of system development.
Validation is performed on the entire system by Verification activities are performed on interim
actually running the system in its real products by applying mostly static analysis
environment and using a variety of tests. techniques, such as inspection, walkthrough, and
reviews, and using standards and checklists.
Verification can also include dynamic analysis,
such as actual program execution.
Detection Vs Prevention
• Detection : Finding defects after they occur
• Prevention: implementing activity to avoid defects in the first
place
Approach Example
Detection Running automated test suits to find bugs
Prevention Conducting design reviews
Software quality Models
• Software quality models provide a systematic framework to
assess and improve the quality of software products.
• They help us evaluate various dimension of software quality,
identify are for improvement, and implement best practices.
ISO/IEC 25000 (SQuaRE)
• ISO/IEC 25000 is the entry point of the SQuaRE (Systems and
Software Quality Requirements and Evaluation) series.
It provides the framework for defining, measuring, and
evaluating software and system quality.
• Think of it as the umbrella guide that organizes all the other ISO
250xx standards.
Purpose
• ISO/IEC 25000 aims to:
• Provide a structured quality model
• Define quality requirements
• Guide measurement of quality
• Support evaluation of software products
• Structure of the ISO 25000 (SQuaRE) Series
• The SQuaRE family is divided into 5 divisions:
• Quality Management Division (ISO 2500n)
• Quality Model Division (ISO 2501n)
• Quality Measurement Division (ISO 2502n)
• Quality Requirements Division (ISO 2503n)
• Quality Evaluation Division (ISO 2504n)
1. Quality Management Division (ISO 2500n)
• Focus: Overall framework and guidance
• Defines terminology and structure
• Explains how the divisions relate
2. Quality Model Division (ISO 2501n)
• Defines: Product Quality Model (8 characteristics) and Quality in Use
Model
Product Quality Characteristics:
[Link] suitability
[Link] efficiency
[Link]
[Link]
[Link]
[Link]
[Link]
[Link]
3. Quality Measurement Division (ISO 2502n)
• Focus: How to measure quality characteristics.
• Defines:
• Base measures: is a direct measurement of a single attribute —
obtained without calculation from other measures. Eg. Number of
defects found or lines of code
• Derived measures: is calculated from two or more base measures.
Eg. Defect density
• Indicators: interprets derived measures to support decision-making.
Eg. Thresholds or benchmarks
• Quality measures: Structured definition that
includes:
• Measurement objective
• Entity being measured
• Attribute being measured
• Measurement method
• Calculation formula
• Interpretation criteria
• This ensures measurements are:
• Repeatable
• Consistent
• Auditable
4. Quality Requirements Division (ISO 2503n)
• Focus: Writing measurable quality requirements.
• Helps answer:
• How do we specify quality in requirements?
• For example:
• “System availability shall be 99.5%.”
• “System response time shall not exceed 2 seconds.”
5. Quality Evaluation Division (ISO 2504n)
• Focus: Evaluating whether the product meets quality
requirements.
• Includes:
• Evaluation process
• Evaluation modules
• Reporting results
• Other models
• Capability Maturity Model Integration (CMMI)
• Six Sigma (DMAIC)
• PDCA Cycle
• SPICE
• Total Quality Management (TQM)
Software quality assurance techniques
• There are several techniques for SQA,
• Auditing : involves inspection of the work products and its related information
to determine if the set of standard processes were followed or not
• Reviewing: a meeting in which software product is examined by both internal
and external stakeholders to seek their comments and approval
• Code inspection: it is the most formal kind of review that does static testing to
find bubs and avoid defect growth in the later stages. It is done by a trained
mediator/peer and is based on rules, checklist, enter and exit criteria. The
review should not be the author of the code.
Software quality assurance techniques
• Design inspection: is done using • Performance
a checklist that inspects the • Error handling and recovery
below are of software design • Testability, extensibility
• Coupling and cohesion
• General requirement and design
• Functional and interface
specification
• Conventions
• Structures and interfaces
• Logic
Software quality assurance techniques
• Simulation: is a tool that models the real-life situation in order to
virtually examine the behavior of the system under test
• Functional test: verifies what the system does without considering how
it does. It is testing system specification (black box testing)
• Standardization: helps decrease ambiguity and guesswork thus
ensuring quality
Software quality assurance techniques
• Static analysis: it is a software analysis that is done by an automated
tool without actually executing the program. Software metrics and
reverse engineering are some popular forms of static analysis.
• Walkthroughs: kind of peer review where the developer guides the
members of the development team to go through the product and raise
queries, suggestions , comments …
• Path testing :white box testing
• Stress testing: testing the system beyond normal conditions
Software testing
• Software is tested to uncover errors that were made
inadvertently as it was designed and constructed.
• The software testing process has two distinct goals:
• To demonstrate to the developer and the customer that the software
meets its requirements.
• To discover faults or defects in the software where the behavior of the
software is incorrect, undesirable or does not conform to its
specification.
• Example of defects can be : system crashes, unwanted interactions with other
systems, incorrect computations and data corruption.
A model of the software testing process
Levels of testing
• Unit testing
• It is the process of testing individual components in the system.
• Integration testing
• This testing phase is crucial to ensure seamless interactions
among various units/components, their functionalities, and how
well they can operate as a single entity.
•
• System Testing
• It involves testing an integrated complete software to check against its
compliance with its requirements.
• Acceptance Testing
• This testing is done to ensure that the software does what the customer
wants it to do and check the acceptability of the system.
• Alpha testing: is a testing performed by both development team and users
using made-up data.
• Beta testing in which users start using the software with real data and
carefully observer the software for errors.
Summery
Criteria Unit Integration System Acceptance
Purpose Correct working Correct working Whole system works Customer’s
of of integrated units well when integrated expectations are met
unit/module
Focus Smallest testable interaction of Interaction and Software working per
part modules working of all given specifications
modules as one
Testing time Once a new code Once new Once the software is Once the software is
is components are complete operationally ready
written added
Developer Development Testing team Development team and
Performed by team End-user