software quality:
· Software quality is a multifaceted concept referring to a
software product's ability to meet defined requirements,
expectations, and standards, encompassing characteristics such
as functional suitability, reliability, usability, maintainability,
and efficiency. It goes beyond just a lack of bugs to include
aspects like good design, portability, security, and overall user
value.
Key Aspects of Software Quality
Functional Suitability:
The software performs its intended functions correctly and
completely.
Reliability:
The software consistently performs without failures, is available,
and can recover from faults.
Usability:
The software is easy to learn, understand, and operate for its
intended users.
Maintainability:
The software's code is easy to modify, fix, and enhance over time,
which includes efficient bug resolution.
Efficiency:
The software performs its functions using minimal resources, such
as processing power and memory.
Portability:
The software can be easily transferred and used in different
environments or platforms.
Security:
The software effectively protects information and prevents
unauthorized access or breaches.
Approaches to Software Quality
Defect Management:
Focuses on identifying, counting, and managing defects (failures to
meet requirements) to measure and improve process capabilities.
Quality Attributes:
Relies on established quality models (like ISO/IEC 25010) to define
and assess desirable product characteristics.
Why Software Quality Matters
Meets Requirements:
Ensures the software fulfills the needs and expectations of users and
stakeholders.
Enhances User Experience:
Leads to more positive interactions and satisfaction for users.
Reduces Costs:
High-quality software often results in fewer costly fixes, less rework,
and lower maintenance costs in the long run.
Increases Value:
Delivers a worthwhile product that customers perceive as valuable
and that generates positive return on investment.
Software Quality – Verification vs. Validation
Verification** and **Validation** are two crucial processes in
software quality assurance (SQA) that ensure software is developed
correctly and meets user needs.
Verification**:
Verification answers the question: *"Are we building the product
right?"*
It is the process of checking whether the software conforms to
specified requirements or design specifications. Verification is
typically conducted through reviews, inspections, walkthroughs, and
testing at each stage of development. It ensures that the software is
being developed in line with the design.
Examples of Verification Activities**:
* Code reviews
* Design inspections
* Static analysis
* Unit testing
* **Validation**:
Validation answers the question: *"Are we building the right
product?"*
It focuses on ensuring the software meets the user’s needs and
requirements. It checks if the software behaves as expected in real-
world use and if it satisfies the end user’s expectations.
**Examples of Validation Activities**:
* System testing
* Acceptance testing
* User feedback
* Beta testing
Components of Quality Assurance (QA)
Software Quality Assurance is a broader discipline aimed at
improving the development and maintenance processes, ensuring
that quality standards are met. The key components of QA include:
1. Process Definition and Improvement:
Establishing clear processes for software development, testing, and
maintenance. Continuous process improvement is crucial for
maintaining high quality over time.
2. Standardization:
Developing and adhering to industry standards (e.g., ISO, IEEE,
CMMI) for processes, testing, and documentation.
3. Testing:
Performing different levels of testing (unit, integration, system,
acceptance) to ensure the software is free of defects and meets
requirements.
4. Reviews and Inspections:
Reviewing requirements, design, code, and other artifacts to
ensure they conform to standards and meet the desired quality
levels.
5. Metrics and Measurements:
Using metrics to measure and track the quality of the software and
development process (e.g., defect density, test coverage, code churn).
6. Tool Support:
Utilizing tools for automated testing, static analysis, build
automation, and continuous integration.
Software Quality Assurance (SQA) Plan
An **SQA Plan** outlines how software quality assurance will be
integrated into the development lifecycle. It defines the roles,
responsibilities, processes, and tools used to ensure quality. Key
components of an SQA plan typically include:
1. Scope of the SQA activities:
Defines the areas where SQA will be applied.
2. Roles and responsibilities:
Specifies the roles of individuals involved in SQA (e.g., testers,
developers, SQA managers).
3. Quality goals:
Defines measurable quality targets, such as defect rates,
performance criteria, and user satisfaction metrics.
4. Methods and techniques:
Details the methods (e.g., reviews, inspections, testing) used to
evaluate software quality.
5. Tools and resources:
Specifies the tools that will be used for automation, defect
tracking, and reporting.
6. Timeline and milestones:
Includes a schedule with quality checkpoints at various stages of
the development lifecycle.
Quality Standards
Quality standards provide a framework for ensuring that software
meets specific quality criteria. Some commonly used quality
standards in software engineering include:
1. ISO/IEC 9126:
Defines a quality model for software, focusing on attributes like
functionality, reliability, usability, efficiency, maintainability, and
portability.
2. ISO/IEC 25010:
The successor to ISO/IEC 9126, this standard defines software
quality characteristics like functional suitability, performance
efficiency, compatibility, usability, reliability, security,
maintainability, and portability.
3. IEEE 829:
Standard for software test documentation, including test plans,
test cases, and test reports.
4. Six Sigma:
Focuses on reducing defects and improving process efficiency. It
uses statistical methods to measure and improve the quality of
software.
CMM (Capability Maturity Model)
"CMM" in the context of "SQT" (Software Quality Testing) refers
to the Capability Maturity Model, a process improvement
framework developed by the Software Engineering Institute that
assesses the maturity of an organization's software development
processes across five levels, helping them systematically improve
their practices and produce higher-quality software.
**CMM** is a model for improving software development
processes. It defines five maturity levels:
This is a process-based model that is used to assess the maturity of
an organization for different domains. The concept of CMM was
introduced by the Software Engineering Institute (SEI) in the USA.
Although this model is applied to the Software Development process,
eventually it is used for other processes like QA and testing as well.
It has 5 different levels of maturity from 1 to 5. As we go towards
level 5 from 1, variability and inconsistency get reduced. Given
below are the details of 5 levels.
Here we will go through the 5 CMM levels with respect to the QA
process and what output/result is expected for each level to mature a
QA/testing process and reach up to level 5.
CMM Levels
Level 1 (Initial) – Ad-Hoc: Unplanned, Unsystematic, and
Inconsistent
As the word “Ad-Hoc” states: unplanned, and unprepared, at this
level, the significance is not given to planning, following processes,
guidelines, and standards.
There is no standardized & consistent way of doing any task. The
only thing that is important at this level is meeting the timelines,
irrespective of the quality of the end product and deliverables. As
there are no pre-defined standards and processes, the same task is
done in different ways by different people.
This becomes even more unsystematic and inconsistent if the same
task is done differently next time as there are no documents
available in the process which can allow the process to be replicated.
So, at this level, the process is poorly controlled, unpredictable, and
reactive.
For Example,
QA – The example would be that in an organization although QA is
one of the phases in a product life cycle, there are not any standards
& no defined process, and no templates for QA deliverables – test
plan, test strategy, test scenarios, and test cases are not
standardized.
Even if these things are defined & documented then all the team
members have their own way of doing the tasks and the process is
not consistent at all. So, basically there is no control over QA and it’s
a chaotic phase.
Level 2 (Repeatable) – Control: Initiate Defining Processes
At A High Level
In this phase, we get a solution to the problem regarding the
unavailability of QA processes, methodology & standards which we
saw at Level 1. We have processes, methodology & standards in
place.
The standards and processes are not only finalized but are also well
documented so that those can be re-used by any of the similar tasks
that have been done previously. That is why this level is known as
“repeatable” – as we can repeat the steps for doing the same kind of
work.
So, the focus is on basic project management at this level.
For Example,
types of testing like functional, data, performance, etc. Define the
role & responsibilities of a QA engineer & test lead in the project’s
life cycle and prepare templates for deliverables in each phase. Test
plans, test strategies, test scenarios, and test cases should all be in
place.
Not only define and prepare but also share the documentation
within the team.
Level 3 (Defined) – Core Competency:
Come Up With A Generalized Process For Wider Audiences And
Domains
At level 3, people are motivated to follow the standards and
processes defined at level 2. For this, first of all, the processes need
to be conveyed to all the involved people. It needs to be identified
that all skills are needed to use the processes & standards effectively
& efficiently and also if there is any training is required for that.
Then, motivate and support the resources to follow those standards
and processes. Here, people with more experience need to share
their knowledge with others.
The focus is on documentation, process standardization, and
integration. By this time, the organization has developed its own
standard process of software testing.
For Example,
QA – Conduct webinars and training sessions to let people get
acquainted with the newly defined QA process and standards and
motivate them to make use of those during their day-to-day project
work.
Level 4 (Managed) – Predictable:
Measure the ProcessesAt this level, processes defined at level 3 are
measured quantitatively. This is done to control the effort required
on any task. Based on this quantitative analysis, processes can be
adjusted if needed, and that too without degrading the quality of the
end product.
An analysis is done by dividing the complete process into smaller
sub-processes and then quantitative techniques are applied to these
sub-processes. As per the results, sub-processes can be adjusted if
needed.
This level is called predictable because based on prior experience,
we can predict the process quantitatively and make use of this
prediction for the upcoming processes.
The key process areas of CMM level 4 are quantitative project
management and organizational process performance.
CMM level 4
In short, the process is measured & controlled at this level.
For Example,
QA – Performing regular audits would be a good idea here. This can
include checking if teams are following the processes defined, using
standard templates, adhere to methodology or not.
If you are into automation testing, then doing periodic code reviews
of the automation test scripts would be an apt example here.
Level 5 (Optimizing) – Innovative: Continuous Improvement
Continuous Improvement
At this level, innovative ways are identified to further improve the
pre-defined processes and standards. This is a continuous process.
For this, our own processes are watched and re-engineered
continuously by adding new tools & technologies, by continuous
research & studies, and by keeping ourselves updated with new
information in the market.
This can also be achieved by benchmarking other organizations and
learning from them and trying to improve our process by adding
new innovations to it. So, the focus is on continuous process
improvement at this level. The key process areas are organizational
performance management & quantitative project management.
PCMM (People Capability Maturity Model)
What is PCMM?
The PCMM is a working framework used for the organization in
defining the maturity structure for improving and developing the
skill set of people who work within the organization.
The PCMM framework defines the complete path from starting an
initial phase, which includes inconsistent business activities, to a
complete, mature phase that includes proper development of skills
and working practices to benefit the organization.
Using the PCMM framework, the organization can benefit from
their business activities and find the critical people issues which they
face in the organization while doing their working activities.
The framework guides the organization to develop the people’s skill
set, knowledge set and solve all the [people issues for improving the
organisation’s business activities.
The PCMM framework includes the five stages for achieving the
goal: the initial level, managed level, defined level, predictable level,
and optimizing level.
All these levels help the organization achieve its desired goals and
objectives and compete with other organizations in the market.
Methods of PCMM
In the PCMM framework, five methods help in continuous
improvement of the knowledge set, skill set, development of effective
methods, and improve the people mindset for the organisation’s
benefit. These five methods are 5 maturity level which has its own
importance which is used for defining the capability and develop the
capability within the organization.
The 5 PCMM maturity level are defined below:
1. Initial Level
This phase deals with inconsistent management. In this stage, there
are no defined process areas. At this level, the organization process
is ritualistic and inconsistent. The process is disorganized because
they are not properly defined and documented, and the organization
growth is dependent on individual efforts. The process is not also in
the repeatable phase because of not properly defined. The people of
an organization have the skill set, but they don’t know how to use
them properly for the organization, and they also have a less
emotional attachment towards the organization. That’s the main
reason for inconsistency in the process that occurred in the working
organization.
2. Managed Level
This phase deals with the actual process of people management. In
this stage, the managers play a crucial role in developing the
workforce practice in the people. They follow different practices like
operating performance, staffing, adjusting the compensation to
maintain the discipline and acquire the managed level. All these
activities are performed repeatedly by the manager to be done,
which could be beneficial for the organization.
The organization set certain goals, objectives that every person who
works for the organization needs to meet and work with full
efficiency. All these practices are done at the unit level, and the
organization handles the performance and the skill set for
developing this capability in the people. All the process areas which
are included in the managed level of the PCMM framework are
training development, work environment, performance
management, staffing, compensation, and communication
coordination within the organization.
3. Defined Level
The defined level is the third phase of the PCMM framework, and
this phase deals with the process named competency management.
The aim of this stage is to develop the competency skills in the
organization so that it can work well so that the organization can
compete with other organizations for better business activities. The
organization has a role in maintaining a proper workforce so that it
can achieve competency and do proper business activities within the
organization.
The workforce competency includes the strategic workforce
competency and improved workforce competency. In strategic
workforce competency, it includes the activity which can benefit
both present and future business activities. The improved workforce
competency is beneficial for the organization as it can be important
for the betterment of the skill set and practices involved in the
organisation’s business activities. And in this level, the organization
of the proper documentation defines the standards and integrates
the process for achieving its business objectives.
4. Predictable Level
The predictable level is the fourth phase of the PCMM framework.
At this level, the organization handles the capability, which is
developed in an earlier phase so that it can achieve its working
requirements. At this level, the organization develops enough skill
set to handle the performance of the business activity and manage
the working capacity within the organization.
As the capability is developed, the organization can predict the
capability and working capacity as the competency methods have
been done in an earlier phase. The prediction of capacity will help
the organization and help to better business activities within the
organization.
5. Optimizing Level
The optimizing level is the last phase of the PCMM framework. In
this stage, the whole organization is more focused on the continuous
improvement of business activity in the organization. Continuous
improvement, it can benefit both the working groups and
individuals and maintain the working efficiency within the
organization. In this stage, the organization looks at the results of all
other stages so that improvement can be done in that business
activities, which can be beneficial for the organization
What is Capability Maturity Model
Integration (CMMI)?
· Capability Maturity Model Integration (CMMI) is a successor
of CMM and is a more evolved model that in corporates best
components of individual disciplines of CMM like Software
CMM, Systems Engineering CMM, People CMM, etc. Since
CMM is a reference model of matured practices in a specific
discipline, so it becomes difficult to integrate these disciplines
as per the requirements. This is why CMMI is used as it allows
the integration of multiple disciplines as and when needed.
· Objectives of CMMI
· Fulfilling customer needs and expectations.
· Value creation for investors/stockholders.
· Market growth is increased.
· Improved quality of products and services.
· Enhanced reputation in Industry.
· CMMI Representation - Staged and Continuous
· A representation allows an organization to pursue a different
set of improvement objectives. There are two representations
for CMMI :
·
· Staged Representation :
· uses a pre-defined set of process areas to define improvement
path.
· provides a sequence of improvements, where each part in the
sequence serves as a foundation for the next.
· an improved path is defined by maturity level.
· maturity level describes the maturity of processes in
organization.
· Staged CMMI representation allows comparison between
different organizations for multiple maturity levels.
· Continuous Representation :
· allows selection of specific process areas.
· uses capability levels that measures improvement of an
individual process area.
· Continuous CMMI representation allows comparison between
different organizations on a process-area-by-process-area
basis.
· allows organizations to select processes which require more
improvement.
· In this representation, order of improvement of various
processes can be selected which allows the organizations to
meet their objectives and eliminate risks.
· CMMI Model - Maturity Levels
· In CMMI with staged representation, there are five maturity
levels described as follows :
·
· Maturity level 1 : Initial
· processes are poorly managed or controlled.
· unpredictable outcomes of processes involved.
· ad hoc and chaotic approach used.
· No KPAs (Key Process Areas) defined.
· Lowest quality and highest risk.
· Maturity level 2 : Managed
· requirements are managed.
· processes are planned and controlled.
· projects are managed and implemented according to their
documented plans.
· This risk involved is lower than Initial level, but still exists.
· Quality is better than Initial level.
· Maturity level 3 : Defined
· processes are well characterized and described using
standards, proper procedures, and methods, tools, etc.
· Medium quality and medium risk involved.
· Focus is process standardization.
· Maturity level 4 : Quantitatively managed
· quantitative objectives for process performance and quality
are set.
· quantitative objectives are based on customer requirements,
organization needs, etc.
· process performance measures are analyzed quantitatively.
· higher quality of processes is achieved.
· lower risk
· Maturity level 5 : Optimizing
· continuous improvement in processes and their performance.
· improvement has to be both incremental and innovative.
· highest quality of processes.
· lowest risk in processes and their performance.
· CMMI Model - Capability Levels
· A capability level includes relevant specific and generic
practices for a specific process area that can improve the
organization's processes associated with that process area. For
CMMI models with continuous representation, there are six
capability levels as described below :
·
· Capability level 0 : Incomplete
· incomplete process - partially or not performed.
· one or more specific goals of process area are not met.
· No generic goals are specified for this level.
· this capability level is same as maturity level 1.
· Capability level 1 : Performed
· process performance may not be stable.
· objectives of quality, cost and schedule may not be met.
· a capability level 1 process is expected to perform all specific
and generic practices for this level.
· only a start-step for process improvement.
· Capability level 2 : Managed
· process is planned, monitored and controlled.
· managing the process by ensuring that objectives are achieved.
· objectives are both model and other including cost, quality,
schedule.
· actively managing processing with the help of metrics.
· Capability level 3 : Defined
· a defined process is managed and meets the organization's set
of guidelines and standards.
· focus is process standardization.
· Capability level 4 : Quantitatively Managed
· process is controlled using statistical and quantitative
techniques.
· process performance and quality is understood in statistical
terms and metrics.
· quantitative objectives for process quality and performance
are established.
· Capability level 5 : Optimizing
· focuses on continually improving process performance.
· performance is improved in both ways - incremental and
innovation.
· emphasizes on studying the performance results across the
organization to ensure that common causes or issues are
identified and fixed.
· Conclusion
· CMMI provides a structured approach to process
improvement, ensuring higher quality and lower risk in
organizational processes. By following the staged or continuous
representation, organizations can achieve different maturity or
capability levels, leading to standardized, managed, and
optimized processes. This systematic improvement enhances
customer satisfaction, market reputation, and overall business
performance. CMMI is a valuable tool for organizations
seeking to integrate and enhance their processes effectively.
Malcolm Baldrige National Quality Award
The **Malcolm Baldrige National Quality Award** is a award
given to U.S. organizations for their achievements in quality
management. It was established in 1987 by the U.S. government and
is intended to recognize organizations that demonstrate excellence in
performance across seven criteria
The Malcolm Baldrige National Quality Award recognizes U.S.
organizations in the business, health care, education, and nonprofit
sectors for performance excellence. The Baldrige Award is the
highest[2] formal recognition of the performance excellence of both
public and private U.S. organizations given by the President of the
United States. It is administered by the Baldrige Performance
Excellence Program, which is based at and managed by the National
Institute of Standards and Technology (NIST), an agency of the U.S.
Department of Commerce.
The Baldrige Performance Excellence Program and the associated
award were established by the Malcolm Baldrige National Quality
Improvement Act of 1987 (Public Law 100–107). The program and
award were named for Malcolm Baldrige, who served as United
States Secretary of Commerce during the Reagan administration,
from 1981 until Baldrige's 1987 death in a rodeo accident.[3] The
first award was given November 13, 1988.[1] By 1991, The New
York Times opinionated that the criteria should be broader and
"tougher to win."[4] In 2010, the program's name was changed to
the Baldrige Performance Excellence Program.[5]
The award is not given for specific products or services.
Baldrige Excellence Framework
Overview
The Baldrige Excellence Framework has three parts: the Criteria
for Performance Excellence, core values and concepts, and scoring
guidelines. The framework serves two main purposes: (1) to help
organizations assess their improvement efforts, diagnose their
overall performance management system, and identify their
strengths and opportunities for improvement and (2) to identify
Baldrige Award recipients that will serve as role models for other
organizations. In addition, the framework and its Criteria help
strengthen U.S. competitiveness by • improving organizational
performance practices, capabilities, and results • facilitating
communication and sharing of information on best practices among
U.S. organizations of all types • serving as a tool for understanding
and managing performance and for guiding planning and
opportunities for learning • The framework provide organizations
with an integrated approach to performance management that
results in • delivery of ever-improving value to customers and
stakeholders, contributing to organizational sustainability •
improved organizational effectiveness and capabilities •
organizational and personal learning
The following three sector-specific versions of the Baldrige
framework are revised every two years:
Baldrige Excellence Framework (Business/Nonprofit)[6]
Baldrige Excellence Framework (Education)[7]
Baldrige Excellence Framework (Health Care)[8]
Framework details
The framework provides organizations with an integrated approach
to performance management that results in delivery of ever-
improving value to customers and stakeholders, contributing to
organizational sustainability improved organizational effectiveness
and capabilities organizational and personal learning
The criteria for performance excellence are based on a set of core
values:
Systems perspective
Visionary leadership
Customer-focused excellence
Valuing people
Organizational learning and agility
Focus on success
Managing for innovation
Management by fact[9]
Societal responsibility
Ethics and transparency
Delivering value and results
The questions that make up the criteria represent seven aspects of
organizational management and performance:
Leadership
Strategy
Customers
Measurement, analysis, and knowledge management
Workforce
Operations
Results
History of the Baldrige Program
In the early and mid-1980s, many U.S. industry and government
leaders saw that a renewed emphasis on quality was necessary for
doing business in an expanding and competitive world market.
The Malcolm Baldrige National Quality Improvement Act of 1987,
signed into law on August 20, 1987, was developed through the
actions of the National Productivity Advisory Committee, chaired by
Jack Grayson. The nonprofit research organization APQC, founded
by Grayson, organized the first White House Conference on
Productivity, spearheading the creation of the Malcolm Baldrige
National Quality Award in 1987. The Baldrige Award was
envisioned as a standard of excellence that would help U.S.
organizations achieve competitive quality.
In the late summer and fall of 1987, Dr. Curt Reimann, the first
director of the Malcolm Baldrige National Quality Program, and his
staff at the National Institute of Standards and Technology
developed an award implementation framework, including an
evaluation scheme, and advanced proposals for what is now the
Baldrige Award. In its first three years, the Baldrige Award was
jointly administered by APQC and the American Society for
Quality, which continues to assist in administering the award
program under contract to NIST.
Up to 18 awards may be given annually across six eligibility
categories—manufacturing, service, small business, education,
health care, and nonprofit. As of 2016, 113 awards have been
presented to 106 organizations (including seven repeat winners).[10]
Program impacts
The ratio of Baldrige Program benefits for the U.S. economy to
program costs has been estimated at 820 to 1[11]
A New York Times-generated investment portfolio composed of
Baldrige awardees "beat the S.& P. by nearly 4 to 1."[12]
Median growth in revenue for two-time Baldrige Award winners is
92%.
Median job growth for two-time Baldrige Award winners is 63%
(compared with 2.5% for a matched set of industries and time
periods).
2010–2015 Baldrige Award applicants represent 567,434 jobs, over
$142 billion in revenue/budgets, and about 449 million customers
served.
The value of services donated in 2015
by 349 national Baldrige examiners was $5.3 million.
by state Baldrige-based examiners was $30 million.
A 2013 study by Truven Health Analytics linked hospitals that adopt
and use the Baldrige Criteria to successful operations, management
practices, and overall performance.
According to survey results reported in Futurescan 2013,[13] by
2018, 65% of hospitals are likely to "use the Baldrige Criteria for
Performance Excellence as a systematic framework for performance
improvement or as an internal assessment tool"
According to the same survey, 41% of hospitals were then
considered likely to submit an application for the Baldrige Award or
a state-level Baldrige-based award by 2018.
Public-private partnership
The Baldrige Award is supported by a public-private partnership.
The following organizations and entities play a key role:
The Foundation for the Malcolm Baldrige National Quality Award
raises funds to permanently endow the award program.
The National Institute of Standards and Technology (NIST), an
agency of the U.S. Department of Commerce, manages the Baldrige
Program.
The American Society for Quality (ASQ) assists in administering the
award program under contract to NIST.[14][15]
The Board of Overseers advises the Department of Commerce on
the Baldrige Program.
Members of the Board of Examiners—consisting of leading experts
from U.S. businesses and education, health care, and nonprofit
organizations—volunteer their time to evaluate award applications
and prepare feedback reports for applicant organizations. Board
members also share information about the program in their
professional, trade, community, and state organizations. The Panel
of Judges, part of the Board of Examiners, makes award
recommendations to the director of NIST.
The network of state, regional, and local Baldrige-based award
programs known as the Alliance for Performance Excellence
provides potential award applicants and examiners, promotes the
use of the Criteria, and disseminates information on the award
process and concepts.
The ISSA, the professional association for cybersecurity, is in formal
partnership with the Baldrige Alliance for Performance Excellence.
The Alliance and the ISSA offer a free Baldrige-based self-
assessment of cybersecurity operations. It may be found at
ManageHub
Award recipients share information on their successful performance
and quality strategies with other U.S. organizations.