PSCMR CET Dept of CSE
Software Quality Assurance
SoftwareQualityAssurance:
According to Phil Crosby oncesaid:
“The problem of quality management is not what people don't know about it. The problem is
what they think theydo know “
Everybody is for it. (Under certain conditions, of course.)
Everyone feels they understand it. (Even though they wouldn't want to explain
it.)
Everyone thinks execution is only a matter of following natural inclinations.
(After all, we do get along somehow.)
Elements of SQA
1. Standards: The IEEE, ISO, and other standards organizations have produced a broad
array of software engineering standards and related documents.
2. Reviews and Audits : Technical reviews are a quality control activity performed by
software engineers for software engineers. Audits are a type of review performed by
SQA personnel with the intent of ensuring that quality guidelines are being
followed for software engineering work.
3. Software testing : It is a quality control function that has one primary goal—to find
errors. The job of SQA is to ensure that testing is properly planned and efficiently
conducted so that it has the highest likelihood of achieving its primary goal.
4. Error/defect collection and analysis : The only way to improve is to measure how
you’re doing. SQA collects and analyzes error and defect data to better understand
how errors are introduced and what software engineering activities are best suited to
eliminating them.
5. Change management: project. If it is not properly managed, change can lead to con-
fusion, and confusion almost always leads to poor quality. SQA ensures that adequate
change management practices have been instituted.
6. Education: Every software organization wants to improve its software engineering
practices. A key contributor to improvement is education of software engineers, their
managers, and other stakeholders.
7. Vendor management : Three categories of software are acquired from external
software vendors—
i. shrink-wrapped packages (e.g., Microsoft Office),
ii. a tailored shell that provides a basic skeletal structure that is custom tailored to
the needs of a purchaser
iii. contracted software that is custom designed and constructed from specifications
provided by the customer organization. The job of the SQA organization is to ensure
that high-quality software results by suggesting specific quality practices that the
vendor should follow and incorporating quality mandates as part of any contract with
an external vendor. Security management
8. Security Management: With the increase in cyber crime and new government
regulations regarding privacy, every software organization should institute policies
that protect data at all levels, establish firewall protection for WebApps,
9. Safety: Because software is almost always a pivotal component of humanrated
systems (e.g., automotive or aircraft applications), the impact of hidden defects can be
Software Engineering II [Link] - I Sem (R20)
PSCMR CET Dept of CSE
catastrophic. SQA may be responsible for assessing the impact of software failure and
for initiating those steps required to reduce risk.
10. Risk management: The analysis and mitigation of risk is the concern of software
engineers, the SQA organization ensures that risk management activities are properly
conducted and that risk-related contingency plans have been established.
SQA Tasks, Goals and Metrics:
Software engineers perform quality control activities by applying solid technical
methods and measures, conducting technical reviews, and performing well-planned
software testing.
SQA Tasks
The charter of the SQA group is to assist the software team in achieving a high quality
end product. The Software Engineering Institute recommends a set of SQA actions
that address quality assurance planning, oversight, record keeping, analysis and
reporting.
a. Prepares an SQA plan for a project: The plan is developed as part of project
planning and is reviewed by all stakeholders. Quality assurance actions performed by
the software engineering team and the SQA group are governed by the plan.
The following figure illustrates the quality paradigm shift from product assurance to process
assurance
Participates in the development of the project’s software process description.
b. Participates in the development of the project’s software process description.
The software team selects a process for the work to be performed. The SQA group
reviews the process description for compliance with organizational policy, internal
software standards, externally imposed standards and other parts of the software
project plan.
c. Reviews software engineering activities to verify compliance with the defined
software process: The SQA group identifies, documents, and tracks deviations from
the process and verifies that corrections have been made.
Software Engineering II [Link] - I Sem (R20)
PSCMR CET Dept of CSE
d. Audits designated software work products to verify compliance with those
defined as part of the software process: The SQA group reviews selected work
products; identifies, documents, and tracks deviations; verifies that corrections have
been made; and periodically reports the results of its work to the project manager.
e. Ensures that deviations in software work and work products are documented
and handled according to a documented procedure: Deviations may be
encountered in the project plan, process description, applicable standards, or software
engineering work products.
f. Records any noncompliance and reports to senior management: Noncompliance
items are tracked until they are resolved.
SQA Goals
a. Requirements quality. The correctness, completeness and consistency of the requirements model
will have a strong influence on the quality of all work products that follow.
b. Design quality. Every element of the design model should be assessed by the software team to
ensure that it exhibits high quality and that the design itself conforms to requirements.
c. Code quality. Source code and related work products (e.g., other descriptive information) must
conform to local coding standards and exhibit characteristics that will facilitate maintainability.
d. Quality control effectiveness. A software team should apply limited resources in a way that has
the highest likelihood of achieving a high quality result.
Statistical SQA
Information about software errors and defects is collected and
categorized.
An attempt is made to trace each error and defect to its
underlying cause (e.g., non-conformance to specifications, design error, violation of
standards, poor communication with the customer).
Using the Pareto principle (80 percent of the defects can be
traced to 20 percent of all possible causes), isolate the 20 percent (the vital few).
Once the vital few causes have been identified, move to
correct the problems that have caused the errors and defects.
Six-Sigma for Software Engineering
The term “six sigma” is derived from six standard deviations— 3.4 instances (defects) per million
occurrences—implying an extremely high quality standard.
The Six Sigma methodology defines three core steps:
Definecustomer requirements and deliverables and project goals via well-defined methods of
customer communication
Measurethe existing process and its output to determine current quality performance (collect
defect metrics)
Analyzedefect metrics and determine the vital few causes.
Improvethe process by eliminating the root causes of defects.
Software Engineering II [Link] - I Sem (R20)
PSCMR CET Dept of CSE
Controlthe process to ensure that future work does not reintroduce the causes of defects.
Software Reliability
Reliability of a software product essentially denotes its trustworthiness or
dependability.
Alternatively, reliability of a software product can also be defined as the probability
of the product working “correctly” over a given period of time.
Reliability of a system improves, if the number of defects in it is reduced.
It has been experimentally observed by analyzing the behavior of a large number of
programs that 90% of the execution time of a typical program is spent in executing only 10%
of the instructions in the program. These most used 10% instructions are often called the core
of the program. The rest 90% of the program statements are called non-core and are executed
only for 10% of the total execution time reliability of a product depends on the number of
latent errors exact location of the errors how the product is used which is called the execution
profile.
Reliability is difficult to measure because:
The reliability improvement due to fixing a single bug depends on where the bug is
located in the code.
The perceived reliability of a software product is highly observer dependent.
The reliability of a product keeps changing as errors are detected and fixed.
Hardware Reliability Vs Software Reliability
Hardware Components fail due to very different reasons as compared to software
components. Hardware components fail mostly due to wear and tear, whereas software
components fail due to bugs.
To fix hardware faults, one has to either replace or repair the failed part. On the other
hand, a software product would continue to fail until the error is tracked down and either the
design or the code is changed.
For this reason, when hardware is repaired its reliability is maintained at the level that
existed before the failure occurred; whereas when a software failure is repaired, the reliability
may either increase or decrease (reliability may decrease if a bug introduces new errors).
Hardware reliability study is concerned with stability (for example, inter-failure times remain
constant). On the other hand, software reliability study aims at reliability growth (i.e. inter-
failure times increase).
Software Engineering II [Link] - I Sem (R20)
PSCMR CET Dept of CSE
Reliability Metrics
1. Rate of Occurrence of Failure (ROCOF): This measures the frequency of
occurrence of failures. It is actually the ratio of total number of failures observed and
the duration of observation
2. Mean Time To Failure (MTTF): MTTF is the time between two successive failures,
We can record the failure data for n failure. Let the failures occur at time instances t1,
t2, t3, t4….tn then the MTTF can be calculated as
3. Mean Time to Repair (MTTR): Measures the average time it takes to track the
errors causing the failure and to fix them
4. Mean Time Between Failure(MTBR): Combination of MTTF and MTTR
MTBR = MTTF + MTTR
MTBF is 300 hrs means that the next failure is expected after 300 hours.
5. Probability of Failure On Demand: POFOD measures the likelihood of the system
failing when a service request is made.
For example, a POFOD of 0.001 would mean that 1 out of every 1000 service
requests would result in a failure.
6. Availability: Measure of how likely would the system be available for use over a
given period of time
Software availability is the probability that a program is operating according to requirements
at a given point in time and is defined as
Availability = [MTTF/(MTTF + MTTR)] x 100%
Software Safety
Software safety is a software quality assurance activity that focuses on the identification and
assessment of potential hazards that may affect software negatively and cause an entire
system tofail.
If hazards can be identified early in the software process, software design features can be specified
that will either eliminate or control potential hazards.
ISO 9001:2000 Standard
ISO 9001:2000 is the quality assurance standard that applies to software engineering.
The standard contains 20 requirements that must be present for an effective quality assurance system.
The requirements delineated by ISO 9001:2000 address topics such as
Management responsibility, quality system, contract review, design control, document and
data control, product identification and traceability, process control, inspection and testing,
Software Engineering II [Link] - I Sem (R20)
PSCMR CET Dept of CSE
corrective and preventive action, control of quality records, internal quality audits, training,
servicing, and statistical techniques.
Software Engineering II [Link] - I Sem (R20)