1.
Introduction to Software Testing
Contents
• Overview of Software Testing
• Importance of Software Quality
• Testing Concepts and Principles
• Difference between Verification and Validation
• Difference between Error, Bug, Defect, and Failure
• Project vs Product
• Objectives of Testing
• Introduction to Quality Assurance (QA) and Quality Control (QC)
• Roles and Responsibilities of a Software Tester
• Skills Required for a Successful Tester
Software That Doesn’t Work
Life Money Time Reputation
BSOD : July 2024
BSOD
Y2K Failure (Millennium Bug)
Iridium Satellite Phone
• Company had spent $5 billion to build and launch its
infrastructure of satellites to provide worldwide wireless
phone service
• Had only 10,000 subscribers whereas expectation was
500,000 subscribers
• Since the service was expensive and handset cost was
$3,000, and talk time was as much as $5 a minute
Software Failures
• Security Issues in Zoom App
• Memory Failure - Tesla Recalls 135,000 Vehicles
to fix computer memory issues in the touchscreen that could cause a loss of
some features
• Yahoo Confirmation on Massive Data Breach,confirmed by Yahoo itself on
September 22, 2016. Yahoo believes that information associated with about
500 million user accounts was stolen.
1.1 What is SoftwareTesting?
Ø common misperception of testing is that it only consists of running tests
Ø Test activities exist throughout the software development lifecycle.
Software testing is a way to
assess the quality of the
software and to reduce the
risk of software failure in
operation
1.1 What is Testing?
Ø Testing may involve execution(Dynamic testing) or non-execution(Static testing)
Ø Testing contributes to reviews.
Ø Testing offers both validation and verification
Why is Testing Necessary?
Testing Contribution to success
• Testing provides a cost-effective means of detecting defects. These defects can then be removed
(by debugging – a non-testing activity), so testing indirectly contributes to higher quality test objects
• Testing provides a means of directly evaluating the quality of a test object at various stages in the
SDLC.
• These measures are used as part of a larger project management activity, contributing to decisions
to move to the next stage of the SDLC, such as the release decision.
• Testers ensure that their understanding of users’ needs are considered throughout the development
lifecycle.
• Testing may also be required to meet contractual or legal requirements, or to comply with regulatory
standards.
Why Testing is Necessary?
• Because software is likely to have faults
• To learn about the reliability of the software
• To fill the time between delivery of the software and the release date
• To prove that the software has no faults
• Because testing is included in the project plan
• Because failures can be very expensive
• To avoid being sued by customers
• To stay in business
Verification and Validation
ISO 9000 Definition:
• ISO 9000 defines verification as "confirmation by objective evidence that specified
requirements have been fulfilled". This means that verification provides proof, through
documented evidence, that the product or service meets the pre-defined requirements.
Verification vs. Validation:
• While both terms involve confirmation, verification focuses on whether the output meets
the requirements as defined, while validation focuses on whether the output is suitable
for its intended use.
• For example, verification might check if a product's dimensions match the specifications,
while validation would check if those dimensions allow the product to function as intended.
Testing and Debugging
Testing and Debugging are different.
Debugging process involves: Testing can trigger :
• Reproduction of a failure • Failures that are caused by defects in the software
• Diagnosis (finding the root cause) (dynamic testing)
• Fixing the cause or
• Directly find defects in the test object (static testing).
• Confirmation testing checks whether the fixes resolved the problem
• Subsequent regression testing can also be performed
• When static testing identifies a defect, debugging is concerned with removing it.
Error,Defects,Failures and Root Causes
Error
(Mistake)
Fault\Defect\Bug
Failure
Ø Bug is informal name of a defect
Ø Fault executed MAY CAUSE failure.
Ø Failure is an event; defect is a state of the software, caused by an error
• Errors may occur for many reasons, such as:
• time pressure
• complexity of work products, processes, infrastructure or interactions
• simply because they are tired
• lack adequate training.
will always result in a failure if executed
• Some defects
will only result in a failure in specific circumstances
may never result in a failure.(condition that can cause the
failure will never arise)
• Failures can also be caused by environmental conditions, such as when radiation or electromagnetic
field cause defects in firmware.
Root Cause
Root Causes and Effects
Root Cause
– A source of a defect such that if it is removed, the occurrence of the defect type is
decreased or removed.
– Should be identified to avoid similar defects in the future
– Root cause analysis can lead to process improvements
Identifying the Root Cause
– 5 why’s
Defects, Root Causes and Effects
• Root Cause
– A source of a defect such that if it is
removed, the occurrence of the defect
type is decreased or removed.
– Should be identified to avoid similar
Symptoms
defects in the future
– Root cause analysis can lead to
process improvements
• Identifying the Root Cause
– 5 why’s Root
causes
Root Cause - Exercise
Situation:
Incorrect bonus amounts are calculated in HR software and customers of
the software have started to complain. Several failures have been
reported to this development team where calculations were not done as
they should be done. The code was written on requirements that were
unclear as the Product Owner did not understand the calculation for
bonuses.
What is the effect?
What is the root cause?
What is the solution?
Product and Project
• A project is a temporary endeavor with a defined beginning and end, aimed at creating a
unique output or achieving a specific goal.
Example : Add online payment feature to the shopping app in 3 months.
• A product is an ongoing offering designed to fulfill customer needs and generate continuous
value
• A product is a software application or system created to solve a user problem and provide
long-term value.
Examples :Microsoft Word ,WhatsApp ,Google Chrome
Product and Project
Product Project
Permanent/long-term Temporary
Focus on users/customers Focus on completing task
Continuously evolves Ends after delivery
Example: Mobile app Example: Developing a new feature
Typical Objectives of Testing
The typical test objectives are:
• Evaluating work products such as requirements, user stories, designs, and code
• Triggering failures and finding defects
• Ensuring required coverage of a test object
• Reducing the level of risk of inadequate software quality
• Verifying whether specified requirements have been fulfilled
• Verifying that a test object complies with contractual, legal, and regulatory requirements
• Providing information to stakeholders to allow them to make informed decisions
• Building confidence in the quality of the test object
• Validating whether the test object is complete and works as expected by the stakeholders
contractual, legal, and regulatory requirements ….
Key Aspects of Software Requirements:
Contractual Obligations:
These arise from agreements between parties, such as software licenses, service level agreements (SLAs), and non-
disclosure agreements (NDAs). They often specify security measures, data protection protocols, and performance
standards.
Legal Requirements:
These are laws and statutes that govern software development, use, and data handling. Examples include copyright
law, data protection laws (like GDPR or CCPA), and export control regulations.
Regulatory Requirements:
These are specific rules and guidelines established by government or industry bodies. Examples include PCI DSS for
payment card data, HIPAA for healthcare data, and FISMA for government systems.
Quality
The degree to which a component or system
satisfies the stated and implied needs of its
various stakeholders
Standards and Regulations such as ISO 9000
series
Quality Metrics
• Defect Rate
• Customer Satisfaction
• User Ratings
• Failure Rate
• Service Quality
Quality Assurance and Testing
• Quality assurance and testing are not the same, but they are related.
• Quality management includes all activities from the software development life cycle regarding quality
• Quality management includes
– Quality Assurance
• Focused on adherence to proper processes
• In order to provide confidence that the appropriate levels of quality will be achieved
• Better the process, better will be the software
• QA supports proper testing
• Use of root cause analysis or retrospective meetings is recommended by QA
• QA applies to both the development and testing processes
– Quality Control
• Involves set of (test) activities
• Support to evaluate level of quality of a component or system
Test results are used by QA and testing.
-In testing they are used to fix defects
-In QA they provide feedback on how well the development and test processes are performing
Quality Assurance and Testing
QA QC
Process Oriented Project Oriented
Defect prevention(Preventive approach) Defect Identification(Corrective
Approach)
Everyone’s responsibility Specific team’s responsibility
• While people often use the terms “testing” and “quality
assurance” (QA) interchangeably, testing and QA are not the
same
• Testing is a major form of QC
Testing Principles
[Link] Shows presence of defects
[Link] testing is impossible
[Link] testing saves time and money
[Link] cluster together
[Link] wear Out
[Link] is Context dependent
[Link] of defects fallacy
Seven Testing Principles
Testing Shows the presence of defects, not their absence
Testing can show that defects are present, but cannot prove that there are no defects.
Testing reduces the probability of undiscovered defects remaining in the software but,
even if no defects are found, testing is not a proof of correctness.
Even when no defects are found during testing, it can’t be proved that
software is defect-free.
Seven Testing Principles
Exhaustive testing is impossible
Testing everything (all combinations of inputs and preconditions) is not feasible
except for trivial cases. Rather than attempting to test exhaustively, risk analysis,
test techniques, and priorities should be used to focus test efforts.
Seven Testing Principles
Early testing saves time and money
To find defects early, both static and dynamic test activities should be started as
early as possible in the software development lifecycle. Early testing is sometimes
referred to as shift left. Testing early in the software development lifecycle helps
reduce or eliminate costly changes
Seven Testing Principles
Defects cluster together
A small number of modules usually contains most of the defects discovered
during pre-release testing, or is responsible for most of the operational
failures.
Testers must cosider and prepare proportional test cases to test such
system.
Seven Testing Principles
Tests Wear Out(Pesticide Paradox)
If the same tests are repeated over and over again, eventually these
tests no longer find any new defects.
To detect new defects, existing tests and test data may need changing,
and new tests may need to be written.
Seven Testing Principles
Testing is context dependent
Testing is done differently in different contexts. For example, safety-critical
industrial control software is tested differently from an e-commerce mobile
app. As another example, testing in an Agile project is done differently than
testing in a sequential lifecycle project
Seven Testing Principles
Absence-of-defects fallacy
Meeting the requirement is equally important.
Detecting and fixing all the defects is of no use if the system does not
fulfill the user needs and expectations.
Test Basis
• Identify the values which you would test to validate the below stated requirement.
• Requirement: “A field on the form accepts the whole numbers from range 0 to 100”
• Test Basis: The body of knowledge used as the basis for test analysis and design.
• Body of knowledge: All documents describing requirements, architecture, design,
interfaces, structure and behavior
Test Condition
• Requirement: “A field on the form accepts the whole numbers from range 0 to 100”
• Test Conditions:
– Range 0 to 100
– Whole numbers
– Numbers
• Test condition is a testable aspect of a component or system identified as a basis for testing
High Level and Low Level Test Cases
Test Procedures and Test Script
• Test procedure: A sequence of test cases in execution order, and any associated actions that
may be required to set up the initial preconditions and any wrap up activities post execution.
• Test script: A sequence of instructions for the execution of a test
Good Read : [Link]
Test Suite
A set of test scripts or test procedures to be
executed in a specific test run
Should match your planning and analysis needs
Examples
– Functional Tests
– Performance Tests
– Smoke Tests
– Lengthy Tests
Test Object
• The work product to be tested
• System / software under test (SUT)
Test Oracle
A source to determine an expected result to compare with the actual result of the system under
test.
Examples:
– Requirements: User requirements, system requirements, user stories, regulatory
requirements etc.
– Code specifications: Design documents, APIs, code comments, and other documentation.
– Historical data: Whether the software is producing output that is consistent with past
behavior, Defect information
– Expert judgment: When there are no other sources of information available
– Manual testing: Testers can manually execute test cases to identify the expected behavior.
– Automated testing: Automated testing can be used as a test oracle to verify the expected
behavior of software or systems.
– Customer feedback: Customer feedback can be used as a test oracle to verify whether the
software or system meets the needs of the customer.
– Industry standards: Industry standards include standards for security, performance,
usability, and other areas.
Test Data
Test Data: Data needed for test execution.
Test Data Preparation: The activity to select data
from existing databases or create, generate,
manipulate and edit data for testing.
One shouldn’t rely on standard test data as it is
better to create the custom ones.
Ways to create the test data
– Manually
– Import from the production environment
– Duplication from prior customer systems
– Using test automation tools
Roles and Responsibilities of a Software Tester
Role of a Software Tester
• Verifying software quality
• Ensuring compliance with requirements
• Detecting defects early
• Supporting process improvement
• Helping deliver reliable software products
Responsibilities of a Software Tester
• Requirement Analysis
• Test Planning
• Test Case Design
• Test Execution
• Defect Management
• Regression and Re-testing
• Documentation and Reporting
• Quality Assurance Activities
• Risk Identification
• Collaboration
The Essential Skills and Good Practices in Testing
Skill is the ability to do something well that comes from one’s knowledge, practice and [Link]
testers should be effective team players and should be able to perform testing on different levels of test
independence.
Generic Skills Required for Testing
• Testing knowledge (to increase effectiveness of testing, e.g., by using test techniques)
• Thoroughness, carefulness, curiosity, attention to details, being methodical (to identify defects,
especially the ones that are difficult to find)
• Good communication skills, active listening, being a team player (to interact effectively with all
stakeholders, to convey information to others, to be understood, and to report and discuss defects)
• Analytical thinking, critical thinking, creativity (to increase effectiveness of testing)
• Technical knowledge (to increase efficiency of testing, e.g., by using appropriate test tools)
• Domain knowledge (to be able to understand and to communicate with end users/business
representatives)
The Essential Skills and Good Practices in Testing
Generic Skills Required for Testing
• Communication skills is crucial for testers.
• Communicating test results may be perceived as criticism of the product and of its author.
Confirmation bias can make it difficult to accept information that disagrees with currently held beliefs.
• Some people may perceive testing as a destructive activity, even though it contributes greatly to project
success and product quality. To try to improve this view, information about defects and failures should
be communicated in a constructive way.
Quiz
1. Software project can be executed successfully without testing
2. The main objective of testing is to find all the defects
3. Poorly documented code can result into Software defects
4. Testing and Quality Assurance (QA) are same
5. Dynamic testing involves executing the software to check whether it works properly
6. One can test the code even before it completes
7. It is better to Involve tester once the coding is complete rather than involving them before
8. Testing is independent of time, place and circumstance
9. The cost of fixing the defect goes higher the later they are found
10. Every project must involve a third-party tester
Thank you !!!