Software Testing Unit - 1
Software Testing Unit - 1
Test-Driven Development
Testing outputs guide and validate the entire development process
Bridging the Gap
The Problem 1
Graduates enter the workforce without
essential testing knowledge, facing
steep learning curves and initial 2 The Solution
struggles.
Dedicated courses in
software quality and
The Future 3 testing that prepare
A generation of developers, testers, and students for industry
engineers equipped with comprehensive demands.
testing expertise from day one.
Essential Knowledge for
Tomorrow
Testing ideas and techniques have become
essential knowledge for software developers,
testers, and students alike.
Destruction-Oriented (1979–82) 3
Myers revolutionized testing:
finding errors became the goal
4 Evaluation-Oriented (1983–87)
Early testing and quality evaluation at
every development stage
Prevention-Oriented (1988–95) 5
Preventing bugs before they
occur through planning and design
Process-Oriented (1996–Present)
6
Testing as a complete, measurable
process throughout SDLC
Phase 1: Before 1957
Debugging-Oriented Phase
In the earliest days of software, testing was an informal
activity. Programmers wrote code and then tested it
themselves until they believed all bugs were eliminated.
The primary goal was simple: get the system to run.
The Gap
Little emphasis on test case design or strategic testing approaches
Phase 3: 1979–1982
Myers fundamentally changed how we think about testing. This revolutionary phase marked a critical turning point in software testing
history.
01 02
Paradigm Shift Testing vs. Debugging
From proving absence of errors to actively Clear separation established between
hunting for bugs these two activities
03 04
Value of Test Cases Early Testing
Emphasis on tests that uncover Recognition that testing should begin
more bugs effectively earlier in development
Phase 4: 1983–1987
Evaluation-Oriented Phase
A crucial realization emerged: bugs found early cost less to fix. This
phase transformed testing into a continuous quality evaluation activity
throughout the entire development lifecycle.
National Standards
In 1983, the National Bureau of
Standards released guidelines for
selecting V&V techniques to evaluate
software at each development step
Phase 5: 1988–1995
Prevention-Oriented Phase
Test Planning
Process-Oriented Phase
Testing matured from a single post-coding phase into a comprehensive
process integrated throughout the software development lifecycle. This
transformation established testing as parallel to development, not
sequential.
Complete Process
Testing runs parallel to SDLC from requirements onward
Measurable Quality
Testing Maturity Model (TMM) measures process performance
Quantification
Emphasis on metrics that determine testing effectiveness
The evolution of software testing was also discussed by Hung Q. Nguyen and Rob Pirozzi in
a white paper [81],
These three phases discuss the evolution in the earlier phases that we described.
01
Process-Based Framework
A defined roadmap guides the entire testing process from planning through execution.
02
Quality-Driven Goals
Clear quality objectives enable effective monitoring and control of testing activities.
03
Active Management
Leadership is deeply involved, making data-driven decisions to ensure quality outcomes.
04
Strategic Integration
Testing is woven throughout the SDLC, not treated as a separate final phase.
Software Testing: Myths and Facts
1. Myth: Testing is a single phase 2. Myth: Testing is easy 3. Myth: Software development
in SDLC Fact: is more important than testing
Fact: •Testing is not an easy task
• Testing is not limited to one •Tools help in automation, not Fact:
phase of SDLC complete testing • Testing is as important as
• Testing begins from •Testers must: development
requirement analysis stage • Understand requirements • Developers focus on building
• Continues through design, and system design software
coding, implementation, and • Manually design test cases • Testers ensure quality and
maintenance • Think critically to find reliability
• Includes verification and defects • Testing is now a well-
validation at every stage •Testing often involves more established career path
• Post-implementation testing is responsibility than development • Both teams have equal status
also required and importance
Software Testing: Myths and Facts
4. Myth: Complete testing is possible 5. Myth: Testing starts after program
development
Fact: Fact:
• Complete testing is not possible • Testing starts before coding
• Input combinations are too large • Testers work during:
• Exhaustive testing may take years • Requirement analysis
• Focus is on effective testing • Design phase
• Objective is to find maximum • Coding phase
critical defects with limited test • Activities include:
cases • Test planning
• Test case design
• Verification & validation
• Testing after coding is only one part of
testing
Software Testing: Myths and Facts
6. Myth: Testing only checks 7. Myth: Anyone can be a tester
functionality
Fact: Fact:
• Testing ensures overall quality, not just • Testing requires specialized skills and
functionality training
• Includes testing of:
• Performance • Testers must know:
• Security • Software Testing Life Cycle (STLC)
• Usability • Test case design techniques
• Reliability • Testing tools and automation
• Compatibility • Professional certifications are available
• Quality goals drive all testing activities for testers
Objectives and Limitations of Software Testing
Objectives of Software Testing:
Software testing is a systematic process carried out to ensure that a software system meets specified requirements and works
correctly under given conditions.
Post-implementation
Immediate Goals Long-term Goals
Goals
Reduced Improved
Bug Bug Customer Risk
Reliability Quality maintenance testing
discovery prevention satisfaction management
cost process
I. Short-Term / Immediate Goals of Software Testing:
• Short-term goals focus on immediate outcomes achieved during the execution of testing activities. These
goals are usually associated with individual phases of the SDLC.
1. Bug Discovery 2. Bug Prevention
•The primary and immediate goal of software testing •Bug prevention is a consequent action that follows bug
is to detect bugs or defects in the software. discovery.
•Bugs may occur during requirements analysis, •By analyzing:
design, coding, or integration. • The type of bugs
•Early detection of defects is crucial because: • Their root causes
• It reduces the cost of fixing errors. • Their frequency and impact
• It prevents defect propagation to later stages. developers and testers learn how to avoid repeating
•The earlier the bug is discovered, the higher the similar mistakes.
success rate of software testing and development. •Bug prevention helps in:
Key Point: • Writing safer and cleaner code
Early bug discovery leads to faster correction and • Improving design and coding standards
improved software quality. • Enhancing development practices
•Although it is impossible to eliminate errors completely, testing
helps minimize them significantly.
Key Point:
Bug prevention is a superior goal of testing because it reduces
future defects and improves development maturity.
II. Long-Term Goals of Software Testing
Long-term goals influence the overall product quality and organizational success after one complete SDLC cycle.
1. Quality Improvement 2. Customer Satisfaction
•Quality is the primary concern from the user’s •The ultimate objective of software testing is to achieve
perspective. customer satisfaction.
•Thorough and systematic testing ensures: •Customers expect:
• Correctness • Software to meet all specified requirements
• Integrity • Software to handle unspecified but implied user
• Efficiency expectations
• Usability •Complete and thorough testing ensures:
• Reliability • Fewer failures after release
•Among all quality attributes, reliability plays a major • Smooth user experience
role. •When software is reliable and high in quality, customer trust
•Reliability refers to the confidence that the software increases.
will not fail under specified conditions.
•Rigorous testing increases reliability, and higher
reliability leads to better quality.
Key Point: Testing enhances reliability, and reliability is Key Point: Quality achieved through testing directly results in
the foundation of software quality. customer satisfaction.
3. Risk Management
•Risk is the probability of undesirable events that may affect business objectives.
•Examples of risks:
• Cost overruns
• Delayed delivery
• Critical defects in production
• Resource shortages
•Software testing acts as a control mechanism to:
• Identify potential risks early
• Minimize or eliminate risks
•Testing provides valuable information to management, such as:
• Project delays
• Unresolved high-priority defects
• Performance or security risks
•Testers assess and categorize risks as:
• High risk
• Medium risk
• Low risk
•Risk analysis becomes the basis for testing priorities and strategies.
Key Point:
Testing supports management decisions by controlling business risks related to cost, time, and quality.
III. Post-Implementation Goals of Software Testing
Post-implementation goals focus on benefits achieved after the software is released.
8. Changing Requirements
Frequent changes in requirements can invalidate existing test cases, increasing rework and reducing testing effectiveness.
9. Limited Coverage
Due to constraints, testing often focuses on critical functionalities, leaving some areas less tested or untested.
• Meaning of Failure: Failure occurs when the software does not behave
as expected during testing.
• Example
1. Requirement: Login should succeed with correct username and password
2. Actual Result: Login fails even with correct credentials
→ This situation is a failure
• Origin of a Fault: A fault is introduced during any phase of the SDLC, such as:
• Requirement analysis
• Design
• Coding
• Maintenance
• Fault Causes Failure: A fault is the actual cause that leads the system to
produce an incorrect result. When a fault is executed, it may result in a failure.
• Execution Dependency
A bug produces a failure only when it is executed.
If the faulty code is not executed, no failure will be observed.
• Hidden Bugs
Some bugs are hidden because:
• Required conditions never occur
• Rare input or environment is needed
• Example:
• Developer writes + instead of - in a calculation.
• Analyst misunderstands a business rule.
Software
Testing
Life
Cycle
• Based on the planning issues as discussed above,
Test Planning: analysis is done for various testing activities.
• The major output of test planning is the test plan
• The goal of test planning is to take into account the important
document.
issues of testing strategy, viz. resources, schedules,
• Test plans are developed for each level of testing.
responsibilities, risks, and priorities, as a roadmap.
• Test planning issues are in tune with the overall project
• After analysing the issues, the following activities are
planning.
performed:
• following are the activities during test planning:
• Develop a test case format.
• Defining the test strategy.
• Develop test case plans according to every phase
• Estimate the number of test cases, their duration, and
of SDLC.
cost.
• Identify test cases to be automated (if applicable).
• Plan the resources like the manpower to test, tools
• Prioritize the test cases according to their
required, documents required.
importance and criticality.
• Identifying areas of risks.
• Define areas of stress and performance testing.
• Defining the test completion criteria.
• Plan the test cycles required for regression testing.
• Identification of methodologies, techniques, and tools
for various testcases.
• Identifying reporting procedures, bug classification,
databases for testing, bug severity levels, and project
metrics.
Test Design
• Test design is a systematic and planned activity of creating test cases.
• It is not an intuitive process; it follows well-defined steps.
• This phase starts after test planning and before test execution.
• The main aim is to ensure maximum test coverage with minimum test cases.
Phases of Test Design
1. Determining Test Objectives and Prioritization: 3. Mapping Items to Test Cases
• Test objectives define what needs to be tested. • A traceability matrix is created to map:
• Objectives are identified using reference documents • Test objectives → Test items → Test cases
such as: • Existing test cases can be reused, saving time and effort.
• Software Requirement Specification (SRS) • This mapping helps in:
• Design documents (a) Identifying major test scenarios
• A team of experts prepares a list of test objectives. (b) Removing redundant or duplicate test cases
• Objectives are prioritized based on risk, (c) Detecting missing test cases and creating new ones
importance, and scope.
• High-risk and critical features are given higher 4. Understanding System Behavior
priority.
• Test case design requires:
2. Preparing List of Items to Be Tested • Functional understanding (what the system does)
• Structural understanding (how the system works
• Each test objective is broken down into testable internally)
items. • The tester must understand cause-and-effect
• These items represent features, functions, or relationships within the system.
conditions to be tested. • As per Tsuneo Yamaura:
• This step converts abstract objectives into
concrete testing elements. “Cover all features, but do not make too many test cases.”
5. Attributes of a Good Test Case 6. Selection of Test Case Design Techniques
• A good test case has the following qualities: • Test cases are designed using two main approaches:
(a) Risk-focused: • Black-box testing: Based on functionality without
• Gives higher priority to critical and high-risk knowing internal code
requirements. • White-box testing: Based on internal structure and
(b) Error-revealing: logic
• Has a high probability of finding defects. • Techniques are selected to ensure:
(c) Non-redundant: • Maximum coverage
• Each test case covers only one unique • Maximum defect detection
functionality.
• Avoids overlapping and duplication.
(d) Modular: 7. Creating Test Cases and Test Data
• Designed in small, independent units.
• Can be reused and combined for different • Test cases are created based on identified objectives.
execution paths. • Each test case includes:
• Prevents masking of errors. • Test objective
(e) Effective: • Input data
• A successful test case is one that finds previously • Expected output
undiscovered defects. • Test data must be carefully chosen:
• Incorrect test data can lead to wrong test results.
8. Setting Up Test Environment and Tools 9. Creating Test Procedure Specification
• Defines the environment required to execute test cases, such • Specifies step-by-step instructions to execute a test case.
as: • Includes:
• Hardware configuration • Sequence of actions
• Software and operating systems • Inputs to be entered
• Test tools • Expected results at each step
• Interfaces and manuals • Used directly by testers during test execution.
• Ensures that test execution conditions match real usage.
• All test design details are documented in the Test Design • The test design phase follows this hierarchy:
Specification. 1. Develop test objectives
• This document includes: 2. Identify and design test cases
• Test objectives 3. Create test case specifications
• Test case specifications 4. Develop test procedure specifications
• Input and output specifications
• Environmental and procedural requirements
Test Execution
• Test execution is the phase where all test cases are run.
• It includes both verification and validation testing.
• Verification test cases are executed at the end of each SDLC phase.
• Validation test cases are executed after completion of a module.
• Test cases may be executed manually or using automation tools.
• Actual results are compared with expected results.
• Test outcomes are recorded as pass, fail, or blocked.
• Defects found are documented for correction.
• Understand the reported bug and its location. • Results from manual and automated testing are
• Reproduce the bug to confirm the failure. collected.
• Analyze the nature and root cause of the bug. • Final bug reports and testing metrics are reviewed.
• Debug and fix the error in the software. • Reliability analysis checks whether reliability goals are
• Inform the testing team after fixing the bug. met:
• The modified software is retested by testers. • If met → product can be released.
• If not → additional time and resources are planned.
4. Early Testing Is the Best Policy 5. Defects Occur in Clusters 6. Testing Should Start at Module
Level
• Testing should begin at the • Modules with more past defects are • Testing should follow an incremental
requirement stage. more error-prone. approach:
• Bugs detected early are: • Example: • Unit testing
• Easier to fix • Module X → many defects • Integration testing
• Less costly • Module Y → few defects • System testing
• If bugs are found late, the cost of • Testing should focus more on • Testing the complete system directly is
fixing them increases many times. modules with defect history. not effective.
• Modules must be tested first and then
combined gradually.
7. Independent Testing Is More 8. Everything Must Be Documented 9. Invalid Inputs Have High Defect-
Effective Testing is a planned and Finding Capability
documented activity.
• Developers are biased toward their • Records include: • Testing should not be limited to valid
own code. Test cases inputs.
• They assume their code works Inputs and outputs • Invalid and unexpected inputs:
correctly. Defects and observations Often reveal hidden defects
• Independent testers: • Documentation helps in: Improve software robustness
• Have a fresh perspective Bug tracking • Negative testing is essential for quality
• Use a destructive mindset Process improvement assurance.
• External or independent testing Future projects
teams find more defects.
11. Relationship Between
10. Testers Should Participate in Specification (S), Program (P), 12. Goal of Good Testing
Requirement and Design Reviews and Tests (T)
• Testers should be involved early in: S → Specified behaviors • Maximize the overlapping area of S,
Requirement reviews P → Program implementation P, and T.
Design discussions T → Test cases • This ensures:
• This ensures: Possible problems: • All specifications are
All requirements are testable • Some specifications are not tested. implemented
No unnecessary or missing test • Some tests check unspecified • All implemented behaviors are
cases behavior. tested
• Early involvement improves test • Some program behaviors are not • Achieved by early tester involvement
coverage. tested. in SDLC.
Testing
Methodology
1. Testing Strategy (Top Level)
• It is influenced by:
• Test factors → Risk, complexity, type of software, criticality
• Test phases → Different stages of testing in SDLC
2. Verification and Validation
The testing strategy is divided into Verification and Validation.
a) Verification – “Are we building the product right?” b) Validation – “Are we building the right product?”
• Focuses on process • Focuses on product
• Ensures software is built according to specifications • Ensures software meets user requirements
• Mostly static testing • Uses dynamic testing
• Examples: • Code is executed
• Requirement review
• Design review • Validation includes different testing levels:
• Code inspection • Unit Testing → Individual modules tested
• No execution of code • Integration Testing → Combined modules tested
• Finds errors early • System Testing → Entire system tested as a whole
3. Testing Tactics 4. Testing Techniques
• After verification and validation, we decide how Testing techniques describe methods used to test software.
exactly to test. This is called Testing Tactics. a) Static Testing
• Testing tactics include: • No execution of code
Testing techniques • Used in verification
Testing tools • Examples:
• Reviews, Walkthroughs, Inspections