Preparing a Test Plan – Explanation

What is a Test Plan?

A Test Plan is a detailed document that describes the strategy, scope, approach, resources, and schedule of testing activities.
It acts as the anchor for the entire testing process — from execution to tracking and reporting.

Key Components of a Test Plan

No. Component Description

Define the scope of testing clearly:

1️⃣ What needs to be tested • What will be tested (in-scope)
• What will not be tested (out-of-scope)

Break down testing into smaller, manageable tasks.

Specify the testing strategies, such as:
2️⃣ How testing will be done
• Manual vs. Automated
• Functional vs. Non-functional

Identify both:
3️⃣ Resources required • Hardware/software resources (computers, tools, environments)
• Human resources (testers, developers, coordinators)

Clearly state the schedule:

4️⃣ Timelines • When each testing phase begins and ends
• Milestones and deadlines for test activities

List the possible risks, such as:

• Delays in development
5️⃣ Risks and mitigation • Resource unavailability
• Tool failures
Then provide a contingency plan or solution for each risk.

Why is a Test Plan Important?

• Acts as a roadmap for the testing team

• Helps in organizing tasks and responsibilities

• Ensures efficient use of time and resources

• Helps track progress and manage risks

• Serves as a reference document for stakeholders

Quote to Remember:
"Failing to plan is planning to fail."

Scope Management: Deciding Features to be Tested / Not Tested

What is Scope Management?

Scope management in testing refers to the process of defining and controlling what will and will not be tested in a project. This ensures that
testing efforts are focused on the most important aspects and prevents wasted resources on irrelevant tasks.

Steps in Scope Management

1️. Understanding the release of a product

o First step: Clearly define what constitutes a release of the product (e.g., version 1️.0, update 2️.1️, etc.).

o Ensure all stakeholders agree on the scope of the release.

2️. Breaking down the release into features

 o Feature identification: Identify the different features or components of the product that need to be tested.

3️. Prioritizing the features for testing

o Based on importance, criticality, and risk, prioritize the features to be tested first.

4️. Deciding which features will be tested and which will not

o Make clear decisions on features that will be in-scope for testing and those that will be out-of-scope.

o This may be based on resources, time constraints, and project needs.

5️. Gathering details for resource estimation

o Gather technical and functional details about the features to estimate the resources (time, human, tools) required for testing.

Factors Influencing Test Scope

Features that drive prioritization and testing decisions:

No. Factor Description

New and critical

1️⃣ Features that are new to the release or critical for product success are tested first.
features

Catastrophic failure
2️⃣ Features whose failure would have a high impact or cause major issues are prioritized for testing.
risk

Features that are expected to be complex to test (e.g., integrations, APIs) should be handled carefully and tested
3️⃣ Complexity of testing
thoroughly.

Features that have been problematic in previous releases or are extensions of earlier features with defects should
4️⃣ Defect-prone features
be given special attention.

Summary of Scope Management

• Scope management helps in focusing testing efforts on the most important features, while avoiding unnecessary work.

• It requires understanding the release, prioritizing features, and making tough decisions on which features to test or not test.

• The right approach minimizes duplication of effort and ensures the critical features are properly tested.

Deciding Test Approach/Strategy

Once the features to be tested have been prioritized, the next step is to define the detailed test approach/strategy. This step helps in
estimating the size, effort, and schedule required for testing by answering several key questions about the testing process:

Key Elements to Identify:

1️. Type of Testing for Each Functionality:

o Determine which testing techniques (e.g., functional, performance, security) are best suited for each feature.

o Choose between manual or automated testing based on the nature of the feature.

2️. Configurations/Scenarios for Testing the Features:

o Identify the different configurations and scenarios under which the feature should be tested.

o This includes different environments, platforms, and use cases to ensure comprehensive coverage.

3️. Integration Testing:

o Identify which integration tests need to be performed to ensure that the different features or components work together as
expected.

o This checks how the new features integrate with existing ones, ensuring no disruption to the overall system.

4️. Localization Validation:

o For global products, determine if localization tests are necessary to ensure the product works as expected in different languages
and regions.
 o This is essential if the product will be used in multiple countries with different languages and formats.

5️. Non-Functional Testing:

o Define which non-functional tests (e.g., performance, scalability, usability, security) are needed.

o These tests evaluate aspects of the product that are not directly related to functionality but are critical to its overall performance
and user experience.

Test Approach/Strategy in the Test Plan:

• The test strategy part of the test plan defines the appropriate methods and tools for testing the chosen features.

• It specifies the types of testing and configurations that will be applied to ensure full coverage.

Objective Criteria for Success:

• There should be clear, objective criteria to evaluate whether a test has passed or failed. These criteria help in measuring the success of
the testing process, ensuring that all necessary checks have been completed and the product meets the required standards.

Clear entry and exit criteria are essential for the effective execution of testing phases. These criteria help in defining when to begin testing and
when to conclude a testing activity.

Entry Criteria:

• Purpose: Specifies the threshold conditions that must be met before testing begins.

• Types of Entry Criteria:

1️. Test-Specific Entry Criteria: These define the conditions for each phase or type of test (e.g., unit testing, integration testing,
system testing). They ensure that prerequisites such as code development or environment setup are in place before testing
starts.

2️. Overall Testing Entry Criteria: These could specify general conditions for the entire testing effort, such as:

▪ Completion of development of features to be tested.

▪ Test environment readiness.

▪ Availability of necessary resources (testers, tools, etc.).

exit Criteria:

• Purpose: Specifies when a test cycle or testing activity can be considered complete.

• Completion/Exit Criteria include:

1️. Test Completion: Tests can be considered finished when the pre-defined conditions for each phase have been met (e.g., all test
cases executed, defects fixed, or an acceptable number of defects remain unresolved).

2️. Test Activity Completion: Criteria like:

▪ Achievement of the required test coverage (e.g., test cases passed, scenarios tested).

▪ Performance metrics met (e.g., response time, resource usage).

▪ No critical defects left unresolved, or a decision is made to defer them.

Best Practices:

• Early Testing: Start testing as early as possible to minimize the last-minute pressure caused by delays in development.

• Pre-defined Entry and Exit Criteria: Having these criteria in place helps set clear expectations and makes the testing process more
structured, reducing the chances of missed testing activities or unfulfilled requirements.

Exit Criteria

• Definition: The exit criteria specify when a test cycle or phase can be concluded, indicating that testing is complete for that stage.

• Purpose: These criteria help determine if testing has achieved its goals, ensuring that all necessary tests have been conducted and the
software meets the desired quality standards.
 • Examples of Exit Criteria:

o All test cases have been executed and passed (or documented as failed with resolution plans).

o Bug fixes from testing are resolved, and no critical issues remain open.

o Code coverage or other test coverage metrics meet the required thresholds.

o The software meets performance or security benchmarks.

o All documentation (including test results and defect reports) is complete.

3. Minimizing Last-Minute Pressure

• Early Testing: Tests should be initiated as early as possible in the development process to identify issues early and minimize the
pressure caused by running tests after development delays.

• By starting testing early, the team can catch defects sooner, reduce the cost of fixing them, and avoid testing delays that could push
back release dates.

dentifying Resource Requirements

When planning for a testing project, the test manager or project manager needs to provide estimates for the hardware and software
resources required. This step ensures that the testing environment is adequately equipped to handle the testing activities.

Factors to Consider:

1️. Machine Configuration:

o RAM, Processor, Disk Space: These are essential hardware resources that the product being tested will need. Adequate machine
specifications ensure smooth execution of the application under test.

2️. Test Automation Tools Overheads:

o Overheads of Automation Tools: Some test automation tools require specific hardware resources, and their overhead (e.g., CPU
and memory usage) should be factored into the resource estimation.

3️. Supporting Tools:

o Compilers, Test Data Generators, Configuration Management Tools: These supporting tools assist in test execution and
management. The resources needed to run these tools should be estimated.

4️. Software Configurations:

o Operating System (OS) Configurations: The different configurations of supporting software (like the operating system) must be
taken into account. Testing on multiple OS configurations might require additional resources.

5️. Special Requirements for Intensive Tests:

o Machine-Intensive Tests (e.g., Load Testing, Performance Testing): Tests like load or performance testing require powerful
machines that can simulate high traffic or heavy processing loads. These special resource requirements should be planned for.

6. Software Licenses:

o Number of Software Licenses: The required number of software licenses (for both test tools and supporting tools) should be
estimated to avoid delays in resource availability.

Why It Matters:

Identifying and allocating the appropriate resources for the testing process ensures that the tests can be executed efficiently, preventing delays
or bottlenecks caused by resource limitations. Proper planning also helps in managing costs and avoiding last-minute scrambling for resources
during the testing phases.

Identifying Test Deliverables

A key part of the testing process is identifying the deliverables that will be produced during the test cycle or testing activity. These deliverables
provide valuable insights into the progress, results, and quality of the software being tested.

Key Test Deliverables:

1. Test Plan

• The test plan serves as the foundation for the entire testing process. It includes:
 o Master Test Plan: The comprehensive plan covering all phases of testing.

o Phase-Specific Test Plans: Plans for individual testing phases, such as functional, integration, or acceptance testing.

2. Test Case Design Specifications

• This document outlines the detailed specifications for creating test cases, including:

o Test case design criteria

o Test conditions and expected results

3. Test Cases

• The test cases themselves are one of the most crucial deliverables. They include:

o Manual test cases: For manual testing efforts.

o Automated test scripts: For automation testing, as specified in the test plan.

• These are essential for validating different features and requirements of the software.

4. Test Logs

• Test logs are created while running tests. These logs record:

o Test execution results

o Any issues or defects encountered during testing

• Test logs serve as evidence for test execution and help in debugging issues.

5. Test Summary Reports

• Test summary reports provide a high-level overview of the testing efforts. These reports include:

o A summary of the test results

o Key issues or defects found

o A conclusion about the quality and readiness of the software

6. Defect Repository

• A defect repository tracks the defects identified during testing. This repository should be continuously updated with:

o New defects: As they are discovered during testing.

o Status updates: After defects are fixed and re-tested, the status of defects should be updated in the repository.

• Keeping this repository current ensures proper defect management and provides insights into the quality of the software.

Summary of Deliverables:

1️. Test Plan: Detailed and structured plans for all phases.

2️. Test Case Design Specifications: Guidelines for creating and executing test cases.

3️. Test Cases: Actual test cases and automation scripts.

4️. Test Logs: Logs that document the results of test executions.

5️. Test Summary Reports: Overviews of test results, key issues, and conclusions.

6. Defect Repository: A continuously updated list of defects, their status, and resolutions.

These deliverables ensure effective tracking of testing progress and quality management throughout the software development lifecycle.

Testing Tasks: Size and Effort Estimation

Estimating the size and effort required for testing is an essential step in the test planning process. It helps in understanding the scope of testing
and allocating resources effectively.

Phases of Estimation:
 1️. Size Estimation: Quantifies the amount of testing needed.

2️. Effort Estimation: Estimates the resources required for the testing.

3️. Schedule Estimation: Provides a timeline for when the testing tasks should be completed.

Size Estimation Factors:

1️. Size of the Product Under Test:

o Lines of Code (LOC): Common measure of software size, though it can be controversial due to factors like programming style.

o Function Points (FP): A method to estimate application size independent of the programming language.

o Screens/Reports/Transactions: A simpler representation of the application's size based on its features.

2️. Extent of Automation:

o Automation increases the size of testing work because:

▪ It requires initial test case design.

▪ Test scripts need to be created using test automation tools.

3️. Number of Platforms/Interoperability:

o If the product needs to be tested on multiple platforms or configurations, the size of testing increases due to the additional test
cases for each environment.

4️. Work Breakdown Structure (WBS):

o The size estimate is broken down into smaller tasks called WBS units, which helps manage the work more effectively.

o Size estimates can be expressed in terms of:

▪ Number of test cases

▪ Number of test scenarios

▪ Number of configurations to be tested

Effort Estimation Factors:

1️. Productivity Data:

o Measures how fast testing activities can be carried out, such as:

▪ Number of test cases developed per day.

▪ Number of test cases executed per day.

▪ Number of pages of documentation tested per day.

2️. Reuse Opportunities:

o If the test architecture supports reuse, effort can be reduced as earlier tests can be reused for new features.

o Reusing test cases decreases the effort required for development.

3️. Robustness of Processes:

o Higher levels of process maturity lead to reduced effort due to:

▪ Well-documented standards for test specifications and scripts.

▪ Proven processes for reviews and audits.

▪ Consistent training and measurement of effectiveness.

4️. Effort Estimation from Size:

o Effort is derived from the size estimate by classifying WBS units as:

▪ Reusable: Parts of previous tests can be reused.

▪ Modifications: Existing tests need slight changes.

▪ New Development: Completely new tests need to be created.

 o Reusable test cases reduce the development effort significantly.

Summary of Key Points:

• Size Estimation: Determines how much testing needs to be done based on factors like the product's size, automation, and platform
requirements.

• Effort Estimation: Considers productivity data, reuse opportunities, and process maturity to calculate the resources needed for testing.

• Work Breakdown Structure: Aids in breaking down the testing into manageable parts to better estimate size and effort.

hoice of Standards in Test Management

Testing standards are essential for ensuring consistency, predictability, and quality in the testing process. There are two types of standards:
External Standards and Internal Standards.

External Standards:

• Definition: Standards that a product must comply with, typically set by external organizations or consortia.

• Examples:

o Standard tests supplied by external consortia.

o Acceptance tests provided by customers.

Internal Standards:

• Definition: Standards formulated within the testing organization to maintain consistency and predictability in processes.

• Purpose: Standardizes the processes and methods used within the organization to ensure uniformity across testing activities.

Examples of Internal Standards:

1️. Naming and Storage Conventions for Test Artifacts:

o Test artifacts (like test specifications, test cases, and results) should be named meaningfully for easy identification and retrieval.

o Example: "PM01️nnnn" where "PM01️" is the product/module, and "nnnn" is a sequence number.

o Helps in organizing and retrieving related tests efficiently, especially when changes occur in product functionality.

2️. Document Standards:

o Specifies the level of detail for documenting manual tests.

o Includes user and system responses at an appropriate level of detail for the tester’s skill level.

o Test scripts should include header comments (outlining the test's function), in-line comments (explaining test parts), and change
history information.

o Ensures that tests are maintainable and changes are documented.

3️. Test Coding Standards:

o Establishes guidelines on how tests should be written, focusing on:

▪ Initialization and clean-up procedures.

▪ Consistent variable naming for clarity.

▪ Encouraging reusability of test artifacts.

▪ Standard interfaces to external entities (like OS and hardware).

o Ensures tests are independent, maintainable, and adaptable to changes, which improves overall test quality.

4️. Test Reporting Standards:

o Defines how test reports are created and shared with stakeholders, ensuring consistency and timely delivery.

o Specifies the level of detail, format, content, and recipients of test reports.
 o Aids in tracking progress, maintaining product quality, and identifying anomalies early.

Benefits of Internal Standards:

• Consistency: Provides a structured approach to testing that maintains uniformity across all phases.

• Predictability: Stakeholders know what to expect in terms of test deliverables and reporting.

• Quality Assurance: Ensures high-quality and maintainable tests, contributing to overall product quality.

• Competitive Edge: Facilitates onboarding of new testers and improves efficiency.

Test Infrastructure Management

Effective test infrastructure is vital to ensure smooth and organized testing throughout a software project. It consists of three essential
elements:

1. Test Case Database (TCDB):

• Purpose: A database that captures all relevant information about the test cases within an organization.

• Contents:

o Test case names.

o Test steps.

o Expected outcomes.

o Test status.

o Any other metadata related to the test cases.

2. Defect Repository:

• Purpose: A repository that stores details of all defects reported during the testing process.

• Contents:

o Defect IDs.

o Defect descriptions.

o Severity/priority levels.

o Assigned team members.

o Status of defect resolution.

3. Software Configuration Management (SCM) Repository:

• Purpose: A repository used for managing change control and version control of all files and entities in a software product.

• Functions:

o Change Control: Ensures that any changes made to test files are approved, tracked, and managed to avoid accidental loss or
overwriting.

o Version Control: Ensures that the test scripts associated with specific product releases are identified, versioned, and available for
future use.

Key Components of SCM:

• Baselining:

o This is akin to taking a snapshot of related files for a particular version, ensuring that the exact set of files associated with a
release can be recreated in the future.
 o Helps to maintain a history of versions, facilitating traceability and reproducibility.

Integrated Workflow of TCDB, Defect Repository, and SCM Repository:

These three elements should complement and integrate with each other for effective test management.

• Defect Repository:

o Links defects, fixes, and tests.

o Contains details of defects that have been reported, along with fixes and related tests.

• SCM Repository:

o Contains all related files for defects, fixes, and test cases.

o Manages the version history of test scripts.

• TCDB:

o Stores metadata about modified test files.

o Helps track which test cases are linked to specific defects and which test scripts correspond to a product release.

Traceability Example:

• Given a defect:

o The defect repository links it to its corresponding fixes and tests.

o The SCM repository stores the test case files associated with the defect.

o The TCDB contains metadata about these test files, helping trace the specific test case(s) involved in testing that defect.

Benefits of Integrated Infrastructure:

• Centralized Management: Having all the testing artifacts, defect details, and configurations in interconnected repositories ensures a
unified view of the testing process.

• Traceability: Ensures that testers can trace defects from their discovery through to resolution, along with the tests involved.

• Reproducibility: Baselining and versioning ensure that tests can be repeated under the exact same conditions, making the testing
process more reliable and consistent.

est Reporting

• Testing requires constant communication between the test team and other teams.

• Test reporting is the means to achieve this communication.

Types of Test Reports

There are two main types of reports:

1️. Test Incident Report

2️. Test Summary Report (also called Test Completion Report)

1. Test Incident Report

• Communicated during the testing cycle whenever defects are found.

• Essentially an entry made into the defect repository.

• Each defect is assigned a unique ID used to track the incident.

• High-impact defects are later highlighted in the Test Summary Report.

2. Test Cycle Report

• A test cycle involves planning and executing tests using different builds of the product.

• The product is expected to stabilize with each cycle.

• A Test Cycle Report includes:

o Summary of test activities.

o Defects found, with their severity and impact.

o Status of previous defects (resolved or pending).

o Outstanding defects yet to be fixed.

o Any variations in effort, time, or schedule.

• Helps in:

o Assessing testing progress.

o Understanding defect trends.

o Planning future cycles.

3. Test Summary Report

• This is prepared at the end of the test cycle to decide if the product is ready for release.

Types of Test Summary Reports:

1️. Phase-wise Test Summary Report

o Generated after each phase of testing.

2️. Final Test Summary Report

o Includes complete testing details across all phases and teams.

o Also called the Release Test Report.

Contents of a Test Summary Report:

1️. Summary of activities carried out during the test phase or cycle.

2️. Variances from the originally planned activities.

3️. Summary of test results (pass/fail, defect stats, etc.).

4️. Final assessment of the product and recommendation for release.