SYSTEM AND ACCEPTANCE
TESTING
System Testing – Overview
It is defined as a testing phase conducted on the complete integrated system to
evaluate the system’s compliance with the specified requirements.
It is done after unit, component, and integration testing phases.
A system is a complete set of integrated components that together deliver product
functionality and features.
A system can also be defined as a set of hardware, software, and other parts
that together provide product features and solutions.
System testing helps in uncovering defects that may not be directly attributable
to a single module or interface.
System testing brings out issues that are fundamental to the design,
architecture, and code of the whole product.
It is the only phase of testing which tests both functional and non-functional
aspects of the system.
On the functional side, system testing focuses on real-life customer usage of
the product and solutions.
On the non-functional side, system testing includes different testing
types (also called quality factors), some of which are as follows:
1. Performance / Load Testing
To evaluate the time taken or response time of the system to
perform the required functions in comparison with different
versions of the same product or different competitive products is
called Performance Testing.
2. Scalability Testing
A testing process that requires an enormous amount of resources to
find out the maximum capability of the system parameters is
called Scalability Testing.
3. Reliability Testing
To evaluate the ability of the system or an independent component
of the system to perform its required functions repeatedly for a
specified period of time is called Reliability Testing.
4. Stress Testing
Evaluating a system beyond the limits of the specified requirements
or system resources (such as disk space, memory, processor
utilization) to ensure the system does not break down unexpectedly
is called Stress Testing.
5. Interoperability Testing
This testing is done to ensure that two or more products can
exchange information, use the information, and work closely
together.
6. Localization Testing
Testing conducted to verify that the localized product works
correctly in different languages is called Localization Testing.
Why is System Testing Done?
• An independent test team normally performs system testing. This
independent test team is different from the team that performs
component testing and integration testing.
• Testing the product with an independent perspective, and
combining that with the customer’s perspective, makes system
testing unique, different, and effective.
• System testing by an independent team removes the bias that may
occur when developers test their own modules or integrated
components.
• System testing removes the bias on the product. Introducing a
“fresh pair of eyes” through an independent system test team
helps in locating defects missed during component and
integration testing.
• The behavior of the complete product is verified during system testing.
• Tests that refer to multiple modules, programs, and functionalities are included
in system testing.
• System testing helps in identifying as many defects as possible before the
customer finds them. This is the last chance for the test team to find remaining
product defects before the product is handed over to the customer.
• System testing tries to achieve a balance between: The objective of finding
defects, and The objective of building confidence in the product before release.
• An impact analysis is done for defects to reduce the risk of releasing a product
with defects.
• If the risk of customers encountering the defect is high, then the defects are
fixed before release.
• Otherwise, the product may be released as it is.
• The analysis of defects and their classification into various categories gives an
idea about the kinds of defects that customers may find after the release.
• This information helps in planning activities such as:
✓ Providing workarounds
✓ Preparing documentation
To summarize, system testing is done for the following reasons:
1. Provide an independent perspective in testing.
2. Bring in the customer perspective in testing.
3. Provide a fresh pair of eyes to discover defects not found earlier.
4. Test product behavior in a holistic, complete, and realistic
environment.
5. Test both functional and non-functional aspects of the product.
6. Build confidence in the product.
7. Analyze and reduce risk before releasing the product.
8. Ensure all requirements are met and the product is ready for
acceptance testing.
FUNCTIONAL VERSUS NON-FUNCTIONAL TESTING
Functional Testing :
• Functional testing involves testing a product’s functionality and
features.
• It focuses on verifying what the system is supposed to do. It
checks the product’s features and functional behavior.
• Functional testing should have very clear expected results
documented in terms of the correct behavior of the product.
• Functional testing comprises simple methods and steps to execute
the test cases.
• Functional testing results normally depend on the product and
environment. It uses a pre-determined set of resources and
configurations, except for a few types of testing.
• This testing requires in-depth customer and domain knowledge to
develop effective test cases and find critical defects because the
focus is on identifying defects.
• Failures in functional testing usually result from errors in the
code, leading to incorrect behavior.
• Functional testing is performed in all phases of testing such as:
✓ Unit Testing
✓ Component Testing
✓ Integration Testing
✓ System Testing
Functional testing in system testing (Functional System Testing)
focuses on complete product features, unlike earlier phases which
focus on component features and interface features.
Non-Functional Testing
• Non-functional testing is performed to verify the quality factors
of the system (such as reliability, scalability, performance,
etc.).These quality factors are also called non-functional
requirements.
• Non-functional testing requires the expected results to be
documented in qualitative and quantifiable terms.
• Non-functional testing requires a large amount of resources, and
the results differ for different configurations and resource setups.
• This type of testing is complex because it involves a large amount
of data to be collected, monitored, and analyzed.
• Test cases for non-functional testing include a clear pass/fail
criteria.
• Test results are evaluated based on:
• Pass/Fail definitions
• Experiences and observations made during test execution