Unit 1
Chapter 1.1Software testing is a process used to identify the correctness, completeness and
quality of developed software, including a set of activities conducted with the intent of finding
errors in software so that it could be corrected before the product is released to the end users.
Testing is necessary because we all make mistakes. Some of those mistakes are unimportant,
but some of them are expensive or dangerous. We need to check everything and anything we
produce because things can always go wrong - humans make mistakes all the time.
A mistake made in coding is known as error.
An Error is a human action that produces incorrect results.
Types of errors
1) Syntax Errors
2) Logical or Semantic Errors
3) Run time errors
Definition of Quality
The degree to which a component, system or process meets specified requirements and/or
user/customer needs and expectations.
When we talk about software quality, we are actually talking about the evaluation of the software
based on certain attributes.
A software quality is defined based on the study of external and internal features of the
software. The external quality is defined based on how software performs in real time scenarios
in operational mode and how useful it is for its users.
The internal on the other hand focuses on the intrinsic aspects that are on the quality of the
code written.
Factors of Quality for user/ Core components of Quality
1. Functionality
2. Reliability
3. Performance
4. Flexibility
5. Usability
6. Security
1. Functionality:
Functionality is the conformity of the software with actual requirements and specifications. An
application should provide all the required functionality as specified in the requirements.
2. Reliability:
The ability of an application or system to consistently perform its intended or required function
on demand and without degradation or failure
3. Performance:
Performance is mostly about response time of the software. This response time should be in
acceptable intervals (e.g. max. a few seconds), and should not increase if transaction count
increases.
4. Flexibility:
It normally refers to the ability of the application or system to adapt to possible or future changes
in its requirements. Flexibility is the ability of software to add/modify/remove functionality without
damaging the current system. Change is inevitable so flexibility becomes an important aspect of
quality.
5. Usability:
Usability is the ease of use and learning ability of an application or system.
6. Security:
The degree to which the application is protected against malicious attacks and other potential
risks. Security is very important especially when the application is dealing with sensitive user
data like bank details.
Quality Management (QM)
Quality management is the act of overseeing all activities and tasks needed to maintain a
desired level of excellence. It ensures that a required level of quality is achieved. The Software
Quality Management comprises Quality Planning, Quality Assurance and Quality Control
Processes.
In the Planning Process Standards are determined that are relevant to the Software Product
and the Organization and the means to achieve them. Once the standards are defined, building
the product starts.
It is very important to have processes to evaluate the project performance and aim to assure
that the Quality standards are being followed and the final product will be in compliance.
Once the software components are built the results are monitored to determine if they comply
with the standards. The data collected helps in measuring the performance trends and as
needed help in identifying defective pieces of code.
Quality Assurance (QA)
Quality Assurance (QA) is the maintenance of a desired level of quality in an application or
software by means of giving attention to every stage of the software development process.
It ensures that the approaches, techniques, methods and processes designed for the projects
are implemented correctly.
Quality assurance activities monitor and verify that the processes used to manage and create
the deliverables have been followed and are operative.
It is a procedure that focuses on providing assurance that quality requested will be achieved. It
is process oriented and mainly focuses on defect prevention.
Quality Control (QC)
Quality Control (QC) ensures that the approaches, techniques, methods and processes
designed in the project are followed correctly.
It's a procedure or set of procedures intended to ensure that an application or system adheres
to a defined set of quality criteria or meets the requirements of the client or customer.
QC activities monitor and verify that the project deliverables meet the defined quality standards.
It is product oriented and focuses on defect identification.
Chapter 1.2 Fundamentals of Testing
Role of testing in software development, maintenance and operations
Human errors can cause a defect or fault to be introduced at any stage within the software
development life cycle, and the consequences can be damaging.
Rigorous testing is necessary during development and maintenance to identify defects, in order
to reduce failures in the operational environment.
This includes looking for places in the user interface where a user might make a mistake in input
of data or in the interpretation of the output, and to look for potential weak points for intentional
and malicious attack.
Executing tests helps us move towards improved quality of product and service, but that is just
one of the verification and validation methods applied to products.
Why is Testing necessary?
Software testing is very important because of the following reasons: 1. Understanding of
customer requirements may differ from person to person. One must challenge the
understanding at each stage of development.
Development people assume that whatever they have developed is as per customer
requirements and will always work. But it is difficult to create real life scenarios and undertake
actual execution of a product at each level of software building.
Different entities are involved in different phases of software development. Their work may not
be matching exactly with each other or with the requirement statements. Gaps between
requirements, design may not be traceable unless testing is performed.
Integration issues may arise when different units do not work together, even though they work
independently. Hence, there must be testing after integration of different units.
There is a possibility of being blindfolded where developers fail to recognize their own faults and
think that code is perfect and there is no chance of improvement. Testers have to challenge
each assumption and decision taken during development.
Following are the principles of testing
Principles of testing:
1. Testing shows presence of defects
2. Exhaustive testing is not possible
3. Early Testing
4. Defect clustering
5. Pesticide Paradox
6. Testing is context dependent
7. Absence of errors fallacy
1. Testing shows presence of defects: Testing shows that there are defects present but it
does not imply that there are no [Link] reduces the probability of undiscovered defects
remaining in the software but, even if no defects are found, it is not a proof of correctness.
2. Exhaustive testing is not possible: Testing everything i.e. all the combinations of inputs is
not possible. Testing the product should be accomplished considering the risk factor and
priorities.
3. Early Testing: Early testing helps identify issues prior to the development stage, which eases
error correction and helps reduce cost
4. Defect clustering: Normally a defect is clustered around a set of modules or functionalities.
Once they are identified, testing can be focused on the defective areas, and yet continue to find
defects in other modules simultaneously.
[Link] Paradox: If the same tests are repeated over and over again then eventually the
test will no longer find any defects. To overcome this the tests should be regularly revised.
To overcome this 'pesticide paradox', the test cases need to be regularly reviewed and revised,
and new and different tests need to be written to exercise different parts of the software or
system to potentially find more defects.
[Link] is context dependent: Testing is done differently depending on the type of software.
For e.g. software which deals with money should be tested keeping security in mind.
[Link] of errors fallacy: Just identifying and fixing issues does not really help in setting
user expectations. Even if testing is performed to showcase the software's reliability, it is better
to assume that none of the software products are bug-free.
Fundamental Test Process/ Testing Life Cycle
The Fundamental of Test Process consists of
1. Test Planning and Control
2. Test Analysis and Design
3. Test implementation and execution
4. Evaluating exit criteria and Reporting
5. Test closure activities
1. Test Planning and Control:
Test Planning is a Fundamental Test Process which defines the objective and goal of the testing
process. Using the requirement specification test plan will make it more detailed. It is a
continuous process and performed in all life cycles.
● Deciding the scope and risk of testing
● Deciding the overall approach of testing
● Scheduling test analysis and
● Design process Assigning resources to different activities
2. Test Analysis and Design :
Test Analysis and Design is a Fundamental Test Process which creates test conditions & test
cases. In this process, we performed major tasks like reviewing the test basis, identifying the
test conditions based on analysis of test items, writing test cases, identifying necessary test
data to support the test conditions and test cases, Designing the test environment.
3. Test Implementation and Execution :
It is a Fundamental Test Process in which actual work is done. In this process executing test
cases with test data
Creating test suites from test executions
Checking test environments, updating traceability between test basis and test cases
Executing test procedures using test execution tools
Checking actual result with expected results
Reporting errors & creating Incident reports
4. Evaluating exit criteria and Reporting:
Evaluating exit criteria is a process defining when to stop testing. It depends on the coverage of
code, functionality or risk. Basically, it also depends on business risk, cost and time.
5. Test Closure Activities:
Test closure activities are the last process in the fundamental test process. This process collects
data from completed test processes and test wares.
Ensure that deliverable has been delivered or not.
Closing incident report
Documenting all the systems
Archiving all the testware, test environment and infrastructure
THE PSYCHOLOGY OF TESTING
The mindset we want to use while testing and reviewing is different from the one we use while
analyzing or developing. By this we mean that, if we are building something we are working
positively to solve problems in the design and to realize a product that meets some need.
However, when we test or review a product, we are looking for defects in the product and thus
are critical of it.
Several stages of reviews and testing are carried out throughout the life cycle and these may be
independent reviews and tests. Early in the life cycle, reviews of requirements and design
documents by someone other than the author helps find defects before coding starts and helps
us build the right software. Following coding, the software can be tested independently. This
degree of independence avoids author bias and is often more effective at finding defects and
failures. Several levels of independence can be identified, listed here from the lowest level of
independence to the highest:
• tests by the person who wrote the item under test;
• tests by another person within the same team, such as another programmer;
• tests by a person from a different organizational group, such as an independent test team;
• tests designed by a person from a different-organization or company, such as outsourced
testing or certification by an external body.
PSYCHOLOGY OF TESTING
• Communicate findings on the product in a neutral, fact-focused way without criticizing the
person who created it. For example, write objective and factual incident reports and review
findings.
- Don't gloat - you are not perfect either!
- Don't blame - any mistakes are probably by the group rather than an individual.
- Be constructively critical and discuss the defect and how you are going to log it.
• Explain that by knowing about this now we can work round it or fix it so the delivered system is
better for the customer.
- Say what you liked and what worked, as well as what didn't work.
- Show what the risk is honestly - not everything is a high priority.
- Don't just see the pessimistic side - give praise as well as criticism.
- Show what risks have been uncovered and the benefits of the review or test.
• Start with collaboration rather than battles. Remind everyone of the common goal of better
quality systems.
- Be polite and helpful, collaborate with your colleagues.
- Try to understand how the other person feels and why they react as they do.
- Confirm that the other person has understood what you have said and vice versa.
- Explain how the test or review helps the author - what's in it for him or her.
- Offer your work to be reviewed, too.