STM Unit 1 (R23)
STM Unit 1 (R23)
Introduction: Purpose of testing, Dichotomies, model for testing, consequences of bugs, taxonomy of
bugs.
Flow graphs and Path testing: Basics concepts of path testing, predicates, path predicates and achievable
paths, path sensitizing, path instrumentation, application of path testing.
Testing is the process of evaluating a system or system components by manual or automated means to verify
that it satisfies specified requirements.
It usually takes a lot of time and effort during software development, and this effort is wasted if the tests do
not find the existing errors. Even after testing carefully, some mistakes can still appear, because no code is
perfect. The best way to reduce these errors is to create good test plans and test as many parts of the software
as possible.
Testing takes at least half of the time and work required to produce a software program.
• MYTH: Good programmers write code without any bugs, but this is not true.
• History says that even well written programs still have 1-3 bugs per hundred statements.
The main goal of testing is to make sure software works correctly by preventing bugs and finding problems
early. Good testing not only detects errors but also provides clear information that helps developers fix them
quickly. Preventing bugs is always better than fixing them later because it saves time, avoids retesting,
reduces effort, and keeps the project on schedule. By designing good tests and thinking carefully during
test creation, issues can be caught at any stage of development, improving the overall quality of the
software.
Phases in a tester's mental life can be categorized into the following 5 phases:
1. Phase 0: Debugging Oriented (Before 1956)
In this phase, people did not see any difference between testing and debugging. Both activities were treated
as the same checking the program and fixing errors. Testing was not recognized as a separate process in
software development during these early years.
2. Phase 1: Demonstration Oriented (1957–1978)
During this time, testing was used mainly to prove that the software works correctly. However, this approach
failed because the more you test a program, the more bugs you are likely to find. So, testing could never
truly prove that the software was perfect.
3. Phase 2: Destruction Oriented (1979–1982)
Here, the goal of testing was the opposite to show that the software does not work. This also failed because
testers would keep finding one bug after another, and fixing one bug often created new ones. This meant
the product would never be ready for release.
4. Phase 3: Evaluation Oriented (1983–1987)
In this phase, testing focused on reducing the risk of software failure to an acceptable level. The idea was
not to prove anything but to improve the product by finding bugs and fixing them. The software was
released when developers felt confident that the remaining risk was very low.
5. Phase 4: Prevention Oriented (1988–2000)
This phase highlighted preventing bugs before they occur. The focus shifted to writing code that is easy to
test, because testable code usually has fewer bugs. The main goal was to identify the best testing techniques
and make testing easier and more effective.
Test Design is the process of creating tests to check whether software works correctly. Good tests should
be designed carefully and verified before using them on the real software.
There are many other methods, apart from testing, that help make software better.
• Inspection Methods: Methods like walkthroughs, reading code, and reviews help find errors.
They can catch bugs that normal testing might miss.
• Design Style: If the software is designed in a clear and simple way, many bugs can be avoided
from the beginning.
• Static Analysis Methods: These methods check the code without running it. Today, compilers
automatically do many of these checks.
• Languages: Some programming languages reduce certain types of mistakes. Programmers find new
bugs while using new languages.
• Development Methodologies & Environment: Good development processes and tools help
prevent bugs early and improve the overall quality of the software.
1.3 DICHOTOMIES
Testing and debugging are often confused, but they have different purposes. The purpose of testing is to
check the program and show that there are errors in it. Debugging, on the other hand, comes after testing
and focuses on finding the exact cause of those errors and fixing them. In simple terms, testing finds the
problems, and debugging solves them.
Testing Debugging
Testing starts with known conditions and follows Debugging starts with unknown conditions, and
fixed steps with expected results. the final outcome is uncertain.
Testing can be planned, designed, and scheduled. Debugging cannot be strictly planned.
Testing shows that an error exists or that the software
Debugging is the process of finding the exact
seems correct. cause of the error.
Testing exposes a programmer’s mistakes. Debugging helps the programmer explain and fix
those mistakes.
Testing can be done without deep design knowledge. Debugging needs strong understanding of the
software design.
Testing can be done by an outsider (tester). Debugging must be done by an insider
(developer).
Many testing tasks can be automated. Automated debugging is not fully possible yet.
In functional testing, the program is treated like a black box. We give inputs and check if the outputs are
correct. We do not look at how the program is built. This is from the user’s point of view, focusing only on
what the software does. In structural testing, checks the inside of the program, including the code, logic,
and design. Both functional and structural tests are important, and each has limitations.
Functional testing could find all bugs but would take too much time. Structural testing is limited and faster
but cannot find every error.
✓ Modularity: Focuses on designing software in separate, independent modules. It makes code easier
to understand, maintain, and test.
✓ Efficiency: Focuses on making software fast and resource-friendly, even if the design becomes
more complex.
Small programs or modules that are easier to understand, test, and debug. Large programs or systems which
are more complex, harder to test thoroughly. Big systems are more complex than small ones.
Most software is made and used by the same organization, but this is not good. If the builder and the user
are the same, it becomes hard to know who is responsible when something goes wrong. If there is no
separation between builder and buyer, there can be no accountability.
All the systems need to be tested. The typical system is that which allows exploration of all testing aspects
without any complications. Given below in figure 1.1 is a model of testing:
i) Environment: A program’s environment includes the hardware and the software required for making it
run. It also includes all the programs interacting with it and used to create this program under test e.g.
operating system, loader, linker, compiler, etc. A testing environment has a test plan, which lists the tests to
be done, and test cases, which explain how each part of the program will be tested. Creation of a test
environment reduces the risk of errors occurring when the software is used.
ii) Program: Many programs are too complex to understand, so we simplify them for testing by ignoring
some details. If the simplified model doesn’t explain unexpected behavior, we can add more details to the
model. And if that fails, we may have to modify the program.
iii) Bugs: A bug is an error in software that causes it to behave unexpectedly or incorrectly. Bugs can occur
in code, design, requirements, or even in how the software interacts with hardware or users. These are listed
below:
• Benign Bug Hypothesis: Only simple bugs are predictable and easy to find; subtle bugs are
unpredictable and have no clear pattern.
• Bug Locality Hypothesis: A bug in one module can affect other modules, it is not always true.
• Control Bug Dominance: Believing most bugs are in control structures like if or switch.
• Code / Data Separation: Bugs do not always respect the boundary between code and data.
• Lingua Salvatore Est.: Programming language does not automatically eliminate bugs.
• Corrections Abide: Fixing a bug once does not guarantee it won’t appear again.
• Silver Bullets: There is no single tool, method, or language that can prevent all bugs.
• Sadism effect: Bugs cannot be found by intuition or cleverness alone; systematic testing methods
are needed.
iv) Tests: Tests follow a formal process. we prepare inputs, predict the outcomes, tests should be
documented, commands need to be executed, and results are to be observed. But mistakes can happen at
any of these steps.
v) Testing Levels: Software is tested at different levels to ensure each part works correctly, both
individually and together. Each type has a different purpose.
• Unit / Component Testing: A unit is the smallest part of a program that can be tested. It is usually
written by a programmer and has only a few lines of code. Unit testing checks whether this small
part meets its requirements and follows the intended design. Any problems found here are called
unit bugs.
• Component Testing: A component is made by combining one or more units. Even a single unit
can be considered a component, depending on the design. Component testing checks whether this
larger combined piece works properly. Bugs found here are called component bugs.
v) Role of Models:
Testing involves creating, choosing, checking, and updating different models. We can do this well only if
we have many useful models that explain how the program works.
i) Importance of bugs:
The importance of bugs depends on frequency, correction cost, installation cost, and consequences.
1. Frequency: It is important to understand how often a bug appears and which bugs occur the most
frequently.
2. Correction Cost: It is important to know how much it will cost to fix a bug once it is found. This
cost includes finding the bug and correcting it. The cost becomes much higher if the bug is
discovered late in the software process, and fixing bugs in large programs is even more expensive.
3. Installation Cost: Installation cost depends on the number of installations. It is low for one user
but much higher for large distributed systems.
4. Consequences: The impact of bugs can be measured by the average compensation paid to the
people affected by them.
Quality of a program can be measured from the combination of number of bugs and their severity. Bugs
and their effects play a significant role. During testing, software quality improves from almost zero to a
reliable for release.
There is no perfect way to categorize bugs. Different people may categorize the same bug in different ways
because it depends on the background of the bug and how the programmer understands it. So, one bug can
belong to different categories in different situations. The major categories are: (1) Requirements, Features
and Functionality Bugs (2) Structural Bugs (3) Data Bugs (4) Coding Bugs (5) Interface, Integration and
System Bugs (6) Test and Test Design Bugs.
The above program, the control bug happens because the code tries to go to “label 1” in the else
part, but that label does not exist. So, when the else condition is true, the program does not know
where to go next, causing an error.
• Logic Bugs: Logic bugs occur when logical statements or operators are used incorrectly. This can
include missing cases, wrong order of cases, incorrect simplification, or overlapping cases. If these
mistakes only affect logic, they are called processing bugs.
Example:
scanf("%d", &money);
if (money_in_store == 0)
{
printf("%d money on exit\n", money_in_store);
exit(0);
}
This is an example of logical bug. After ‘scanf’, we are checking for ‘money_in_store’ instead of ‘money’.
• Processing Bugs: Processing bugs include arithmetic, algebraic, mathematical function evaluation,
algorithm selection, incorrect conversion from one data type to another and general processing.
Other common problems are ignoring overflow, not handling positive and negative zero correctly,
and using comparison operators like greater than (>) or less than (<) wrongly. For example, dividing
a number by zero (b = 10 / 0) causes a serious processing error because such a calculation is not
allowed.
• Initialization Bugs: Initialization bugs occur when variables are not properly prepared before use.
Some unnecessary initializations may slow down the program. Common mistakes include forgetting
to initialize a variable, assuming it already has a value, using the wrong type, or using an incorrect
format.
• Third Law - Code migrates to data: This law highlights that bugs in the code. It also shows that
code bugs are only part of the problem, and data-related bugs are equally important.
• Information, parameter, and control: Static or dynamic data can be used in three ways, or a
combination of them: as a parameter, to control the program, or to store information.
Coding bugs can lead to other bugs. Syntax errors are usually detected by the compiler, including
undeclared variables, missing routines, or unused code. These errors must be fixed before testing, because
if there are many syntax errors, the program may also have many logic and coding bugs.
i) Motivation and Assumption/ Path Testing: Path testing focuses on selecting specific paths through a
program to check its behavior. By choosing the right paths, every part of the code can be executed at least
once. It is an old structural testing method, mainly used for unit testing by programmers. Path testing
requires complete knowledge of the program’s structure, but its effectiveness decreases as the program size
increases.
ii) Control Flow Graphs: The control flow-graph (or flow-graph) is a graphical representation of a
program’s control structure. It is similar to the flowchart. It uses the elements as shown is Figure 4.1: process
blocks, decisions and junctions.
iv) Loops: There are three kinds of loops: nested, concatenated and horrible loops.
• Nested Loops: Nested Loops are loops placed inside another loop, so that the inner loop executes
completely for each iteration of the outer loop.
• Horrible Loops: Horrible loops are un-structured loops that include jumps into or out of loops
using statements like GOTO, hidden loops, or cross-linked loops. Horrible Loops are loops that are
difficult to test because of their complex structure or behavior.
v) Variations: Variations refer to different methods used in path-based testing to handle complexity and
improve effectiveness. They mainly fall into two classes:
1. Strategies between C2 (branch coverage) and full path testing.
2. Strategies weaker than C1 (statement coverage) or C2.
i) Predicates: A predicate is a Boolean condition (true or false) used in a decision statement. The value of
this condition decides which path the program follows. This expression is called a predicate. Each program
path is defined by a sequence of true/false results from these predicates. The predicate that represents a
particular path is called a path predicate.
ii) Predicate Expressions: A predicate expression is a condition that has been simplified by rewriting all
earlier operations in terms of the original input variables. The main purpose of a predicate expression is to
find exactly which input values will make the condition on a chosen path evaluate to true or false.
Example: function example(x):
y=x+1
z=y*2
iv) Testing Blindness: Testing Blindness occurs when a test follows the correct program path, but not for
the correct reason. This means a bug exists, but the test does not detect it. There are three types of Testing
Blindness:
• Assignment Blindness: Assignment Blindness occur when a wrong assignment does not show a
bug because the chosen test value works for both the correct and incorrect code.
For Example:
Correct code: Buggy code:
x = y + 1; x = y;
if (x > 10) if (x > 10)
If we test with y = 11, both versions make x > 10 true, so the program seems correct.
But with y = 10, the correct code gives x = 11 (true) and the buggy code gives x = 10 (false),
revealing the bug.
• Equality Blindness: Equality Blindness happens when a bug is missed because a value set earlier
satisfies both the correct and the wrong condition, so the program follows the same path and the
error is hidden.
For Example:
Correct code: Buggy code:
if (x == 10) if (x >= 10)
If a previous step sets x = 10, both conditions evaluate to true, so the program follows the same
path and the bug is not detected. But if x = 12, the correct condition becomes false while the buggy
one is true, exposing the error.
Both conditions mean the same thing, so the program follows the same path every time.
Path Sensitizing is the process of finding input values that will make a program follow a specific path
during testing. It confirms that the selected path is actually achievable and helps in designing effective test
cases.
Software testing instrumentation involves adding code or tools to a program to monitor its behavior,
performance, and code coverage during execution. The main types of instrumentation methods focus on
tracking execution paths and gathering data. The types of instrumentation methods include:
2. Traversal Marker or Link Marker: A simple and effective instrumentation is called a traversal marker
or link marker. Each link in the program is assigned a letter, which is recorded when the link is executed.
The sequence of letters from start to finish should match the expected path if there are no bugs. However,
a single link marker may not be sufficient because links can be affected due to bugs.
Why single link markers aren’t enough: A single link marker may fail because bugs can alter the links,
preventing accurate tracking of the path.
We planned to follow the ikm path, but a faulty GOTO in the m link takes us to process B instead. If
coincidental correctness occurs, the results may look correct, hiding the bug.
3. Two Link Marker Method: The solution to the single link marker problem is to use two markers per
link: one at the start and one at the end. This way, the markers specify the path and confirm that each link
was executed from beginning to end.