Software Testing and Quality Assurance

Theory and Practice

Chapter 3
Unit Testing

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 1
 Outline of the Chapter
• Concept of Unit Testing
• Static Unit Testing
• Defect Prevention
• Dynamic Unit Testing
• Mutation Testing
• Debugging
• Unit Testing in eXtreme Programming
• Tools For Unit Testing

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 2
• A program unit is tested in isolation, that is, in a stand-alone
manner.
• There are two reasons for testing a unit in a stand-alone manner.
– First, errors found during testing can be attributed to a specific unit so that
it can be easily fixed.
– Moreover, unit testing removes dependencies on other program units.
• Not everything pertinent to a unit can be tested in isolation
because of the limitations of testing in isolation.
• This means that some errors in a program unit can only be found
later, when the unit is integrated with other units in the integration
testing and system testing phases.
• Unit testing is performed by the programmer who writes the
program unit because the programmer is intimately familiar with
the internal details of the unit

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 3
 Concept of Unit Testing
• Static Unit Testing
– Code is examined over all possible behaviors that might arise during run
time
– Code of each unit is validated against requirements of the unit by reviewing
the code
– For example, in the C programming language the two program-halting
instructions are abort() and exit().
– While the two are closely related, they have different effects as explained
below:
• Abort(): This means abnormal program termination. By default, a call to
abort() results in a run time diagnostic and program self-destruction.
• The program destruction may or may not flush and close opened files or
remove temporary files, depending on the implementation.
• Exit(): This means graceful program termination. That is, the exit() call
closes the opened files and returns a status code to the execution
environment.

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 4
 Static Unit Testing
• In static unit testing code is reviewed by applying techniques:
– Inspection
– Walkthrough
• The idea here is to examine source code in detail in a systematic
manner
• The objective of code review is to review the code, and not to
evaluate the author of the code
• Code review must be planned and managed in a professional manner
• The key to the success of code is to divide and conquer
– An examiner inspect small parts of the unit in isolation
• nothing is overlooked
• the correctness of all examined parts of the module implies the
correctness of the whole module

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 5
 Static Unit Testing (Code Review)
• Step 1: Readiness
– Criteria
• Completeness
• Minimal functionality
• Readability
• Complexity
• Requirements and design documents
– Roles
• Moderator(review meeting is chaired by the moderator)
• Author
• Presenter(The presenter reads thecode beforehand to
understand it, other than author)
• Record keeper
• Reviewers
• Observer(intern)
• Step 2: Preparation
– List of questions
– Potential Change Request (CR)
– Suggested improvement opportunities
Figure 3.1: Steps in the code review process
Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 6
• Completeness
– All the code relating to the unit to be reviewed must be available.
• Minimal Functionality
– The code must compile and link, the code must have been tested to some
extent to make sure that it performs its basic functionalities.
• Readability
• Complexity
– The code to be reviewed must be of sufficient complexity to warrant group
review.
– Here, complexity is a composite term referring to the number of conditional
statements in the code, the number of input data elements of the unit, the
number of output data elements produced by the unit, real-time processing of
the code, and the number of other units with which the code communicates.
• Requirements and Design Documents

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 7
 Static Unit Testing (Code Review)
• Step 3: Examination A Change Request (CR) includes the
– The author makes a presentation following details:
– The presenter reads the code – Give a brief description of the issue
– The record keeper documents the CR – Assign a priority level (major or
minor) to a CR
– Moderator ensures the review is on
track – Assign a person to follow it up
• Step 4: Re-work – Set a deadline for addressing a CR
– Make the list of all the CRs
– Make a list of improvements
– Record the minutes meeting
– Author works on the CRs to fix the
issue
• Step 5: Validation
– CRs are independently validated
• Step 6: Exit
– A summary report of the meeting
minutes is distributes

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 8
 Static Unit Testing (Code Review)

The following metrics can be collected from a code review:

• It is important to collect measurement data pertinent toa review
process, so that the review process can be evaluated, made visible to
the upper management
– The number of lines of code (LOC) reviewed per hour
– The number of CRs generated per thousand lines of code (KLOC)
– The number of CRs generated per hour
– The total number of hours spend on code review process

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 9
 Defect Prevention
• Use standard control to detect possible occurrences of error
conditions
• Ensure that code exists for all return values
• Ensure that counter data fields and buffer overflow/underflow are
appropriately handled
• Provide error messages and help texts from a common source
• Validate input data
• Use assertions to detect impossible conditions
• Leave assertions in the code.
• Fully document the assertions that appears to be unclear
• After every major computation reverse-compute the input(s) from
the results in the code itself
• Include a loop counter within each loop

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 10
 Dynamic Unit Testing
• A program unit is actually executed and its outcomes are observed
• One observe some representative program behavior, and reach
conclusion about the quality of the system
• Static unit testing is not an alternative to dynamic unit testing
• Static and Dynamic analysis are complementary in nature
• In practice, partial dynamic unit testing is performed concurrently
with static unit testing
• It is recommended that static unit testing be performed prior to the
dynamic unit testing

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 11
 Dynamic Unit Testing
• Unit Under Test (UUT): This is the part of the program you’re testing
• The caller unit is known as test driver
– A test driver is a program that invokes the unit under test (UUT)
– It provides input data to unit under test and report the test result
• The emulation of the units called by the UUT are called stubs
– It is a dummy program
– Imagine the UUT needs to interact with other parts of the program (like getting information from a database or making
calculations). These other parts are called dependencies.
– A stub is a stand-in for these dependencies. It pretends to be the real thing without actually being fully functional.
– Stubs provide simplified responses or actions that the UUT expects from its dependencies during testing. They help
create a controlled environment for testing the UUT alone.
• The test driver and the stubs are together called scaffolding

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 12
Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 13
 Dynamic Unit Testing
Selection of test data is broadly based on the following techniques:
• Control flow testing
– Draw a control flow graph (CFG) from a program unit
– Select a few control flow testing criteria
– Identify a path in the CFG to satisfy the selection criteria
– Derive the path predicate expression from the selection paths
– By solving the path predicate expression for a path, one can generate the data
• Data flow testing
– Draw a data flow graph (DFG) from a program unit and then follow the
procedure described in control flow testing.
• Functional program testing
– Input/output domains are defined to compute the input values that will cause
the unit to produce expected output values

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 14
 Mutation Testing
• Modify a program by introducing a single small change to the code
• A modified program is called mutant
• A mutant is considered "killed" if the test case detects a difference in
output compared to the original program
• A mutant is an equivalent to the given program if it always produce
the same output as the original program. On the other hand, mutants
that produce different outputs are non-equivalent mutants.
• A mutation score for a set of test cases is the percentage of non-
equivalent mutants killed by the test suite
• The test suite is said to be mutation-adequate if its mutation score is
100%

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 15
• Killable Mutant: This is a mutated version of the program that can be
"killed" by the test suite if the right test case is added. In other words,
the mutant is not inherently undetectable; it just requires a specific test
case to reveal its difference from the original program.

• Stubborn Mutant: it is called "stubborn" because the existing set of

test cases is insufficient to detect the difference and thus "kill" the
mutant. Stubborn mutants expose gaps in the test suite's coverage,
indicating areas where additional test cases or improvements in testing
strategies are needed.

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 16
 Consider the following code

• #include <stdio.h>
• #include <stdlib.h>

• int main() {
• int x = 10;
• int y = 20;
•
• if (x > y) {
• printf("x is greater than y\n");
• } else {
• printf("y is greater than or equal to x\n");
• }
• return 0;
• }

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 17
 Mutation testing

Mutant 1.1: if (x >= y) (Replace > with >=)

Mutant 1.2: if (x != y) (Replace > with !=)

Mutant 2.1: if (x < y) (Replace > with <)

Mutant 2.2: if (x == y) (Replace > with ==)

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 18
Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 19
 Mutation testing
• Add a test case, because of which the mutant can be “killable”

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 20
 Mutation Testing
Mutation testing makes two major assumptions:

• Competent Programmer hypothesis

– Programmers are generally competent and they do not create random programs

• Coupling effects
– Complex faults are coupled to simple faults in such a way that a test suite
detecting simple faults in a program will detect most of the complex faults

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 21
 Debugging
• The process of determining the cause of a failure is known as
debugging
• It is a time consuming and error-prone process
• Debugging involves a combination of systematic evaluation,
intuition and a little bit of luck
• The purpose is to isolate and determine its specific cause, given a
symptom of a problem
• There are three approaches to debugging
– Brute force
– Cause elimination
• Induction
• Deduction
– Backtracking

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 22
• Induction
– all pertinent data related to the failure are collected
– the collected data are organized
– in terms of behavior and symptoms
– cause hypothesis is devised
• Deduction
– list of all possible causes is developed in order of their likelihoods,
– and tests are conducted to eliminate or substantiate each cause in
decreasing order of their likelihoods
• Backtracking
– starts at a point in the code where a failure was observed and traces
back the execution to the point where it occurred.

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 23
 JUnit – A Framework for Unit Testing
• JUnit provides a basic class called TestCase.
• The tester
– Extends the TestCase class for each test case. 10 extensions for 10 test
cases.
– Alternatively, extend TestCase to have 10 methods for 10 test cases.
• The TestCase class provides methods to make assertions.
– assertTrue(Boolean condition)
– assertFalse(Boolean condition)
– assertEquals(Object expected, Object actual)
– assertEquals(int expected, int actual)
– assertEquals(double expected, double actual, double tolerance)
– assertSame(Object expected, Object actual)
– assertNull(Object testobject)
– …
• The tester can have her own assertions.
Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 24
 JUnit – A Framework for Unit Testing
• Each assertion accepts an optional first parameter of type String; if
the assertion fails, the string is displayed.  Help for the tester…
• The assertEquals() method displays a message upon failure.
• [Link]: expected: <x> but
was: <y>
• Note that only failed tests are reported.
• The following shows how assertTrue() works.

static public void assertTrue(Boolean condition) {

if (!condition)
throw new AssertionFailedError();
}

Figure 3.5: The assertTrue() assertion throws an exception

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 25
 JUnit – A Framework for Unit Testing
import TestMe; // TestMe is the class whose methods are going to be tested.
import [Link].*; // This contains the TestCase class.

public class MyTestSuite extends TestCase { // Create a subclass of TestCase

public void MyTest1() { // This method is the first test case

TestMe object1 = new TestMe( ... ); // Create an instance of TestMe with
desired params
int x = object1.Method1(...); // invoke Method1 on object1
assertEquals(365, x); // 365 and x are expected and actual values,
respectively.
}

public void MyTest2() { // This method is the second test case

TestMe object2 = new TestMe( ... ); // Create another instance of
// TestMe with desired parameters
double y = object2.Method2(...); // invoke Method2 on object2
assertEquals(2.99, y, 0.0001d); // 2.99 is the expected value;
// y is the actual value;
// 0.0001 is tolerance level
}
}

Figure 3.5: An example test suite

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 26
 Tools For Unit Testing
• Code auditor
– This tool is used to check the quality of the software to ensure that it meets
some minimum coding standard
• Bound checker
– This tool can check for accidental writes into the instruction areas of memory,
or to other memory location outside the data storage area of the application
• Documenters
– These tools read the source code and automatically generate descriptions and
caller/callee tree diagram or data model from the source code
• Interactive debuggers
– These tools assist software developers in implementing different debugging
techniques
Examples: Breakpoint and Omniscient debuggers
• In-circuit emulators
– It provides a high-speed Ethernet connection between a host debugger and a
target microprocessor, enabling developers to perform source-level debugging
Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 27
 Tools for Unit Testing
• Memory leak detectors
– These tools test the allocation of memory to an application which request for memory
and fail to de-allocate memory
• Static code (path) analyzer
– These tool identify paths to test based on the structure of code such as McCabe’s
cyclomatic complexity measure

Table 3.3: McCabe complexity

measure

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 28
 Tools for Unit Testing
• Software inspection support
– Tools can help schedule group inspection
• Test coverage analyzer
– These tools measure internal test coverage, often expressed in terms of control
structure of the test object, and report the coverage metric
• Test data generator
– These tools assist programmers in selecting test data that cause program to
behave in a desired manner
• Test harness
– This class of tools support the execution of dynamic unit tests
• Performance monitors
– The timing characteristics of the software components be monitored and
evaluate by these tools
• Network analyzers
– These tools have the ability to analyze the traffic and identify problem areas

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 29
 Tools for Unit Testing
• Simulators and emulators
– These tools are used to replace the real software and hardware that are not
currently available. Both the kinds of tools are used for training, safety, and
economy purpose
• Traffic generators
– These produces streams of transactions or data packets.
• Version control
– A version control system provides functionalities to store a sequence of
revisions of the software and associated information files under development

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 30