Software Testing and Quality Assurance

Theory and Practice

Chapter 3
Unit Testing

UNIT 2

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 1
 Outline of the Chapter
• Concept of Unit Testing
• Static Unit Testing
• Defect Prevention
• Dynamic Unit Testing
• Mutation Testing
• Debugging
• JUnit – A Framework for Unit Testing

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 2
 Concept of Unit Testing
• First level of testing.
• Unit testing refers to testing program units in isolation. Some examples of units
are functions, procedures, or methods.
• Can invoke other program units.
• A program unit may be viewed as a piece of code implementing a “low”-level
function.
• Thus, a program unit is tested in isolation, that is, in a stand-alone manner.
• There are two reasons for testing a unit in a stand-alone manner.
– First, errors found during testing can be attributed to a specific unit so that it
can be easily fixed.
– Second, during unit testing it is desirable to verify that each distinct execution
of a program unit produces the expected result. Requires careful selection
of input data for each distinct execution
• Unit testing is performed by the programmer who writes the program unit
because the programmer is intimately familiar with the internal details of the unit.
• All programmers are accountable for the quality of their own work, which may
include both new code and modifications to the existing code.

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 3
 Concept of Unit Testing
• Unit testing has a limited scope.
• A programmer will need to verify whether or not a code works correctly by performing unit-
level testing. Intuitively, a programmer needs to test a unit as follows:
– Execute every line of code. This is desirable because the programmer needs to know
what happens when a line of code is executed.
– Observe that the unit performs its intended function and ensure that it contains no
known errors.
• Even though it is not possible to find all errors in a program unit in isolation, it means that
some errors in a program unit can only be found later, when the unit is integrated with
other units in the integration testing and system testing phases.
• It serves no purpose to integrate an erroneous unit with other units for the following
reasons: (i) many of the subsequent tests will be a waste of resources and
• (ii) finding the root causes of failures in an integrated system is more resource
consuming.
• The defects found during unit testing are internal to the software development group and
are not reported up the personnel hierarchy to be counted in quality measurement metrics.
• Unit testing is conducted in two complementary phases:
– Static unit testing , Dynamic unit testing

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 4
 Concept of Unit Testing
• Static Unit Testing
– A programmer does not execute the unit instead code is examined over all
possible behaviors that might arise during run time
– Code of each unit is validated against requirements of the unit by reviewing
the code.
– Static unit testing is also known as non-execution-based unit testing.
• Dynamic Unit Testing
– A program unit is actually executed and its outcomes are observed
– One observe some representative program behavior, and reach conclusion
about the quality of the system
– Dynamic unit testing is also known as execution based.
• Static unit testing is not an alternative to dynamic unit testing
• Static and Dynamic analysis are complementary in nature

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 5
 Concept of Unit Testing
Methods of testing :
1. Static Testing :
• Static testing is the testing of the software work products
manually, or with a set of tools, but they are not executed.
• It starts early in the Life cycle and so it is done during the
verification process.
• It does not need computer as the testing of program is done without
executing the program. For example: reviewing, walk through,
inspection, etc.
• Static testing consists of following methods
1) Walkthrough
2) Inspection
3) Technical Review

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 6
 Concept of Unit Testing
Advantages of Static Testing:
• Since static testing can start early in the life cycle, early feedback
on quality issues can be established.
• By detecting defects at an early stage, rework costs are most
often relatively low.
• Since rework effort is substantially reduced, development
productivity figures are likely to increase.

Disadvantages of Static Testing:

• Demand great amount of time when done manually

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 7
 Concept of Unit Testing
2. Dynamic Testing
• Dynamic testing (or dynamic analysis) is a term used in software
engineering to describe the testing of the dynamic behaviour of
code.
• That is, dynamic analysis refers to the examination of the physical
response from the system to variables that are not constant and
change with time.
• In dynamic testing the software must actually be compiled and
run.
• It involves working with the software, giving input values and
checking if the output is as expected by executing specific test
cases.
• Dynamic testing can be divided into unit testing, integration
testing, system testing, acceptance testing and finally regression
testing.

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 8
 Concept of Unit Testing
Advantages of Dynamic Testing:
• Dynamic testing could identify the weak areas in the runtime
environment.
• Dynamic testing supports application analysis even if the tester
does not have an actual code.
• Dynamic testing could identify some vulnerabilities that are
difficult to find by static testing.
• Dynamic testing also could verify the correctness of static testing
results.
• Dynamic testing could be applied to any application.
Disadvantages of Dynamic Testing:
• Finding trained dynamic test professionals is not easy.
• Dynamic testing is hard to track down the vulnerabilities in the
code, and it takes longer to fix the problem. Therefore, fixing
bugs becomes expensive.
Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 9
 Concept of Unit Testing
• In practice, partial dynamic unit testing is performed concurrently
with static unit testing
• It is recommended that static unit testing be performed prior to
the dynamic unit testing.
• If the entire dynamic unit testing has been performed and a
static unit testing identifies significant problems, the dynamic
unit testing must be repeated.
• As a result of this repetition, the development schedule may be
affected.
• To minimize the probability of such an event, it is required that
static unit testing be performed prior to the final dynamic unit
testing.

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 10
 Static Unit Testing
• In static unit testing code is reviewed by applying techniques, the original
definition of inspection was coined by Michael Fagan [1] and that of
walkthrough by Edward Yourdon [2]::
– Inspection: It is a step by step peer group review of a work product, with each step
checked against pre-determined criteria
– Walkthrough: It is review where the author leads the team through a manual or
simulated execution of the product using pre-defined scenarios
• The idea here is to examine source code in detail in a systematic manner
• The objective of code review is to review the code, and not to evaluate the author
of the code
• Code review must be planned and managed in a professional manner
• The key to the success of code is to divide and conquer
– An examiner inspect small parts of the unit in isolation
• nothing is overlooked
• the correctness of all examined parts of the module implies the correctness of the
whole module

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 11
 Static Unit Testing
1)Walkthrough
• In walkthrough, author guides the review team via the document to fulfill the
common understanding and collecting the feedback.
• Walkthrough is not a formal process.
• In walkthrough, a review team does not require to do detailed study before
meeting while authors are already in the scope of preparation.
• Walkthrough is useful for higher-level documents i.e requirement specification and
architectural documents.
Participants of Structured Walkthrough:
• Author
• Presenter
• Moderator
• Reviewers

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 12
BA(Business Analyst )- BA is Author who prepared BRS (Business
Requirement Specification) document

BA conducts informal meeting with Developers, testers, project

managers, Architects to review the BRS document.

Collecting the data informally is walk through

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 13
 Static Unit Testing
2) Inspection
• It is most formal type of review.
• The moderator guides the Inspection- conducting the meeting or organiser of the meeting. The
reviewers are prepared and check the documents before the meeting.
• Author – prepares the document related to project. E.g: tester prepares test document.
• In Inspection, a separate preparation is achieved when the product is examined and defects are found
i.e. finding the mistakes. These defects are documented in issue log.
• In Inspection, moderator performs a formal follow-up by applying exit criteria.
• Reader – Read the author’s document infront of everyone whether its code or document.
• Scribe – Note down the meeting discussion

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 14
 Static Unit Testing
2) Inspection
• The trained moderator guides the Inspection- conducting the meeting or organiser of the meeting. It is
most formal type of review.
• The reviewers are prepared and check the documents before the meeting.
• In Inspection, a separate preparation is achieved when the product is examined and defects are found.
These defects are documented in issue log.
• In Inspection, moderator performs a formal follow-up by applying exit criteria.

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 15
 Static Unit Testing
Difference between Inspection and Walkthrough.

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 16
 Static Unit Testing (Code Review)
• Step 1: Readiness
– Criteria
• Completeness
• Minimal functionality
• Readability
• Complexity
• Requirements and design
documents
– Roles
• Moderator
• Author
• Presenter
• Record keeper
• Reviewers
• Observer
• Step 2: Preparation
– List of questions
– Potential Change Request (CR)
– Suggested improvement opportunities
Figure 3.1: Steps in the code review process
Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 17
 Static Unit Testing (Code Review)
• Step 1: Readiness : A unit is said to be ready if it satisfies the
following criteria:
• Criteria
– Completeness : All the code relating to the unit to be
reviewed must be available. It is unproductive to review
partially written code
– Minimal functionality: The code must compile and link.
Code should performs its basic functionalities.
– Readability: are proper formatting, using meaningful
identifier names, use of programming language
constructs, and an appropriate level of abstraction using
function calls.
– Complexity: complexity is a composite term referring to the
number of conditional statements in the code, the
number of input data elements of the unit, the number of
output other units with which the code communicates. Figure 3.1: Steps in the code review process

– Requirements and design documents : Below documents

should be available. documents help the reviewers in
verifying whether or not the code under review
implements the expected functionalities.

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 18
 Static Unit Testing (Code Review)
• All the people involved in the review process are informed of the
group review meeting schedule two or three days before the meeting.
• Reviews are conducted in bursts of 1–2 hours.
• The composition of the review group involves a number of people
with different roles.
• Roles
– Moderator:
• The moderator is a trained individual who guides the pace of the
review process.
• The moderator selects the reviewers and schedules the review
meetings.
– Author : who has written the code
– Presenter:
• The presenter reads the code and he presents the author’s code in the
review meeting for the following reason:
• if the original programmer leaves the company with a short notice, at
least one other programmer in the company knows what is being Figure 3.1: Steps in the code review
done. process
– Record keeper:
• The recordkeeper documents the problems found during the review
process and follow-up actions suggested.
– Reviewers:
• experts in the subject area of the code under review.
– Observer:
• These are people who want to learn about the code under review.
These people do not participate in the review process but are simply
passive observers.
Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 19
 Static Unit Testing (Code Review)
• Step 2: Preparation : each reviewer carefully reviews the work
package before the meeting. Each reviewer develops the
following
– List of questions:
• reviewer prepares a list of questions to be asked, if needed, to
the author to clarify issues arising from his or her reading.
– Potential Change Request (CR):
• A reviewer may make a formal request to make a change. These
are called change requests rather than defect reports.
• At this stage, since the programmer has not yet made the code
public, it is more appropriate to make suggestions to the author
to make changes, rather than report a defect.
– Suggested improvement opportunities:
• The reviewers may suggest how to fix the problems, if there are
any, in the code under review. Since reviewers are experts in the
subject area of the code. Figure 3.1: Steps in the code
A Change Request (CR) includes the following details: review process
– Give a brief description of the issue
– Assign a priority level (major or minor) to a CR
– Assign a person to follow it up
– Set a deadline for addressing a CR
•

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 20
 Static Unit Testing (Code Review)
• Step 3: Examination • Step 4: Re-work
– Make the list of all the CRs
– The author makes a presentation
– Make a list of improvements
– The presenter reads the code line by line.
– Record the minutes meeting
– The reviewers may raise questions if the
code is seen to have defects and – Author works on the CRs to fix the
suggestions to fix the defects. issue
– The record keeper documents the CR • Step 5: Validation
along with deadline for addressing a CR. – CRs are independently validated
– Moderator ensures the review is on track• Step 6: Exit
• A Change Request (CR) includes the – Every line of code in the unit has been
following details: inspected.
– Give a brief description of the issue – if more than 5% of the total lines of
– Assign a priority level (major or minor) code are thought to be contentious,
to a CR then a second review is scheduled
– Assign a person to follow it up – All the CRs are documented and
– Set a deadline for addressing a CR validated by the moderator.
•
– A summary report of the meeting
including the CRs is distributed to all
the members of the review group.

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 21
 Static Unit Testing (Code Review Metrics)

Code Review Metrics

The following metrics can be collected from a code review:
• The number of lines of code (LOC) reviewed per hour
• The number of CRs generated per thousand lines of code (KLOC)
• The number of CRs generated per hour
• The total number of hours spend on code review process per project
• Total number of CRs generated per project

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 22
 Defect Prevention
• In general to reduce the number of CRs generated during code review..
• CRs are potential problems in the code, and those problems must be resolved
before different program units are integrated.
• Addressing CRs means spending more resources and potentially delaying the
project.
• It is useful to develop a set of guidelines to construct code for defect minimization
as explained in the following:
– Build internal diagnostic tools, also known as instrumentation code, into the units.
– Instrumentation codes are useful in providing information about the internal states of the units.
– These codes allow programmers to realize built-in tracking and tracing mechanisms.
• Instrumentation plays a passive role in dynamic unit testing.
• The role is passive in the sense of observing and recording the internal behavior
without actively testing a unit.

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 23
 Defect Prevention
• Use standard control to detect possible occurrences of error
conditions(eg: array index out of bounds)
• Ensure that code exists for all return values
• Ensure that counter data fields and buffer overflow/underflow are
appropriately handled.
• Provide error messages and help texts from a common source, Good error
messages identify the root causes of the problems and help users in
resolving the problems
• Validate input data
• Use assertions to detect impossible conditions
• Leave assertions in the code.
• Fully document the assertions that appears to be unclear
• After every major computation reverse-compute the input(s) from the
results in the code itself
• Include a loop counter within each loop

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 24
 Dynamic Unit Testing
• Execution-based unit testing is referred to as dynamic unit testing.
• The environment of a unit is emulated and tested in isolation.
• The context of a unit test consists of two parts:
– (i) a caller of the unit and (ii) all the units called by the unit.
• The caller unit is known as test driver
– A test driver is a program that invokes the unit under test (UUT)
– UUT executes with input values received from the driver and, upon termination, returns a value to
the driver.
– The driver compares the actual value returned by the unit under test, with the expected outcome
from the unit and reports the ensuing test result.
– The test driver functions as the main unit in the execution process.

Figure 3.2: Dynamic unit test environment

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 25
 Dynamic Unit Testing
• The emulation of the units called by the UUT are called stubs
– It is a dummy subprogram
• Stubs replace the units called by the unit under test.
• A stub performs two tasks.
– First, it shows an evidence that the stub was, in fact, called by printing a message.
– Second, the stub returns a precomputed value to the caller so that the unit under test can continue its execution.
• The test driver and the stubs are together called scaffolding.
• The driver and the stubs are never discarded those are reused in the future in regression testing of the
unit if there is such a need.

Figure 3.2: Dynamic unit test environment

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 26
 Stubs Example
• Consider an example of a web application, which consists of 4 modules i.e., Module-
A, Module-B, Module-C and Module-D.
Each of the following modules is responsible for some specific activity or functionality, as under:
Module-A  Login page of the web application.
Module-B → Home page of the web application.
Module-C → Print Setup.
Module-D → Log out page.
modules A, B, C & D involves the interdependencies of each module over other.

Module-A will be tested, as soon as, it develops. However, to carry out and validate the testing
procedures in respect of module-A, there urges the need of Module-B, which is not yet developed.

• The expected functionality of the login page (module-A) could be validated, only if it is
directed to the home page (Module-B), based on the valid and correct inputs.
But, on the non-availability of the Module-B, it will not be possible to test module-A. These types
of circumstances, introduces the stubs.
• A dummy module, representing the basic functionality or feature of the module-B, is being
developed, and thereafter, it is being integrated with the module-A, to perform testing,
efficiently.
• Log out page (Module-D), needs to be directed to the login page (Module-A), after
successfully logging out from the application. In the event of unavailability of Module-A, stubs
and drivers will work as a substitute for it, in order to carry out the testing of module-D.

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 27
 Dynamic Unit Testing
The low-level design document provides guidance for selection of input test data.
Selection of test data is broadly based on the following techniques:

• Control flow testing

– Draw a control flow graph (CFG) from a program unit
– Select a few control flow testing criteria
– Identify a path in the CFG to satisfy the selection criteria
– Derive the path predicate expression from the selection paths
– By solving the path predicate expression for a path, one can generate the data
• Data flow testing
– Draw a data flow graph (DFG) from a program unit and then follow the procedure
described in control flow testing.
• Domain testing
– Domain errors are defined and then test data are selected to catch the domain faults
• Functional program testing
– Input/output domains are defined to compute the input values that will cause the unit to
produce expected output values.

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 28
Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 29
 Mutation Testing
• Mutation testing is a technique that focuses on:
– measuring the adequacy of test data (or test cases).
– expose and locate weaknesses in test cases.
– measure the quality of test cases,
• A mutation of a program is a modification of the program created by
introducing a single, small, legal syntactic change in the code.
• A modified program is called mutant.
• Case 1: A mutant is said to be killed when the execution of test case
cause it to fail. The mutant is considered to be dead.

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 30
 Mutation Testing
• Case 2: A mutant is an equivalent to the given program if it always
produce the same output as the original program.
• Case 3: The result of executing a mutant may be different from the
expected result, but a test suite does not detect the failure because it
does not have the right test case. In this scenario the mutant is called
killable or stubborn.
• A mutant is called killable or stubborn, if the existing set of test
cases is insufficient to kill it.
• A mutation score for a set of test cases is the
– percentage of non-equivalent mutants killed by the test suite.
• The test suite is said to be mutation-adequate if its mutation score is
100%

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 31
 Mutation Testing
• Mutation analysis is a two-step process:
1. Those non-equivalent mutants that could not be identified by the
given test suite are called stubborn mutants.
2. New test cases are added to the existing test suite to kill the
stubborn mutants. The test suite enhancement process iterates until
the test suite has reached a desired level of mutation score.

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 32
 Mutation Testing
• Mutant 1:

• Mutant 2: if a<=-2 instead of 1st condition a<0

• Test case 1: a=1
• Test case 2: a=0
Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 33
 Mutation Testing

Test case Original Mutant 1 Mutant 2

Program
Removed elif a==0 a<=-1 instead of a<0

1) input a=1 input is greater input is greater than input is greater than zero
than zero zero Equivalent Mutant
Equivalent Mutant

2) input a=0 Input is equal to input is greater than Input is equal to zero
zero zero
Equivalent Mutant
Killable Mutant
3)input a=-1 input is less than input is less than No output
zero zero Killable Mutant
Equivalent Mutant

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 34
Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 35
 Mutation testing
• In order to use the mutation testing technique to build a robust test suite, the test
engineer needs to follow the steps that are outlined below:
• Step 1: Begin with a program P and a set of test cases T known to be correct.
• Step 2: Run each test case in T against the program P. If a test case fails, the output
is incorrect, program P must be modified and retested. If there are no failures, then
continue with step 3.
• Step 3: Create a set of mutants {Pi }, each differing from P by a simple,
syntactically correct modification of P.
• Step 4: Execute each test case in T against each mutant Pi . If the output of the
mutant Pi differs from the output of the original program P, the mutant Pi is
considered incorrect and is said to be killed by the test case. If Pi produces exactly
the same results as the original program P for the tests in T, then one of the
following is true:
– P and Pi are equivalent. Their behaviors cannot be distinguished by any set of test cases. Note that
the general problem of deciding whether or not a mutant is equivalent to the original program is
undecidable.
– Pi is killable. That is, the test cases are insufficient to kill the mutant Pi . In this case, new test
cases must be created.

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 36
 Mutation testing
• Step 5: Calculate the mutation score for the set of test cases T. The mutation score
is the percentage of non-equivalent mutants killed by the test data, that is, Mutation
score = 100×D/(N −E), where D is the dead mutants, N the total number of
mutants, and E the number of equivalent mutants.
• Step 6: If the estimated mutation adequacy of T in step 5 is not sufficiently high,
then design a new test case that distinguishes Pi from P, add the new test case to T,
and go to step 2. If the computed adequacy of T is more than an appropriate
threshold, then accept T as a good measure of the correctness of P with respect to
the set of mutant programs Pi , and stop designing new test cases.

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 37
 Mutation testing
Consider the following program P • Test Case 1:
• main(argc,argv) input: 1 2 3
• int argc, r, i; output: Value of the rank is 3
• char *argv[]; • Test Case 2:
• { r = 1; input: 1 2 1
• for i = 2 to 3 do output: Values of the rank is 2
• if (atoi(argv[i]) > atoi(argv[r])) r = i; • Test Case 3:
• printf(“Value of the rank is %d \n”, r); input: 3 1 2
• exit(0); } output: Value of the rank is 1

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 38
 Mutation testing
Consider the following program P • Test Case 1:
• main(argc,argv) input: 1 2 3
• int argc, r, i; output: Value of the rank is 3
• char *argv[]; • Test Case 2:
• { r = 1; input: 1 2 1
• for i = 2 to 3 do output: Values of the rank is 2
• if (atoi(argv[i]) > atoi(argv[r])) r = i; • Test Case 3:
• printf(“Value of the rank is %d \n”, r); input: 3 1 2
• exit(0); } output: Value of the rank is 1
Mutant 1: Change line 5 to for i = 1 to 3 do
Mutant 2: Change line 6 to if (i > atoi(argv[r])) r = i;
Mutant 3: Change line 6 to if (atoi(argv[i]) >= atoi(argv[r])) r = i;
Mutant 4: Change line 6 to if (atoi(argv[r]) > atoi(argv[r])) r = i;
Execute modified programs against the test suite, you will get the results:
Mutants 1 & 3: Programs will pass the test suite, i.e., mutants 1 & 3 are not killable
Mutant 2: Program will fail test cases 2
Mutant 1: Program will fail test case 1 and test cases 2
Mutation score is 50%, assuming mutants 1 & 3 non-equivalent
Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 39
 Mutation testing
• The score is found to be low because we assumed mutants 1 & 3 are nonequivalent
• We need to show that mutants 1 and 3 are equivalent mutants or those are killable
• To show that those are killable, we need to add new test cases to kill these two
mutants
• First, let us analyze mutant 1 in order to derive a “killer” test. The difference
between P and mutant 1 is the starting point
• Mutant 1 starts with i = 1, whereas P starts with i = 2. There is no impact on the
result r. Therefore, we conclude that mutant 1 is an equivalent mutant
• Second, if we add a fourth test case as follows:
Test Case 4:
input: 2 2 1
• Program P will produce the output “Value of the rank is 1” and mutant 3 will
produce the output “Value of the rank is 2”
• Thus, this test data kills mutant 3, which give us a mutation score 100%

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 40
 Mutation Testing
Mutation testing makes two major assumptions:
• Competent Programmer hypothesis
– Programmers are generally competent and they do not create random programs
– Assume that for a given problem a programmer will create a correct program
except for simple errors.
– the mutants to be considered are the ones falling within a small deviation
from the original program.

• Coupling effects
– Complex faults are coupled to simple faults in such a way that a test suite
detecting simple faults in a program will detect most of the complex faults.
– If the software contains a fault, there will usually be a set of mutants that can
only be killed by a test case that also detect that fault.

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 41
 Debugging
• The process of determining the cause of a failure is known as
debugging
• It is a time consuming and error-prone process
• Debugging involves a combination of systematic evaluation,
intuition and a little bit of luck
• The purpose is to isolate and determine its specific cause, given a
symptom of a problem
• There are three approaches to debugging
– Brute force : “let the computer find the error” philosophy is used.
– Cause elimination
• Induction
• Deduction
– Backtracking

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 42
 Debugging
– Brute force :
• “let the computer find the error” philosophy is used.
• Print statements are scattered throughout the source code, this helps in tracing of
the source code being executed.
• A dynamic debugger allows the software engineer to navigate by stepping through
the code, observe which paths have executed, and observe how values of variables
change during the controlled execution.
– Cause elimination
• Induction :
– all pertinent data related to the failure are collected , such as what happened and what
the symptoms are.
– Next, the collected data are organized in terms of behavior and symptoms, and their
relationship is studied to find a pattern to isolate the causes.
• Deduction:
– a list of all possible causes is developed in order of their likelihoods, and tests are
conducted to eliminate or substantiate each cause in decreasing order of their
likelihoods.
– Backtracking :
• the programmer starts at a point in the code where a failure was observed and traces back the
execution to the point where it occurred.

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 43
 Debugging
– Brute force :
• “let the computer find the error” philosophy is used.
• Print statements are scattered throughout the source code, this helps in tracing of
the source code being executed.

• The brute-force methods are characterized by either

– debugging with a memory dump;

– scattering print statements throughout the program, or

– debugging with automated debugging tools(check point can be used to debug

the issue)
Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 44
 Debugging

- Backtracking :
The programmer starts at a point in the code where a failure was observed and
traces back the execution to the point where it occurred.

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 45
 Debugging
– Debugging by Induction
– The induction process is illustrated in below Figure

– Locate the pertinent data

– Organize the data
– Devise a hypothesis
– Prove the hypothesis

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 46
 Debugging
– Debugging by Deduction

– The induction process is illustrated in below Figure

– Enumerate the possible causes or hypotheses

– Use the data to eliminate possible causes
– Refine the remaining hypothesis.
– Prove the remaining hypothesis

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 47
 Debugging
The following heuristic may be followed to isolate and correct it:
• Step 1: Reproduce the symptom(s).
• Step 2: Formulate some likely hypotheses for the cause of the
problem based on the causal analysis.
• Step 3: Develop a test scenario for each hypothesis to be proved
or disproved.
• Step 4: Prioritize the execution of test cases.
• Step 5: Execute the test cases in order to find the cause of a
symptom.
• Step 6: Fix the problem.

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 48
 JUnit – A Framework for Unit Testing
• JUnit: It is a framework for performing unit testing of Java programs.
– Other frameworks: NUnit (C#), CPPUnit (C++), fUnit (Fortran)
• Intuitive steps to test a method in Java (Ex. Move() method of
PlanetClass)
– Create an object instance of PlanetClass. Call it Mars.
– Select values of all input parameters of Move().
– Compute the expected value to be returned by Move(). Let it be y.
– Execute method Move() on Mars with the selected input values.
• Let Move() return a value called z.
– Compare the actual output (z) returned by Move() with the expected value (y).
• If (z == y), Move() passes the test; otherwise it fails.  Report the result.
• JUnit makes writing of test cases easier.  Next slide …

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 49
 JUnit – A Framework for Unit Testing
• JUnit provides a basic class called TestCase.
• The tester
– Extends the TestCase class for each test case. 10 extensions for 10 test cases.
– Alternatively, extend TestCase to have 10 methods for 10 test cases.
• The TestCase class provides methods to make assertions.
– assertTrue(Boolean condition)
– assertFalse(Boolean condition)
– assertEquals(Object expected, Object actual)
– assertEquals(int expected, int actual)
– assertEquals(double expected, double actual, double tolerance)
– assertSame(Object expected, Object actual)
– assertNull(Object testobject)
– …
• The tester can have her own assertions.

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 50
 JUnit – A Framework for Unit Testing
• Each assertion accepts an optional first parameter of type String; if
the assertion fails, the string is displayed.  Help for the tester…
• The assertEquals() method displays a message upon failure.
• [Link]: expected: <x> but was: <y>
• Note that only failed tests are reported.
• The following shows how assertTrue() works.

static public void assertTrue(Boolean condition) {

if (!condition)
throw new AssertionFailedError();
}

Figure 3.5: The assertTrue() assertion throws an exception

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 51
 JUnit – A Framework for Unit Testing
import TestMe; // TestMe is the class whose methods are going to be tested.
import [Link].*; // This contains the TestCase class.

public class MyTestSuite extends TestCase { // Create a subclass of TestCase

public void MyTest1() { // This method is the first test case

TestMe object1 = new TestMe( ... ); // Create an instance of TestMe with desired params
int x = object1.Method1(...); // invoke Method1 on object1
assertEquals(365, x); // 365 and x are expected and actual values, respectively.
}

public void MyTest2() { // This method is the second test case

TestMe object2 = new TestMe( ... ); // Create another instance of
// TestMe with desired parameters
double y = object2.Method2(...); // invoke Method2 on object2
assertEquals(2.99, y, 0.0001d); // 2.99 is the expected value;
// y is the actual value;
// 0.0001 is tolerance level
}
}

Figure 3.5: An example test suite

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 52
 References
– [Link]
– [Link]
– [Link]

Software Testing and QA Theory and Practice (Chapter 3: Unit Testing) © Naik & Tripathy 53