 Development is accompanied by “bugs.

”
 Catching bugs early saves money
◦ The further a bug progresses the more impact it has
on the system
 A bug fix requires all related modules to be retested.
 A bug fix may require redesigning related modules.
◦ For example
 PCB design flaw
 VLSI layout error
 Subtle coding flaw
 Testing doesn’t remove bugs, it just makes it
less likely they exist.

2
By the end of this chapter, you should:
 Understand the concepts of black box tests, white
box tests, observability, and controllability.
 Understand the principles of debugging.
 Understand when a unit test is used and how it is
constructed.
 Understand when an integration test is used and
how it is constructed.
 Understand when an acceptance test is used and
how it is constructed.

3
 Testing proceeds with design process
◦ Write tests while designing modules,
◦ Perform tests while implementing modules
 The Test-Vee illustrates this process
Development Testing

Requirements Acceptance
Specification Test

System Integration
Design Test

Unit
Construct
Test

Debugging

4
 Quality design documents
◦ Requires you to reason about system behavior
◦ Gives you design clear goals
◦ Look at your system from the tester’s perspective
 How should we perform testing?
◦ Tradeoff
 Apply every possible input, or
 Apply selected inputs which might yield errors
◦ Test should be chosen to increase likelihood of finding
an error.

5
 Test cases define exactly what the module must do.
 Testing prevents feature creep, since the development
of a module is complete when its test is passed.
 Test cases motivate developers by providing immediate
feedback.
 Test cases force designers to think about extreme
cases.
 Test cases are a form of documentation.
 Test cases force the designer to consider the design of
the module before building it.

6
 Black box
◦ No knowledge of internal organization
◦ Only access input and outputs
◦ Change inputs and observe outputs
 White box
◦ Knowledge of internal organization
◦ Might have expectation of fault model
◦ Create test instance which reveal physical or logical
errors

7
 Testable – failure of a component can
◦ Quickly detected
◦ Quickly located
 Controllability
◦ When any node of the system can be set to a
desired value
◦ Black box has no controllability
 Observability
◦ When any node of the system can be measured.
◦ Black box has low observability

8
 Stub
◦ A placeholder for future functionality
◦ A device which mimics subsystem
 Simulates input or monitors outputs
 For a unit under test (UUT)
 Insure good behavior
 before wreaking havoc on the rest of the system
 For example
◦ A function generator for a audio input
◦ A printf() instead of a file write
◦ DIP switch instead of a bus connection

9
 stub simulating stub simulating
expected input expected output
R C

function
BJT Amplifier RL
generator

10
 Accurate - The test should check what it is supposed to and
exercise an area of intent.
 Economical - The test should be performed in a minimal
number of steps.
 Limited in complexity - Tests should consist of a moderate
number (10-15) of steps.
 Repeatable - The test should be able to be performed and
repeated by another person.
 Appropriate - The complexity of the test should be such that
it is able to be performed by other individuals who are
assigned the testing task.
 Traceable - The test should verify a specific requirement.
 Self cleaning - The system should return to the pre-test state
after the test is complete.

11
Debugging
 Bohrbugs
◦ Bohrbugs are reliable bugs
◦ The error is always in the same place
◦ An electrons having a definite position
◦ Solution = set a good trap
 Heisenbugs
◦ Innocuous changes of input yield buggy behavior
◦ May not be reproducible
◦ They seemingly move around within a system
◦ Electron is elusive density function
◦ Solution = think outside the box

12
 Observe the problem under different
operating conditions
 Form a hypothesis as to what the potential

problem is
 Conduct experiments to confirm or eliminate

the hypothesized source of the problem

 Repeat until the problem is eliminated

13
 Check easiest problems first
◦ You can perform more in a given time
 Start at lowest levels of abstraction
◦ Upper levels rely on lower level
 Example
◦ Is the system powered up?
◦ Is the testing equipment adjusted properly?
◦ Are the bus lines being correctly manipulated?
◦ Have you initialized the system?
◦ Are you printing out the right variable/type?

14
 A unit test is a test of the functionality of a
system module in isolation
 Should be traceable to the detailed design.
 Consists of a set of test cases
 Each test case establish that a subsystem
performs a single unit of functionality to
some specification.
 Test cases should be written with the express
intent of uncovering undiscovered defects.

15
 Write unit test during implementation;
why?
◦ White box tests
◦ Thinking about test situations and conditions
may lead the designer to uncover errors before
they are ever designed into the system.
◦ Unit tests need to be written with an
understanding of the internal organization of
the component, and a system is best
understood during its development.

16
if (16 < Celsius Temperature < 32)
Fahrenheit Temperature = ROM[input-16];
else
Fahrenheit Temperature = (9* Celsius Temperature)/5 + 32;

 What inputs are used to check the ROM?

 What inputs check the conditionals?
 Processing path – sequence from A to B

◦ Test coverage
◦ Path complete coverage

17
 Determine inputs to check all paths
◦ ACTION1
◦ ACTION2
◦ ACTION3

if ((x >= 30) && (x <= 39)) {

ACTION1
} else if ((x >= 80) && (x <= 96)) {
ACTION2
} else if ((x >= 40) && (x <= 56)) {
ACTION 3
}

18
 Matrix Test
 Automated Scripts Tests
 Step-by-step Test

Note: these can be applied to any level of test:

unit, integration, acceptance, etc.

Copyright 2005 19
 Test Writer: Sue L. Engineer
  Test Case Name: ADC function test Test ID #: ADC-FT-01
  Description: Verify conversion range and clock Type: white box
frequency. Output goes to 0 in presence of  black box
null clock.

Tester Information
  Name of Tester: Date:
  Hardware Ver: 1.0 Time:
Setup: Isolate the ADC from the system by removing configuration jumpers.
VT Clock Expected output Comments
Test

Pass

Fail
N/A
Decimal Hexadecimal
1 0.0V 10kHz 0 0x000

5 2.0V 0Hz 0 0x000

  Overall test result:        

20
Test Writer: Sue L. Engineer

  Test Case Name: Finite State Machine Path Test #1 Test ID #: FSM-Path-01

  Description: Simulate insertion of money with a mix of nickels and Type: white box
dimes. Verifies FSM, outputs candy in response to black box
a total deposit of $0.30.

Tester Information

  Name of Tester: Date:

  Hardware Ver: 1.0 Time:

Setup: Make sure that the system was reset sometime prior and is in state $0.00.

Action Expected Result Comments

Step

Pass

Fail
N/A
1 Strobe Nickel State should go to $0.05

2 Strobe Dime State should go to $0.15

3 Wait State should remain $0.15

4 Strobe Nickel State should go to $0.20

5 Strobe Dime State should go to $0.25

6 Nothing State should go to $0.00

21
  Overall test result:        
 Write integration test during level 1 design
◦ Help insure requirements are being met.
◦ Help to firm-up design
◦ Requires the designer think about the
expected behavior of the subsystems.
◦ Requires designer to think about extreme
behaviors of subsystems.

22
 What are the different paths of execution
through the system?
 Are all modules exercised at least once
during integration testing?
 Have all the interface signals been tested?
 Have all the interface modes been
exercised?
 Does the system process information at the
required rate and met timing requirements?

23
 Might be formal legal document
 Written along with requirements
 Traceable to engineering requirements
 Identifies
◦ Scope – how much of the system is tested?
◦ Level – how deep will testing be
performed?

24
 Autonomous navigating robot
 Engineering requirements
◦ The robot’s center must stay within 12 to 18
centimeters of the wall over 90% of the course,
while traveling parallel to a wall over a 3 meter
course.
◦ The robot’s heading should never deviate no
more than 10 degrees from the wall’s axis, while
traveling parallel to a straight wall over a 3 meter
course.

25
26
 Switches

Digital
H-bridge DC motor
Compass

MCU

Range
H-bridge DC motor
Finder

LCD

27
 MCU + motors + bridge + switches

 Chassis + digital compass + MCU + motors

+ bridge +LCD

 Chassis + range finder + MCU + motors +

bridge

28
29
 MCU (hardware)
 LCD
 Switches
 Compass
 Range finder
 H-bridge
 Motors
 Chassis
 MCU (software)

30
Module Digital Compass – Geosensor version 2.3
Inputs -
Earth’s magnetic field: An orientated field of magnetic
force beginning and ending at the earth’s magnetic poles.
-
SClk – Clock signal to clock data through the module.
Maximum Frequency is 10Mhz.
-
SDIn – Serial data input to send data into the compass
module. Date is valid on positive SClk edges.

Outputs -
SDOut – Serial data output from the compass module.
Data is valid on negative clock edges.
Functionality Senses the earth’s magnetic field and determines the
orientation of the compass with respect to the field. This
orientation is stored in an internal register and can be
retrieved through the SPI interface.
Test Comp-UT-01

31
32
Reasons to develop and conduct tests
 Testing reduces the number of bugs in existing and new
features.
 Tests are good documentation.
 Tests improve design.
 Tests allow you to refactor.
 Tests constrain features.
 Tests defend against other designers.
 Testing is fun.
 Testing forces you to slow down and think.
 Testing makes development faster.
 Tests reduce fear.

33
 Testing helps to ensure proper operation of
system.
 Types of Testing
◦ Block box
◦ White box
◦ Unit Testing
◦ Integration Testing
◦ Acceptance Testing
◦ Matrix tests
◦ Step-by-step test
◦ Automated tests

34