Testing, Debugging and Exceptions
CMPS 203: PROGRAMMING FOR EVERYONE
1
Testing and
Debugging
2
Programming in CMPS 203
[Link]
EXPECTATION
[Link]
REALITY
What you want the program to do
What the program actually does
3
Debugging: catching and
killing bugs
4
September 9, 1947
▪ Mark II Aiken Relay Computer
5
Grace Hopper
6
7
Analogy with Soup
You are making soup but bugs keep falling in from the
ceiling. What do you do?
▪ Check soup for bugs
• testing
▪ Keep lid closed
• defensive
programming
▪ Clean kitchen
• eliminate source
of bugs
Analogy thanks to Prof Srini Devadas
8
DEFENSIVE PROGRAMMING
• Modularize programs into functions
• Write specifications for functions
• Check conditions on inputs/outputs (assertions)
TESTING/VALIDATION DEBUGGING
• Compare input/output • Study events leading up
pairs to specification to an error
• “It’s not working!” • “Why is it not working?”
• “How can I debug my • “How can I fix my
program?” program?”
9
Set yourself up for easy testing
and debugging
▪ From the start, design code to ease this part
▪ Break program up into modules that can be tested
and debugged individually
▪ Document constraints on modules
• what do you expect the input to be?
• what do you expect the output to be?
▪ Document assumptions behind code design
10
A Complete Program
▪ Write a Python program that reads the lyrics of your
favorite song from a file and then stores the frequency
of each word in the song
▪ It should also print the list of words with the highest
frequencies
11
Decomposition
▪ Three functions:
◦ One that reads words from the file and returns them as a
list of words
◦ One that loops over a list of words and count how many
times the word occurs in the list. It returns a word-
frequency dictionary
◦ One that loops over word-frequency dictionary and finds
the words with the highest frequency. It returns a tuple
consisting of a list of words and the maximum frequency
12
When are you ready to test?
▪ Ensure code runs
• remove syntax errors
• remove static semantic errors
• Python interpreter can usually find these for you
▪ Have a set of test cases
• an input set
• for each input, the expected output
13
Debugging
▪ Steep learning curve
▪ Goal is to have a bug-free program
▪ Tools
• Code Editor and Interpreter
• Python Tutor and Internet
• print statements
• Use your brain and be systematic in your hunt
14
Semantic (Logical) Errors -
Hard
▪ Think before writing new code
▪ Draw pictures, take a break
▪ Explain the code to
• someone else
• a rubber ducky
15
DON’T DO
• Write entire program • Write a function
• Test entire program • Test the function, debug the function
• Debug entire program • Write a function
• Test the function, debug the function
• *** Do integration testing ***
• Change code • Backup code
• Remember where bug was • Change code
• Test code • Write down potential bug in a
• Forget where bug was or what change comment
you made • Test code
• Panic • Compare new version with old
version
16
A Recipe for Success
▪ Form hypothesis of what is wrong
▪ Use print statements to test hypotheses
• Print when you enter functions
• Print parameters
• Print function results
▪ Use bisection method
• Put print halfway in code
• Decide where bug may be depending on values
17
Debugging Example
def isPal(x): def silly(s):
for e in s:
temp = x
[Link] result = []
if temp == x:
[Link](e)
return True
if isPal(result):
else:
print('Yes’)
return False
else:
print(‘No’)
18
Debugging Example
▪ Test on
◦ “abba” returns True as expected
◦ “abcba” returns True as expected
◦ “ab” returns True, so we found a bug
▪ Let’s use binary search to “isolate” bug(s)
◦ pick a spot about halfway through code, and devise
experiment
19
Debugging Example
def isPal(x): def silly(s):
for e in s:
temp = x
[Link] result = []
if temp == x:
[Link](e)
return True
print(result)
else:
if isPal(result):
return False
print('Yes’)
else:
print(‘No’)
20
Debugging Example
▪ At this point in the code, we expect (for our test case
of ‘ab’), that result should be a list [‘a’, ‘b’]
▪ We run the code, and get [‘b’]
▪ Because of binary search, we know that at least one
bug must be present earlier in the code
▪ So we add a second print, this time inside the loop
21
Debugging Example
def isPal(x): def silly(s):
for e in s:
temp = x
[Link] result = []
if temp == x:
[Link](e)
return True
print(result)
else:
if isPal(result):
return False
print('Yes’)
else:
print(‘No’)
22
Debugging Example
▪ When we run on our example, the print statement
returns
◦ [‘a’]
◦ [‘b’]
▪ This suggests that result is not keeping all elements
▪ So let’s move the initialization of result outside the
loop and retry
23
Debugging Example
def isPal(x): def silly(s):
result = []
temp = x
[Link] for e in s:
if temp == x:
[Link](e)
return True
print(result)
else:
if isPal(result):
return False
print('Yes’)
else:
print(‘No’)
24
Debugging Example
▪ This now shows we are getting the list result
properly setup, but we still have a bug somewhere
◦ A reminder that there may be more than one problem!
◦ This suggests second bug must lie below print statement;
◦ Let’s look at isPal()
◦ Pick a point in middle of code, and add print statement
again; remove the earlier print statement
25
Debugging Example
def isPal(x): def silly(s):
result = []
temp = x
[Link] for e in s:
print(temp,x) [Link](e)
if temp == x:
if isPal(result):
return True
print('Yes’)
else:
else:
return False
print(‘No’)
26
Debugging Example
▪ At this point in the code, we expect that x should be
[‘a’, ‘b’], but temp should be [‘b’, ‘a’]
▪ However they both have the value [‘a’, ‘b’]
▪ So let’s add another print statement, earlier in the
code
27
Debugging Example
def isPal(x): def silly(s):
result = []
temp = x
for e in s:
print(temp,x)
[Link](e)
[Link]
if isPal(result):
print(temp,x)
if temp == x: print('Yes’)
return True else:
else: print(‘No’)
return False
28
Debugging Example
▪ We see that temp has the same value before and after
the call to reverse
▪ If we look at our code, we realize we have committed
a standard bug – we forgot to actually invoke the
reverse method
◦ Need [Link]()
◦ So let’s make that change and try again
29
Debugging Example
def isPal(x): def silly(s):
result = []
temp = x
for e in s:
print(temp,x)
[Link](e)
[Link]()
if isPal(result):
print(temp,x)
if temp == x: print('Yes’)
return True else:
else: print(‘No’)
return False
30
Debugging Example
▪ But now when we run on our simple example, both x
and temp have been reversed!
▪ We have also narrowed down this bug to a single line.
The error must be in the reverse step
▪ In fact, we have an aliasing bug – reversing temp has
also caused x to be reversed
◦ because they are referring to the same object
31
Debugging Example
def isPal(x): def silly(s):
result = []
temp = x[:]
for e in s:
print(temp,x)
[Link](e)
[Link]()
if isPal(result):
print(temp,x)
if temp == x: print('Yes’)
return True else:
else: print(‘No’)
return False
32
Debugging Example
▪ Now running this shows that before the reverse step,
the two variables have the same form, but afterwards
only temp is reversed
▪ We can now go back and check that our other tests
cases still work correctly
33
Debugging Summary
▪ Look for the usual suspects
▪ Ask why the code is doing what it is, not why it is not
doing what you want
▪ The bug is probably not where you think it is so
eliminate locations through binary search
▪ Explain the problem to someone else
▪ Don’t believe the documentation
▪ Take a break and come back to the bug later
34
Exceptions
35
Exceptions
▪ What happens when procedure execution hits an
unexpected condition?
▪ We get an exception… to what was expected
• trying to access beyond list limits
test = [1,7,4]
test[4] → IndexError
• trying to convert an inappropriate type
int(test) → TypeError
• referencing a non-existing variable
a → NameError
• mixing data types without coercion
'a'/4 → TypeError
36
Types of Exceptions
▪ Common error types:
• SyntaxError: Python can’t parse program
• NameError: local or global name not found
• TypeError: operand doesn’t have correct type
• ValueError: operand type okay, but value is illegal
• IOError: IO system reports malfunction (e.g. file not
found)
37
What to do with exceptions?
▪ What to do when we encounter an error?
▪ So far, interpreter stops program execution and signals
error message
a = int(input(“Enter a number: ”))
b = int(input(“Enter another number: ”))
print(a/b)
▪ What can go wrong with the above code?
◦ cannot convert user input to integers
◦ b is zero so division by zero
38
Dealing with Exceptions
▪ Instead, you should catch exceptions and deal with them
try:
a = int(input(“Enter a number: ”))
b = int(input(“Enter another number: ”))
print(a/b)
except:
print(“non integers or second is zero.”)
▪ Exceptions raised by any statement in body of try are
handled by the except statement and execution continues
with the body of the except statement
39
Handling Specific Exceptions
▪ Have separate except clauses to deal with a particular type of
exception
try:
a = int(input(”Enter a number: "))
b = int(input(”Enter another number: "))
print(a/b)
except ValueError:
print ("Could not convert to integers.”)
except ZeroDivisionError:
print("Cannot divide by zero.”)
except:
print(“Something went really wrong.”)
40
Sanity Check
▪ Explain how the following code will be executed for
different inputs
try:
a = int(input(”Enter a number: "))
b = int(input(”Enter another number: "))
print(”so far so good”)
print(a/b)
print(“all good”)
except:
print(“Something went wrong!”)
print(“done!”)
41
Another Complete Example
▪ Assume we are given a dictionary for CMPS 203:
◦ Key is the name of the student
◦ Value is a list of grades on the assignments
classdict = {‘Ali’: [80.0, 70.0, 85.0],
‘Hoda’: [100.0, 80.0, 74.0]}
▪ Create a new dictionary, with name and average
{‘Ali’:78.33333, ‘Hoda’: 84.666667}
42
Example classdict = {‘Ali’: [80.0, 70.0, 85.0],
‘Hoda’: [100.0, 80.0, 74.0]}
Code
def get_stats(classdict):
stats = {}
for e in classdict:
stats[e] = avg(classdict[e])
return stats
def avg(grades):
return sum(grades)/len(grades)
43
Error if no grades for a student
▪ If one or more students don’t have any grades,
we get an error ZeroDivisionError
classdict = {‘Ali’: [80.0, 70.0, 85.0],
‘Hoda’: [100.0, 80.0, 74.0],
‘Alain’: []}
return sum(grades)/len(grades)
44
Option 1: Flag the error by
printing a message
▪ Decide to notify that something went wrong with a msg
def avg(grades):
try:
return sum(grades)/len(grades)
except ZeroDivisionError:
print(‘No grades.)’
▪ Running on previous test data gives
No grades.
{'Ali': 78.33333333333333, 'Hoda':
84.66666666666667, 'Alain': None}
45
Option 2: Change the policy
▪ Decide that a student with no grades gets a zero
def avg(grades):
try:
return sum(grades)/len(grades)
except ZeroDivisionError:
print(No grades.)’
return 0.0
▪ Running on previous test data gives
No grades.
{'Ali': 78.33333333333333, 'Hoda':
84.66666666666667, 'Alain’: 0.0}
46
Raising Exceptions
▪ You can also raise an exception when you are unable
to produce a result consistent with function’s
specification
raise ValueError("something is wrong")
▪ The exception gets raised to the caller
47
Example: ratio between lists
def get_ratios(L1, L2):
""" Assumes: L1 and L2 are lists of equal length of numbers
Returns: a list containing L1[i]/L2[i] """
ratios = []
for index in range(len(L1)):
try:
[Link](L1[index]/L2[index])
except ZeroDivisionError:
[Link](float('NaN')) #NaN = Not a Number
except:
raise ValueError('get_ratios called with bad arg')
return ratios
48