Downloaded 13 times
![6 Chapter 1. Why should you learn to write programs?
and del from not while
as elif global or with
assert else if pass yield
break except import print
class exec in raise
continue finally is return
def for lambda try
That is it, and unlike a dog, Python is already completely trained. When you say
“try”, Python will try every time you say it without fail.
We will learn these reserved words and how they are used in good time, but for
now we will focus on the Python equivalent of “speak” (in human-to-dog lan-
guage). The nice thing about telling Python to speak is that we can even tell it
what to say by giving it a message in quotes:
print 'Hello world!'
And we have even written our first syntactically correct Python sentence. Our
sentence starts with the reserved word print followed by a string of text of our
choosing enclosed in single quotes.
1.5 Conversing with Python
Now that we have a word and a simple sentence that we know in Python, we need
to know how to start a conversation with Python to test our new language skills.
Before you can converse with Python, you must first install the Python software on
your computer and learn how to start Python on your computer. That is too much
detail for this chapter so I suggest that you consult www.pythonlearn.com where
I have detailed instructions and screencasts of setting up and starting Python on
Macintosh and Windows systems. At some point, you will be in a terminal or
command window and you will type python and the Python interpreter will start
executing in interactive mode and appear somewhat as follows:
Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
The >>> prompt is the Python interpreter’s way of asking you, “What do you want
me to do next?” Python is ready to have a conversation with you. All you have to
know is how to speak the Python language.
Let’s say for example that you did not know even the simplest Python language
words or sentences. You might want to use the standard line that astronauts use
when they land on a faraway planet and try to speak with the inhabitants of the
planet:](https://image.slidesharecdn.com/pythonforinformatics-160323130021/85/Python-for-informatics-18-320.jpg)
![6 Chapter 1. Why should you learn to write programs?
and del from not while
as elif global or with
assert else if pass yield
break except import print
class exec in raise
continue finally is return
def for lambda try
That is it, and unlike a dog, Python is already completely trained. When you say
“try”, Python will try every time you say it without fail.
We will learn these reserved words and how they are used in good time, but for
now we will focus on the Python equivalent of “speak” (in human-to-dog lan-
guage). The nice thing about telling Python to speak is that we can even tell it
what to say by giving it a message in quotes:
print 'Hello world!'
And we have even written our first syntactically correct Python sentence. Our
sentence starts with the reserved word print followed by a string of text of our
choosing enclosed in single quotes.
1.5 Conversing with Python
Now that we have a word and a simple sentence that we know in Python, we need
to know how to start a conversation with Python to test our new language skills.
Before you can converse with Python, you must first install the Python software on
your computer and learn how to start Python on your computer. That is too much
detail for this chapter so I suggest that you consult www.pythonlearn.com where
I have detailed instructions and screencasts of setting up and starting Python on
Macintosh and Windows systems. At some point, you will be in a terminal or
command window and you will type python and the Python interpreter will start
executing in interactive mode and appear somewhat as follows:
Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
The >>> prompt is the Python interpreter’s way of asking you, “What do you want
me to do next?” Python is ready to have a conversation with you. All you have to
know is how to speak the Python language.
Let’s say for example that you did not know even the simplest Python language
words or sentences. You might want to use the standard line that astronauts use
when they land on a faraway planet and try to speak with the inhabitants of the
planet:](/p?url=https%3A%2F%2Fimage.slidesharecdn.com%2Fpythonforinformatics-160323130021%2F85%2FPython-for-informatics-18-320.jpg&__src=https%3A%2F%2Fwww.slideshare.net%2Fslideshow%2Fpython-for-informatics%2F59933486%2369&__type=image)
![1.6. Terminology: interpreter and compiler 9
An interpreter reads the source code of the program as written by the program-
mer, parses the source code, and interprets the instructions on the fly. Python is
an interpreter and when we are running Python interactively, we can type a line
of Python (a sentence) and Python processes it immediately and is ready for us to
type another line of Python.
Some of the lines of Python tell Python that you want it to remember some value
for later. We need to pick a name for that value to be remembered and we can use
that symbolic name to retrieve the value later. We use the term variable to refer
to the labels we use to refer to this stored data.
>>> x = 6
>>> print x
6
>>> y = x * 7
>>> print y
42
>>>
In this example, we ask Python to remember the value six and use the label x so
we can retrieve the value later. We verify that Python has actually remembered
the value using print. Then we ask Python to retrieve x and multiply it by seven
and put the newly computed value in y. Then we ask Python to print out the value
currently in y.
Even though we are typing these commands into Python one line at a time, Python
is treating them as an ordered sequence of statements with later statements able
to retrieve data created in earlier statements. We are writing our first simple para-
graph with four sentences in a logical and meaningful order.
It is the nature of an interpreter to be able to have an interactive conversation
as shown above. A compiler needs to be handed the entire program in a file,
and then it runs a process to translate the high-level source code into machine
language and then the compiler puts the resulting machine language into a file for
later execution.
If you have a Windows system, often these executable machine language programs
have a suffix of “.exe” or “.dll” which stand for “executable” and “dynamic link
library” respectively. In Linux and Macintosh, there is no suffix that uniquely
marks a file as executable.
If you were to open an executable file in a text editor, it would look completely
crazy and be unreadable:
ˆ?ELFˆAˆAˆAˆ@ˆ@ˆ@ˆ@ˆ@ˆ@ˆ@ˆ@ˆ@ˆBˆ@ˆCˆ@ˆAˆ@ˆ@ˆ@xa0x82
ˆDˆH4ˆ@ˆ@ˆ@x90ˆ]ˆ@ˆ@ˆ@ˆ@ˆ@ˆ@4ˆ@ ˆ@ˆGˆ@(ˆ@$ˆ@!ˆ@ˆFˆ@
ˆ@ˆ@4ˆ@ˆ@ˆ@4x80ˆDˆH4x80ˆDˆHxe0ˆ@ˆ@ˆ@xe0ˆ@ˆ@ˆ@ˆE
ˆ@ˆ@ˆ@ˆDˆ@ˆ@ˆ@ˆCˆ@ˆ@ˆ@ˆTˆAˆ@ˆ@ˆTx81ˆDˆHˆTx81ˆDˆHˆS
ˆ@ˆ@ˆ@ˆSˆ@ˆ@ˆ@ˆDˆ@ˆ@ˆ@ˆAˆ@ˆ@ˆ@ˆAˆDˆHQVhTx83ˆDˆHxe8
....](/p?url=https%3A%2F%2Fimage.slidesharecdn.com%2Fpythonforinformatics-160323130021%2F85%2FPython-for-informatics-21-320.jpg&__src=https%3A%2F%2Fwww.slideshare.net%2Fslideshow%2Fpython-for-informatics%2F59933486%2369&__type=image)
![1.8. What is a program? 11
1.8 What is a program?
The definition of a program at its most basic is a sequence of Python statements
that have been crafted to do something. Even our simple hello.py script is a pro-
gram. It is a one-line program and is not particularly useful, but in the strictest
definition, it is a Python program.
It might be easiest to understand what a program is by thinking about a problem
that a program might be built to solve, and then looking at a program that would
solve that problem.
Lets say you are doing Social Computing research on Facebook posts and you are
interested in the most frequently used word in a series of posts. You could print out
the stream of Facebook posts and pore over the text looking for the most common
word, but that would take a long time and be very mistake prone. You would be
smart to write a Python program to handle the task quickly and accurately so you
can spend the weekend doing something fun.
For example, look at the following text about a clown and a car. Look at the text
and figure out the most common word and how many times it occurs.
the clown ran after the car and the car ran into the tent
and the tent fell down on the clown and the car
Then imagine that you are doing this task looking at millions of lines of text.
Frankly it would be quicker for you to learn Python and write a Python program
to count the words than it would be to manually scan the words.
The even better news is that I already came up with a simple program to find the
most common word in a text file. I wrote it, tested it, and now I am giving it to
you to use so you can save some time.
name = raw_input('Enter file:')
handle = open(name, 'r')
text = handle.read()
words = text.split()
counts = dict()
for word in words:
counts[word] = counts.get(word,0) + 1
bigcount = None
bigword = None
for word,count in counts.items():
if bigcount is None or count > bigcount:
bigword = word
bigcount = count
print bigword, bigcount
You don’t even need to know Python to use this program. You will need to get
through Chapter 10 of this book to fully understand the awesome Python tech-
niques that were used to make the program. You are the end user, you simply use](/p?url=https%3A%2F%2Fimage.slidesharecdn.com%2Fpythonforinformatics-160323130021%2F85%2FPython-for-informatics-23-320.jpg&__src=https%3A%2F%2Fwww.slideshare.net%2Fslideshow%2Fpython-for-informatics%2F59933486%2369&__type=image)
![46 Chapter 4. Functions
To choose an element from a sequence at random, you can use choice:
>>> t = [1, 2, 3]
>>> random.choice(t)
2
>>> random.choice(t)
3
The random module also provides functions to generate random values from con-
tinuous distributions including Gaussian, exponential, gamma, and a few more.
4.5 Math functions
Python has a math module that provides most of the familiar mathematical func-
tions. Before we can use the module, we have to import it:
>>> import math
This statement creates a module object named math. If you print the module
object, you get some information about it:
>>> print math
<module 'math' from '/usr/lib/python2.5/lib-dynload/math.so'>
The module object contains the functions and variables defined in the module. To
access one of the functions, you have to specify the name of the module and the
name of the function, separated by a dot (also known as a period). This format is
called dot notation.
>>> ratio = signal_power / noise_power
>>> decibels = 10 * math.log10(ratio)
>>> radians = 0.7
>>> height = math.sin(radians)
The first example computes the logarithm base 10 of the signal-to-noise ratio. The
math module also provides a function called log that computes logarithms base e.
The second example finds the sine of radians. The name of the variable is a hint
that sin and the other trigonometric functions (cos, tan, etc.) take arguments in
radians. To convert from degrees to radians, divide by 360 and multiply by 2π:
>>> degrees = 45
>>> radians = degrees / 360.0 * 2 * math.pi
>>> math.sin(radians)
0.707106781187
The expression math.pi gets the variable pi from the math module. The value of
this variable is an approximation of π, accurate to about 15 digits.
If you know your trigonometry, you can check the previous result by comparing it
to the square root of two divided by two:](/p?url=https%3A%2F%2Fimage.slidesharecdn.com%2Fpythonforinformatics-160323130021%2F85%2FPython-for-informatics-58-320.jpg&__src=https%3A%2F%2Fwww.slideshare.net%2Fslideshow%2Fpython-for-informatics%2F59933486%2369&__type=image)
![60 Chapter 5. Iteration
statement to skip to the next iteration without finishing the body of the loop for
the current iteration.
Here is an example of a loop that copies its input until the user types “done”, but
treats lines that start with the hash character as lines not to be printed (kind of like
Python comments).
while True:
line = raw_input('> ')
if line[0] == '#' :
continue
if line == 'done':
break
print line
print 'Done!'
Here is a sample run of this new program with continue added.
> hello there
hello there
> # don't print this
> print this!
print this!
> done
Done!
All the lines are printed except the one that starts with the hash sign because when
the continue is executed, it ends the current iteration and jumps back to the while
statement to start the next iteration, thus skipping the print statement.
5.6 Definite loops using for
Sometimes we want to loop through a set of things such as a list of words, the lines
in a file, or a list of numbers. When we have a list of things to loop through, we
can construct a definite loop using a for statement. We call the while statement
an indefinite loop because it simply loops until some condition becomes False,
whereas the for loop is looping through a known set of items so it runs through
as many iterations as there are items in the set.
The syntax of a for loop is similar to the while loop in that there is a for state-
ment and a loop body:
friends = ['Joseph', 'Glenn', 'Sally']
for friend in friends:
print 'Happy New Year:', friend
print 'Done!'
In Python terms, the variable friends is a list1 of three strings and the for loop
goes through the list and executes the body once for each of the three strings in
the list resulting in this output:
1We will examine lists in more detail in a later chapter.](/p?url=https%3A%2F%2Fimage.slidesharecdn.com%2Fpythonforinformatics-160323130021%2F85%2FPython-for-informatics-72-320.jpg&__src=https%3A%2F%2Fwww.slideshare.net%2Fslideshow%2Fpython-for-informatics%2F59933486%2369&__type=image)
![5.7. Loop patterns 61
Happy New Year: Joseph
Happy New Year: Glenn
Happy New Year: Sally
Done!
Translating this for loop to English is not as direct as the while, but if you think
of friends as a set, it goes like this: “Run the statements in the body of the for loop
once for each friend in the set named friends.”
Looking at the for loop, for and in are reserved Python keywords, and friend
and friends are variables.
for friend in friends:
print ’Happy New Year’, friend
In particular, friend is the iteration variable for the for loop. The variable
friend changes for each iteration of the loop and controls when the for loop
completes. The iteration variable steps successively through the three strings
stored in the friends variable.
5.7 Loop patterns
Often we use a for or while loop to go through a list of items or the contents of
a file and we are looking for something such as the largest or smallest value of the
data we scan through.
These loops are generally constructed by:
• Initializing one or more variables before the loop starts
• Performing some computation on each item in the loop body, possibly
changing the variables in the body of the loop
• Looking at the resulting variables when the loop completes
We will use a list of numbers to demonstrate the concepts and construction of
these loop patterns.
5.7.1 Counting and summing loops
For example, to count the number of items in a list, we would write the following
for loop:
count = 0
for itervar in [3, 41, 12, 9, 74, 15]:
count = count + 1
print 'Count: ', count](/p?url=https%3A%2F%2Fimage.slidesharecdn.com%2Fpythonforinformatics-160323130021%2F85%2FPython-for-informatics-73-320.jpg&__src=https%3A%2F%2Fwww.slideshare.net%2Fslideshow%2Fpython-for-informatics%2F59933486%2369&__type=image)
![62 Chapter 5. Iteration
We set the variable count to zero before the loop starts, then we write a for loop
to run through the list of numbers. Our iteration variable is named itervar and
while we do not use itervar in the loop, it does control the loop and cause the
loop body to be executed once for each of the values in the list.
In the body of the loop, we add 1 to the current value of count for each of the
values in the list. While the loop is executing, the value of count is the number of
values we have seen “so far”.
Once the loop completes, the value of count is the total number of items. The
total number “falls in our lap” at the end of the loop. We construct the loop so that
we have what we want when the loop finishes.
Another similar loop that computes the total of a set of numbers is as follows:
total = 0
for itervar in [3, 41, 12, 9, 74, 15]:
total = total + itervar
print 'Total: ', total
In this loop we do use the iteration variable. Instead of simply adding one to the
count as in the previous loop, we add the actual number (3, 41, 12, etc.) to the
running total during each loop iteration. If you think about the variable total, it
contains the “running total of the values so far”. So before the loop starts total
is zero because we have not yet seen any values, during the loop total is the
running total, and at the end of the loop total is the overall total of all the values
in the list.
As the loop executes, total accumulates the sum of the elements; a variable used
this way is sometimes called an accumulator.
Neither the counting loop nor the summing loop are particularly useful in practice
because there are built-in functions len() and sum() that compute the number of
items in a list and the total of the items in the list respectively.
5.7.2 Maximum and minimum loops
To find the largest value in a list or sequence, we construct the following loop:
largest = None
print 'Before:', largest
for itervar in [3, 41, 12, 9, 74, 15]:
if largest is None or itervar > largest :
largest = itervar
print 'Loop:', itervar, largest
print 'Largest:', largest
When the program executes, the output is as follows:](/p?url=https%3A%2F%2Fimage.slidesharecdn.com%2Fpythonforinformatics-160323130021%2F85%2FPython-for-informatics-74-320.jpg&__src=https%3A%2F%2Fwww.slideshare.net%2Fslideshow%2Fpython-for-informatics%2F59933486%2369&__type=image)
![5.7. Loop patterns 63
Before: None
Loop: 3 3
Loop: 41 41
Loop: 12 41
Loop: 9 41
Loop: 74 74
Loop: 15 74
Largest: 74
The variable largest is best thought of as the “largest value we have seen so far”.
Before the loop, we set largest to the constant None. None is a special constant
value which we can store in a variable to mark the variable as “empty”.
Before the loop starts, the largest value we have seen so far is None since we
have not yet seen any values. While the loop is executing, if largest is None
then we take the first value we see as the largest so far. You can see in the first
iteration when the value of itervar is 3, since largest is None, we immediately
set largest to be 3.
After the first iteration, largest is no longer None, so the second part of the com-
pound logical expression that checks itervar > largest triggers only when we
see a value that is larger than the “largest so far”. When we see a new “even larger”
value we take that new value for largest. You can see in the program output that
largest progresses from 3 to 41 to 74.
At the end of the loop, we have scanned all of the values and the variable largest
now does contain the largest value in the list.
To compute the smallest number, the code is very similar with one small change:
smallest = None
print 'Before:', smallest
for itervar in [3, 41, 12, 9, 74, 15]:
if smallest is None or itervar < smallest:
smallest = itervar
print 'Loop:', itervar, smallest
print 'Smallest:', smallest
Again, smallest is the “smallest so far” before, during, and after the loop exe-
cutes. When the loop has completed, smallest contains the minimum value in
the list.
Again as in counting and summing, the built-in functions max() and min() make
writing these exact loops unnecessary.
The following is a simple version of the Python built-in min() function:
def min(values):
smallest = None
for value in values:
if smallest is None or value < smallest:
smallest = value
return smallest](/p?url=https%3A%2F%2Fimage.slidesharecdn.com%2Fpythonforinformatics-160323130021%2F85%2FPython-for-informatics-75-320.jpg&__src=https%3A%2F%2Fwww.slideshare.net%2Fslideshow%2Fpython-for-informatics%2F59933486%2369&__type=image)
![Chapter 6
Strings
6.1 A string is a sequence
A string is a sequence of characters. You can access the characters one at a time
with the bracket operator:
>>> fruit = 'banana'
>>> letter = fruit[1]
The second statement extracts the character at index position 1 from the fruit
variable and assigns it to the letter variable.
The expression in brackets is called an index. The index indicates which character
in the sequence you want (hence the name).
But you might not get what you expect:
>>> print letter
a
For most people, the first letter of 'banana' is b, not a. But in Python, the index
is an offset from the beginning of the string, and the offset of the first letter is zero.
>>> letter = fruit[0]
>>> print letter
b
So b is the 0th letter (“zero-eth”) of 'banana', a is the 1th letter (“one-eth”), and
n is the 2th (“two-eth”) letter.
b a n n aa
[0] [1] [2] [3] [4] [5]
You can use any expression, including variables and operators, as an index, but the
value of the index has to be an integer. Otherwise you get:
>>> letter = fruit[1.5]
TypeError: string indices must be integers](/p?url=https%3A%2F%2Fimage.slidesharecdn.com%2Fpythonforinformatics-160323130021%2F85%2FPython-for-informatics-79-320.jpg&__src=https%3A%2F%2Fwww.slideshare.net%2Fslideshow%2Fpython-for-informatics%2F59933486%2369&__type=image)
![68 Chapter 6. Strings
6.2 Getting the length of a string using len
len is a built-in function that returns the number of characters in a string:
>>> fruit = 'banana'
>>> len(fruit)
6
To get the last letter of a string, you might be tempted to try something like this:
>>> length = len(fruit)
>>> last = fruit[length]
IndexError: string index out of range
The reason for the IndexError is that there is no letter in ’banana’ with the
index 6. Since we started counting at zero, the six letters are numbered 0 to 5. To
get the last character, you have to subtract 1 from length:
>>> last = fruit[length-1]
>>> print last
a
Alternatively, you can use negative indices, which count backward from the end
of the string. The expression fruit[-1] yields the last letter, fruit[-2] yields
the second to last, and so on.
6.3 Traversal through a string with a loop
A lot of computations involve processing a string one character at a time. Often
they start at the beginning, select each character in turn, do something to it, and
continue until the end. This pattern of processing is called a traversal. One way
to write a traversal is with a while loop:
index = 0
while index < len(fruit):
letter = fruit[index]
print letter
index = index + 1
This loop traverses the string and displays each letter on a line by itself. The
loop condition is index < len(fruit), so when index is equal to the length of
the string, the condition is false, and the body of the loop is not executed. The
last character accessed is the one with the index len(fruit)-1, which is the last
character in the string.
Exercise 6.1 Write a while loop that starts at the last character in the string and
works its way backwards to the first character in the string, printing each letter on
a separate line, except backwards.
Another way to write a traversal is with a for loop:](/p?url=https%3A%2F%2Fimage.slidesharecdn.com%2Fpythonforinformatics-160323130021%2F85%2FPython-for-informatics-80-320.jpg&__src=https%3A%2F%2Fwww.slideshare.net%2Fslideshow%2Fpython-for-informatics%2F59933486%2369&__type=image)
![6.4. String slices 69
for char in fruit:
print char
Each time through the loop, the next character in the string is assigned to the
variable char. The loop continues until no characters are left.
6.4 String slices
A segment of a string is called a slice. Selecting a slice is similar to selecting a
character:
>>> s = 'Monty Python'
>>> print s[0:5]
Monty
>>> print s[6:12]
Python
The operator [n:m] returns the part of the string from the “n-eth” character to the
“m-eth” character, including the first but excluding the last.
If you omit the first index (before the colon), the slice starts at the beginning of
the string. If you omit the second index, the slice goes to the end of the string:
>>> fruit = 'banana'
>>> fruit[:3]
'ban'
>>> fruit[3:]
'ana'
If the first index is greater than or equal to the second the result is an empty string,
represented by two quotation marks:
>>> fruit = 'banana'
>>> fruit[3:3]
''
An empty string contains no characters and has length 0, but other than that, it is
the same as any other string.
Exercise 6.2 Given that fruit is a string, what does fruit[:] mean?
6.5 Strings are immutable
It is tempting to use the [] operator on the left side of an assignment, with the
intention of changing a character in a string. For example:
>>> greeting = 'Hello, world!'
>>> greeting[0] = 'J'
TypeError: object does not support item assignment](/p?url=https%3A%2F%2Fimage.slidesharecdn.com%2Fpythonforinformatics-160323130021%2F85%2FPython-for-informatics-81-320.jpg&__src=https%3A%2F%2Fwww.slideshare.net%2Fslideshow%2Fpython-for-informatics%2F59933486%2369&__type=image)
![70 Chapter 6. Strings
The “object” in this case is the string and the “item” is the character you tried to
assign. For now, an object is the same thing as a value, but we will refine that
definition later. An item is one of the values in a sequence.
The reason for the error is that strings are immutable, which means you can’t
change an existing string. The best you can do is create a new string that is a
variation on the original:
>>> greeting = 'Hello, world!'
>>> new_greeting = 'J' + greeting[1:]
>>> print new_greeting
Jello, world!
This example concatenates a new first letter onto a slice of greeting. It has no
effect on the original string.
6.6 Looping and counting
The following program counts the number of times the letter a appears in a string:
word = 'banana'
count = 0
for letter in word:
if letter == 'a':
count = count + 1
print count
This program demonstrates another pattern of computation called a counter. The
variable count is initialized to 0 and then incremented each time an a is found.
When the loop exits, count contains the result—the total number of a’s.
Exercise 6.3 Encapsulate this code in a function named count, and generalize it
so that it accepts the string and the letter as arguments.
6.7 The in operator
The word in is a boolean operator that takes two strings and returns True if the
first appears as a substring in the second:
>>> 'a' in 'banana'
True
>>> 'seed' in 'banana'
False
6.8 String comparison
The comparison operators work on strings. To see if two strings are equal:](/p?url=https%3A%2F%2Fimage.slidesharecdn.com%2Fpythonforinformatics-160323130021%2F85%2FPython-for-informatics-82-320.jpg&__src=https%3A%2F%2Fwww.slideshare.net%2Fslideshow%2Fpython-for-informatics%2F59933486%2369&__type=image)
![6.9. string methods 71
if word == 'banana':
print 'All right, bananas.'
Other comparison operations are useful for putting words in alphabetical order:
if word < 'banana':
print 'Your word,' + word + ', comes before banana.'
elif word > 'banana':
print 'Your word,' + word + ', comes after banana.'
else:
print 'All right, bananas.'
Python does not handle uppercase and lowercase letters the same way that people
do. All the uppercase letters come before all the lowercase letters, so:
Your word, Pineapple, comes before banana.
A common way to address this problem is to convert strings to a standard format,
such as all lowercase, before performing the comparison. Keep that in mind in
case you have to defend yourself against a man armed with a Pineapple.
6.9 string methods
Strings are an example of Python objects. An object contains both data (the actual
string itself) and methods, which are effectively functions that are built into the
object and are available to any instance of the object.
Python has a function called dir which lists the methods available for an object.
The type function shows the type of an object and the dir function shows the
available methods.
>>> stuff = 'Hello world'
>>> type(stuff)
<type 'str'>
>>> dir(stuff)
['capitalize', 'center', 'count', 'decode', 'encode',
'endswith', 'expandtabs', 'find', 'format', 'index',
'isalnum', 'isalpha', 'isdigit', 'islower', 'isspace',
'istitle', 'isupper', 'join', 'ljust', 'lower', 'lstrip',
'partition', 'replace', 'rfind', 'rindex', 'rjust',
'rpartition', 'rsplit', 'rstrip', 'split', 'splitlines',
'startswith', 'strip', 'swapcase', 'title', 'translate',
'upper', 'zfill']
>>> help(str.capitalize)
Help on method_descriptor:
capitalize(...)
S.capitalize() -> string
Return a copy of the string S with only its first character
capitalized.
>>>](/p?url=https%3A%2F%2Fimage.slidesharecdn.com%2Fpythonforinformatics-160323130021%2F85%2FPython-for-informatics-83-320.jpg&__src=https%3A%2F%2Fwww.slideshare.net%2Fslideshow%2Fpython-for-informatics%2F59933486%2369&__type=image)
![6.10. Parsing strings 73
Some methods such as startswith return boolean values.
>>> line = 'Please have a nice day'
>>> line.startswith('Please')
True
>>> line.startswith('p')
False
You will note that startswith requires case to match, so sometimes we take a line
and map it all to lowercase before we do any checking using the lower method.
>>> line = 'Please have a nice day'
>>> line.startswith('p')
False
>>> line.lower()
'please have a nice day'
>>> line.lower().startswith('p')
True
In the last example, the method lower is called and then we use startswith to
see if the resulting lowercase string starts with the letter “p”. As long as we are
careful with the order, we can make multiple method calls in a single expression.
Exercise 6.4 There is a string method called count that is similar to the function
in the previous exercise. Read the documentation of this method at https://
docs.python.org/2/library/stdtypes.html#string-methods and write an
invocation that counts the number of times the letter a occurs in 'banana'.
6.10 Parsing strings
Often, we want to look into a string and find a substring. For example if we were
presented a series of lines formatted as follows:
From stephen.marquard@ uct.ac.za Sat Jan 5 09:14:16 2008
and we wanted to pull out only the second half of the address (i.e., uct.ac.za)
from each line, we can do this by using the find method and string slicing.
First, we will find the position of the at-sign in the string. Then we will find the
position of the first space after the at-sign. And then we will use string slicing to
extract the portion of the string which we are looking for.
>>> data = 'From stephen.marquard@uct.ac.za Sat Jan 5 09:14:16 2008'
>>> atpos = data.find('@')
>>> print atpos
21
>>> sppos = data.find(' ',atpos)
>>> print sppos
31
>>> host = data[atpos+1:sppos]
>>> print host
uct.ac.za
>>>](/p?url=https%3A%2F%2Fimage.slidesharecdn.com%2Fpythonforinformatics-160323130021%2F85%2FPython-for-informatics-85-320.jpg&__src=https%3A%2F%2Fwww.slideshare.net%2Fslideshow%2Fpython-for-informatics%2F59933486%2369&__type=image)
![6.12. Debugging 75
>>> '%d %d %d' % (1, 2)
TypeError: not enough arguments for format string
>>> '%d' % 'dollars'
TypeError: illegal argument type for built-in operation
In the first example, there aren’t enough elements; in the second, the element is
the wrong type.
The format operator is powerful, but it can be difficult to use. You can read
more about it at https://docs.python.org/2/library/stdtypes.html#
string-formatting.
6.12 Debugging
A skill that you should cultivate as you program is always asking yourself, “What
could go wrong here?” or alternatively, “What crazy thing might our user do to
crash our (seemingly) perfect program?”
For example, look at the program which we used to demonstrate the while loop
in the chapter on iteration:
while True:
line = raw_input('> ')
if line[0] == '#' :
continue
if line == 'done':
break
print line
print 'Done!'
Look what happens when the user enters an empty line of input:
> hello there
hello there
> # don't print this
> print this!
print this!
>
Traceback (most recent call last):
File "copytildone.py", line 3, in <module>
if line[0] == '#' :
The code works fine until it is presented an empty line. Then there is no zero-th
character, so we get a traceback. There are two solutions to this to make line three
“safe” even if the line is empty.
One possibility is to simply use the startswith method which returns False if
the string is empty.
if line.startswith('#') :](/p?url=https%3A%2F%2Fimage.slidesharecdn.com%2Fpythonforinformatics-160323130021%2F85%2FPython-for-informatics-87-320.jpg&__src=https%3A%2F%2Fwww.slideshare.net%2Fslideshow%2Fpython-for-informatics%2F59933486%2369&__type=image)
![76 Chapter 6. Strings
Another way is to safely write the if statement using the guardian pattern and
make sure the second logical expression is evaluated only where there is at least
one character in the string.:
if len(line) > 0 and line[0] == '#' :
6.13 Glossary
counter: A variable used to count something, usually initialized to zero and then
incremented.
empty string: A string with no characters and length 0, represented by two quo-
tation marks.
format operator: An operator, %, that takes a format string and a tuple and gen-
erates a string that includes the elements of the tuple formatted as specified
by the format string.
format sequence: A sequence of characters in a format string, like %d, that spec-
ifies how a value should be formatted.
format string: A string, used with the format operator, that contains format se-
quences.
flag: A boolean variable used to indicate whether a condition is true.
invocation: A statement that calls a method.
immutable: The property of a sequence whose items cannot be assigned.
index: An integer value used to select an item in a sequence, such as a character
in a string.
item: One of the values in a sequence.
method: A function that is associated with an object and called using dot nota-
tion.
object: Something a variable can refer to. For now, you can use “object” and
“value” interchangeably.
search: A pattern of traversal that stops when it finds what it is looking for.
sequence: An ordered set; that is, a set of values where each value is identified
by an integer index.
slice: A part of a string specified by a range of indices.
traverse: To iterate through the items in a sequence, performing a similar opera-
tion on each.](/p?url=https%3A%2F%2Fimage.slidesharecdn.com%2Fpythonforinformatics-160323130021%2F85%2FPython-for-informatics-88-320.jpg&__src=https%3A%2F%2Fwww.slideshare.net%2Fslideshow%2Fpython-for-informatics%2F59933486%2369&__type=image)
![6.14. Exercises 77
6.14 Exercises
Exercise 6.5 Take the following Python code that stores a string:‘
str = 'X-DSPAM-Confidence: 0.8475'
Use find and string slicing to extract the portion of the string after the colon
character and then use the float function to convert the extracted string into a
floating point number.
Exercise 6.6 Read the documentation of the string methods at https://docs.
python.org/2/library/stdtypes.html#string-methods. You might want
to experiment with some of them to make sure you understand how they work.
strip and replace are particularly useful.
The documentation uses a syntax that might be confusing. For example, in
find(sub[, start[, end]]), the brackets indicate optional arguments. So sub
is required, but start is optional, and if you include start, then end is optional.](/p?url=https%3A%2F%2Fimage.slidesharecdn.com%2Fpythonforinformatics-160323130021%2F85%2FPython-for-informatics-89-320.jpg&__src=https%3A%2F%2Fwww.slideshare.net%2Fslideshow%2Fpython-for-informatics%2F59933486%2369&__type=image)
![80 Chapter 7. Files
We will primarily focus on reading and writing text files such as those we create
in a text editor. Later we will see how to work with database files which are binary
files, specifically designed to be read and written through database software.
7.2 Opening files
When we want to read or write a file (say on your hard drive), we first must open
the file. Opening the file communicates with your operating system, which knows
where the data for each file is stored. When you open a file, you are asking the
operating system to find the file by name and make sure the file exists. In this
example, we open the file mbox.txt, which should be stored in the same folder
that you are in when you start Python. You can download this file from www.
py4inf.com/code/mbox.txt
>>> fhand = open('mbox.txt')
>>> print fhand
<open file 'mbox.txt', mode 'r' at 0x1005088b0>
If the open is successful, the operating system returns us a file handle. The file
handle is not the actual data contained in the file, but instead it is a “handle” that
we can use to read the data. You are given a handle if the requested file exists and
you have the proper permissions to read the file.
H
A
N
D
L
E
Your
Program
mbox.txt
open
read
write
close
From stephen.m..
Return-Path: <p..
Date: Sat, 5 Jan ..
To: source@coll..
From: stephen...
Subject: [sakai]...
Details: http:/...
...
If the file does not exist, open will fail with a traceback and you will not get a
handle to access the contents of the file:
>>> fhand = open('stuff.txt')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IOError: [Errno 2] No such file or directory: 'stuff.txt'
Later we will use try and except to deal more gracefully with the situation where
we attempt to open a file that does not exist.](/p?url=https%3A%2F%2Fimage.slidesharecdn.com%2Fpythonforinformatics-160323130021%2F85%2FPython-for-informatics-92-320.jpg&__src=https%3A%2F%2Fwww.slideshare.net%2Fslideshow%2Fpython-for-informatics%2F59933486%2369&__type=image)
![7.3. Text files and lines 81
7.3 Text files and lines
A text file can be thought of as a sequence of lines, much like a Python string can
be thought of as a sequence of characters. For example, this is a sample of a text
file which records mail activity from various individuals in an open source project
development team:
From stephen.marquard@uct.ac.za Sat Jan 5 09:14:16 2008
Return-Path: <postmaster@collab.sakaiproject.org>
Date: Sat, 5 Jan 2008 09:12:18 -0500
To: source@collab.sakaiproject.org
From: stephen.marquard@uct.ac.za
Subject: [sakai] svn commit: r39772 - content/branches/
Details: http://source.sakaiproject.org/viewsvn/?view=rev&rev=39772
...
The entire file of mail interactions is available from www.py4inf.com/code/
mbox.txt and a shortened version of the file is available from www.py4inf.com/
code/mbox-short.txt. These files are in a standard format for a file containing
multiple mail messages. The lines which start with “From ” separate the mes-
sages and the lines which start with “From:” are part of the messages. For more
information about the mbox format, see en.wikipedia.org/wiki/Mbox.
To break the file into lines, there is a special character that represents the “end of
the line” called the newline character.
In Python, we represent the newline character as a backslash-n in string constants.
Even though this looks like two characters, it is actually a single character. When
we look at the variable by entering “stuff” in the interpreter, it shows us the n
in the string, but when we use print to show the string, we see the string broken
into two lines by the newline character.
>>> stuff = 'HellonWorld!'
>>> stuff
'HellonWorld!'
>>> print stuff
Hello
World!
>>> stuff = 'XnY'
>>> print stuff
X
Y
>>> len(stuff)
3
You can also see that the length of the string 'XnY' is three characters because
the newline character is a single character.
So when we look at the lines in a file, we need to imagine that there is a special
invisible character called the newline at the end of each line that marks the end of
the line.
So the newline character separates the characters in the file into lines.](/p?url=https%3A%2F%2Fimage.slidesharecdn.com%2Fpythonforinformatics-160323130021%2F85%2FPython-for-informatics-93-320.jpg&__src=https%3A%2F%2Fwww.slideshare.net%2Fslideshow%2Fpython-for-informatics%2F59933486%2369&__type=image)
![82 Chapter 7. Files
7.4 Reading files
While the file handle does not contain the data for the file, it is quite easy to
construct a for loop to read through and count each of the lines in a file:
fhand = open('mbox.txt')
count = 0
for line in fhand:
count = count + 1
print 'Line Count:', count
python open.py
Line Count: 132045
We can use the file handle as the sequence in our for loop. Our for loop simply
counts the number of lines in the file and prints them out. The rough translation
of the for loop into English is, “for each line in the file represented by the file
handle, add one to the count variable.”
The reason that the open function does not read the entire file is that the file might
be quite large with many gigabytes of data. The open statement takes the same
amount of time regardless of the size of the file. The for loop actually causes the
data to be read from the file.
When the file is read using a for loop in this manner, Python takes care of splitting
the data in the file into separate lines using the newline character. Python reads
each line through the newline and includes the newline as the last character in the
line variable for each iteration of the for loop.
Because the for loop reads the data one line at a time, it can efficiently read and
count the lines in very large files without running out of main memory to store
the data. The above program can count the lines in any size file using very little
memory since each line is read, counted, and then discarded.
If you know the file is relatively small compared to the size of your main memory,
you can read the whole file into one string using the read method on the file
handle.
>>> fhand = open('mbox-short.txt')
>>> inp = fhand.read()
>>> print len(inp)
94626
>>> print inp[:20]
From stephen.marquar
In this example, the entire contents (all 94,626 characters) of the file
mbox-short.txt are read directly into the variable inp. We use string slicing
to print out the first 20 characters of the string data stored in inp.
When the file is read in this manner, all the characters including all of the lines
and newline characters are one big string in the variable inp. Remember that this](/p?url=https%3A%2F%2Fimage.slidesharecdn.com%2Fpythonforinformatics-160323130021%2F85%2FPython-for-informatics-94-320.jpg&__src=https%3A%2F%2Fwww.slideshare.net%2Fslideshow%2Fpython-for-informatics%2F59933486%2369&__type=image)
![7.6. Letting the user choose the file name 85
From: david.horwitz@uct.ac.za
Author: david.horwitz@uct.ac.za
...
7.6 Letting the user choose the file name
We really do not want to have to edit our Python code every time we want to
process a different file. It would be more usable to ask the user to enter the file
name string each time the program runs so they can use our program on different
files without changing the Python code.
This is quite simple to do by reading the file name from the user using raw_input
as follows:
fname = raw_input('Enter the file name: ')
fhand = open(fname)
count = 0
for line in fhand:
if line.startswith('Subject:') :
count = count + 1
print 'There were', count, 'subject lines in', fname
We read the file name from the user and place it in a variable named fname and
open that file. Now we can run the program repeatedly on different files.
python search6.py
Enter the file name: mbox.txt
There were 1797 subject lines in mbox.txt
python search6.py
Enter the file name: mbox-short.txt
There were 27 subject lines in mbox-short.txt
Before peeking at the next section, take a look at the above program and ask
yourself, “What could go possibly wrong here?” or “What might our friendly user
do that would cause our nice little program to ungracefully exit with a traceback,
making us look not-so-cool in the eyes of our users?”
7.7 Using try, except, and open
I told you not to peek. This is your last chance.
What if our user types something that is not a file name?
python search6.py
Enter the file name: missing.txt
Traceback (most recent call last):
File "search6.py", line 2, in <module>
fhand = open(fname)
IOError: [Errno 2] No such file or directory: 'missing.txt'](/p?url=https%3A%2F%2Fimage.slidesharecdn.com%2Fpythonforinformatics-160323130021%2F85%2FPython-for-informatics-97-320.jpg&__src=https%3A%2F%2Fwww.slideshare.net%2Fslideshow%2Fpython-for-informatics%2F59933486%2369&__type=image)
![86 Chapter 7. Files
python search6.py
Enter the file name: na na boo boo
Traceback (most recent call last):
File "search6.py", line 2, in <module>
fhand = open(fname)
IOError: [Errno 2] No such file or directory: 'na na boo boo'
Do not laugh, users will eventually do every possible thing they can do to break
your programs—either on purpose or with malicious intent. As a matter of fact,
an important part of any software development team is a person or group called
Quality Assurance (or QA for short) whose very job it is to do the craziest things
possible in an attempt to break the software that the programmer has created.
The QA team is responsible for finding the flaws in programs before we have
delivered the program to the end users who may be purchasing the software or
paying our salary to write the software. So the QA team is the programmer’s best
friend.
So now that we see the flaw in the program, we can elegantly fix it using the
try/except structure. We need to assume that the open call might fail and add
recovery code when the open fails as follows:
fname = raw_input('Enter the file name: ')
try:
fhand = open(fname)
except:
print 'File cannot be opened:', fname
exit()
count = 0
for line in fhand:
if line.startswith('Subject:') :
count = count + 1
print 'There were', count, 'subject lines in', fname
The exit function terminates the program. It is a function that we call that never
returns. Now when our user (or QA team) types in silliness or bad file names, we
“catch” them and recover gracefully:
python search7.py
Enter the file name: mbox.txt
There were 1797 subject lines in mbox.txt
python search7.py
Enter the file name: na na boo boo
File cannot be opened: na na boo boo
Protecting the open call is a good example of the proper use of try and except in
a Python program. We use the term “Pythonic” when we are doing something the
“Python way”. We might say that the above example is the Pythonic way to open
a file.](/p?url=https%3A%2F%2Fimage.slidesharecdn.com%2Fpythonforinformatics-160323130021%2F85%2FPython-for-informatics-98-320.jpg&__src=https%3A%2F%2Fwww.slideshare.net%2Fslideshow%2Fpython-for-informatics%2F59933486%2369&__type=image)
CR
Uploaded byChristoforos Rekatsinas
1,149 views
Python for informatics
This document provides information about the book "Python for Informatics: Exploring Information". It was originally based on the book "Think Python: How to Think Like a Computer Scientist" by Allen B. Downey but has been significantly revised by Charles Severance to focus on using Python for data analysis and exploring information. The revisions include replacing code examples and exercises with data-oriented problems, reorganizing some topics, and adding new chapters on real-world Python applications like web scraping, APIs, and databases. The goal is to teach useful data skills to students who may not become professional programmers.
