5.
4 Dictionaries
Dictionaries are key-value mappings where keys must be immutable types (like strings,
numbers, tuples) and values can be any type. They are not sequences and do not support
indexing or slicing like lists.
Creating Dictionaries
Empty dictionary:
english_spanish = {}
Add key:value pairs:
english_spanish["one"] = "uno"
english_spanish["two"] = "dos"
Initializing directly with pairs:
english_spanish = {"one": "uno", "two": "dos", "three": "tres"}
Accessing Values
Access via key:
print(english_spanish["two"]) # 'dos'
Keys are used for lookup, not indices.
Dictionary Operations
Remove key with del:
del english_spanish["two"]
Modify or add key:
english_spanish["two"] = "dos"
Increase values:
inventory["bananas"] += 200
Length:len(inventory)
Dictionary Methods
keys(): returns iterable of keys.
values(): returns iterable of values.
items(): returns iterable of (key, value) tuples.
Example:
for key, value in english_spanish.items():
print(key, value)
in checks key presence:
"one" in english_spanish
get(key, default): safely get value or default if key missing.
Aliasing and Copying
Aliasing means multiple variables refer to the same dict.
To avoid unwanted changes, use copy() to make a shallow copy:
copy_dict = original_dict.copy()
Example: Counting Letters
Create frequency table with dictionary and .get() method:
letter_counts = {}
for letter in "Mississippi":
letter_counts[letter] = letter_counts.get(letter, 0) + 1
Sort items alphabetically:
letter_items = list(letter_counts.items())
letter_items.sort()
print(letter_items)
Common Additional Methods
pop(key): remove and return value for key.
popitem(): remove and return last inserted (key, value) pair.
update(other_dict): merge another dictionary into current.
clear(): remove all items.
NumPy
NumPy arrays are designed specifically for mathematical and numerical operations,
addressing the limitations of standard Python lists. Unlike Python lists which are collections
of items without inherent mathematical behavior, NumPy arrays support element-wise
operations directly, such as scalar multiplication and element-wise powers, with automatic
type conversion (e.g., multiplying an integer array by a float results in a float array). This
makes them more elegant and efficient for mathematical computations compared to Python
lists that require explicit looping to perform similar operations.
Key features of NumPy arrays include:
• Element-wise operations like multiplication and exponentiation act on each element
independently.
• Functions like [Link] provide true vector and matrix algebra operations such as dot product.
• Arrays have a shape attribute describing their dimensions, enabling multi-dimensional arrays
(2D, 3D, etc.).
• Powerful indexing and slicing for accessing subsets of data, including multi-dimensional
slicing.
• Masking allows filtering and modifying elements based on conditions without loops.
• Broadcasting enables operations between arrays of different shapes by "stretching" smaller
arrays as needed.
• Data types (dtype) control memory usage and numeric precision, and can be changed with
methods like astype.
• Arrays use less memory and provide faster computation than Python lists due to contiguous
memory storage and optimized, compiled backend.
For example, multiplying a NumPy array by a float scales each element correctly, which is
not possible with a Python list without explicit looping. Additionally, NumPy handles more
complex operations like dot products and matrix multiplications with simple function calls,
while lists cannot do this natively.
Overall, NumPy arrays are far superior to Python lists for mathematical and numerical tasks,
providing both concise, readable code and significant performance improvements in speed
and memory efficiency.
Numpy
The standard Python data types are not very suited for mathematical operations. For example,
suppose we have the
list a = [2, 3, 8]. If we multiply this list by an integer, we get:
>>> a = [2, 3, 8]
>>> 2 * a
[2, 3, 8, 2, 3, 8]
And float’s are not even allowed:
>>> a = [2, 3, 8]
>>> 2 * a
>>> 2.1 * a
TypeError: can't multiply sequence by non-int of type 'float'
In order to solve this using Python lists, we would have to do something like:
values = [2, 3, 8]
result = []
for x in values:
[Link](2.1 * x)
This is not very elegant, is it? This is because Python list’s are not designed as mathematical
objects. Rather, they are purely a collection of items. In order to get a type of list which
behaves like a mathematical array or matrix, we use Numpy.
>>> import numpy as np
>>> a = [Link]([2, 3, 8])
>>> 2.1 * a
array([ 4.2, 6.3, 16.8])
As we can see, this worked the way we expected it to. We note a couple of things: - We
abbreviated numpy to np, this is conventional. - [Link] takes a Python list as argument. -
The list [2, 3, 8] contains int’s, yet the result
contains float’s. This means numpy changed the data type automatically for us.
Now let’s take it a step further and see what happens when we multiply together array’s.
133
>>> import numpy as np
>>> a = [Link]([2, 3, 8])
>>> a * a
array([ 4, 9, 64])
>>> a**2
array([ 4, 9, 64])
This has nicely squared the array element-wise.
Note: Those in the know might be a bit surprised by this. After all, if a is a vector, shouldn’t
a**2
be the dot product of the two vectors, ⃗𝑎 · ⃗𝑎? Well, numpy arrays are not vectors in the
algebraic sense.
Arithmetic operations between arrays are performed element-wise, not on the arrays as a
whole.
To tell numpy we want the dot product we simply use the [Link] function:
>>> a = [Link]([2, 3, 8])
>>> [Link](a,a)
77
Furthermore, if you pass 2D arrays to [Link] it will behave like matrix multiplication. Several
other similar NumPy algebraic functions are available (like [Link], [Link], etc.)
Bottom line: when you want to treat numpy array operations as vector or matrix operations,
make use of the specialized functions to this end.
6.1 Shape
One of the most important properties an array is its shape. We have already seen 1
dimensional (1D) arrays, but arrays can have any dimensions you like. Images for example,
consist of a 2D array of pixels. But in color images every pixel is an RGB tuple: the intensity
in red, green and blue. Every pixel itself is therefore an array as well. This makes
a color image 3D overall.
To get the shape of an array, we use shape:
>>> import numpy as np
>>> a = [Link]([2, 3, 8])
>>> [Link]
(3,)
Something slightly more interesting:
>>> b = [Link]([
[2, 3, 8],
[4, 5, 6],
])
>>> [Link]
(2, 3)
6.2 Slicing
Just like with lists, we might want to select certain values from an array. For 1D arrays it
works just like for normal python lists:
>>> a = [Link]([2, 3, 8])
>>> a[2]
8
>>> a[1:]
[Link]([3, 8])
However, when dealing with higher dimensional arrays something else happens:
>>> b = [Link]([
[2, 3, 8],
[4, 5, 6],
])
>>> b[1]
array([4, 5, 6])
>>> b[1][2]
6
We see that using b[1] returns the 1th row along the first dimenion, which is still an array.
After that, we can select
individual items from that. This can be abbreviated to:
>>> b[1, 2]
6
But what if I wanted the 1th column instead of the first row? Then we use : to select all items
along the first dimension,
and then a 1:
>>> b[:, 1]
array([3, 5])
By comparing with the definition of b, we see that this is the column we were looking for.
Note: Instead of first, I write 1th on purpose to signify the existence of a 0th element.
Remember that in Python, as in any self-respecting programming language, we start counting
at zero. Find out more about advanced slicing at the Numpy indexing documentation page.
6.3 Masking
. Suppose we have an array, and we want to throw away all values above a certain cutoff:
>>> a = [Link]([230, 10, 284, 39, 76])
>>> cutoff = 200
>>> a > cutoff
[Link]([True, False, True, False, False])
Simply using the larger than operator lets us know in which cases the test was positive. Now
we set all the values
above 200 to zero:
>>> a = [Link]([230, 10, 284, 39, 76])
>>> cutoff = 200
>>> a[a > cutoff] = 0
>>> a
[Link]([0, 10, 0, 39, 76])
The crucial line is a[a > cutoff] = 0. This selects all the points in the array where the test was
positive and
assigns 0 to that position. Without knowing this trick we would have had to loop over the
array:
>>> a = [Link]([230, 10, 284, 39, 76])
>>> cutoff = 200
>>> new_a = []
>>> for x in a:
>>> if x > cutoff:
>>> new_a.append(0)
>>> else:
>>> new_a.append(x)
>>> a = [Link](new_a)
6.4 Broadcasting
Another powerful feature of Numpy is broadcasting. Broadcasting takes place when you
perform operations between arrays of different shapes. For instance
>>> a = [Link]([
[0, 1],
[2, 3],
[4, 5],
])
>>> b = [Link]([10, 100])
>>> a * b
array([[ 0, 100],
[ 20, 300],
[ 40, 500]])
The shapes of a and b don’t match. In order to proceed, Numpy will stretch b into a second
dimension, as if it were stacked three times upon itself. The operation then takes place
element-wise. One of the rules of broadcasting is that only dimensions of size 1 can be
stretched (if an array only has one dimension, all other dimensions are considered for
broadcasting purposes to have size 1). In the example above b is 1D, and has shape (2,). For
broadcasting with a, which has two dimensions,
Numpy adds another dimension of size 1 to b. b now has shape (1, 2). This new dimension
can now be stretched three times so that b’s shape matches a’s shape of (3, 2). The other rule
is that dimensions are compared from the last to the first. Any dimensions that do not match
must be stretched to become equally sized. However, according to the previous rule, only
dimensions of size 1 can stretch.
This means that some shapes cannot broadcast and Numpy will give you an error:
>>> c = [Link]([
[0, 1, 2],
[3, 4, 5],
])
>>> b = [Link]([10, 100])
>>> c * b
ValueError: operands could not be broadcast together with shapes (2,3) (2,)
What happens here is that Numpy, again, adds a dimension to b, making it of shape (1, 2).
The sizes of the last dimensions of b and c (2 and 3, respectively) are then compared and
found to differ. Since none of these dimensions
is of size 1 (therefore, unstretchable) Numpy gives up and produces an error. The solution to
multiplying c and b above is to specifically tell Numpy that it must add that extra dimension
as the second dimension of b. This is done by using None to index that second dimension.
The shape of b then becomes (2, 1), which is compatible for broadcasting with c:
>>> c = [Link]([
[0, 1, 2],
[3, 4, 5],
])
>>> b = [Link]([10, 100])
>>> c * b[:, None]
array([[ 0, 10, 20],
[300, 400, 500]])
A good visual description of these rules, together with some advanced broadcasting
applications can be found in this
tutorial of Numpy broadcasting rules.
6.5 dtype
A commonly used term in working with numpy is dtype - short for data type. This is typically
int or float, followed by some number, e.g. int8. This means the value is integer with a size of
8 bits. As an example, let’s discuss the properties of an int 8.
Each bit is either 0 or 1. With 8 of them, we have 28 = 256 possible values. Since we also
have to count zero itself, the largest possible value is 255. The data type we have now
described is called uint8, where the u stands for unsigned: only positive values are allowed. If
we want to allow negative numbers we use int8. The range then shifts to -128 to +127.
The same holds for bigger numbers. An int64 for example is a 64 bit unsigned integer with a
range of - 9223372036854775808 to 9223372036854775807.
It is also the standard type on a 64 bits machine. You might think bigger is better. You’d be
wrong. If you know the elements of your array are never going to be bigger than 100, why
waste all the memory space? You might be better off setting your array to uint8 to conserve
memory. In general however, the default setting is fine. Only when you run into memory
related problems should you remember this comment.
What happens when you set numbers bigger than the maximum value of your dtype?
>>> import numpy as np
>>> a = [Link]([200], dtype='uint8')
>>> a + a
array([144], dtype=uint8)
That doesn’t seem right, does it? If you add two uint8, the result of 200 + 200 cannot be 400,
because that doesn’t
fit in a uint8. In standard Python, Python does a lot of magic in the background to make sure
the result is the 400
you would expect. But numpy doesn’t, and will return 144. Why 144 is left as an exercise. To
fix this, you should
make sure that your numbers where not stored as uint8, but as something larger; uint16 for
example. That way
the resulting 400 will fit.
>>> import numpy as np
>>> a = [Link]([200], dtype='uint16')
>>> a + a
array([400], dtype=uint16)
6.5. dtype
when you load an image from your hard drive this dtype is selected for you, and if you are
not aware of this, what will happen when you add an image to itself? (In other words, place
two copies on top of each other) You might expect that everything will become more dense.
Instead,
6.6 Changing dtype
To change the dtype of an existing array, you can use the astype method:
>>> import numpy as np
>>> a = [Link]([200], dtype='uint8')
>>> [Link]('uint64')
Files:
7.1 About Files
Data during a program’s execution is stored in volatile RAM, which loses data once the
program ends or the computer shuts down.
To preserve data, it must be saved to non-volatile storage such as a hard drive, USB, or CD-
RW in locations called files.
Files must be opened to read or write; opening creates a file handle representing the file.
The file handle controls access, like a TV remote controls a TV. Files should be closed after
use, which is handled automatically by Python’s with block.
7.2 Writing Our First File
with open("[Link]", "w") as myfile:
[Link]("My first file written from Python\n")
[Link]("---------------------------------\n")
[Link]("Hello, world!\n")
open() with mode "w" opens for writing, creating the file if it doesn’t exist or replacing it if
it does.
write() writes text strings to the file.
Files are closed automatically at the end of the with block, even if errors occur.
7.3 Reading a File Line-at-a-Time
with open("[Link]", "r") as my_new_handle:
for the_line in my_new_handle:
print(the_line, end="")
Mode "r" opens the file for reading.
Looping over the file yields lines including newline characters, handled
by print(end="") to avoid extra lines.
Trying to open a non-existent file in "r" mode raises an error.
7.4 Turning a File into a List of Lines
with open("[Link]", "r") as input_file:
all_lines = input_file.readlines()
all_lines.sort()
with open("[Link]", "w") as output_file:
for line in all_lines:
output_file.write(line)
readlines() reads all lines into a list of strings to allow sorting or other list operations.
The sorted content is then written to a new file.
7.5 Reading the Whole File at Once
with open("[Link]") as f:
content = [Link]()
words = [Link]()
print("There are {0} words in the file.".format(len(words)))
read() reads the entire file content as one string.
String operations like split() can then be applied for processing.
7.6 Filtering Lines While Copying
def filter(oldfile, newfile):
with open(oldfile, "r") as infile, open(newfile, "w") as outfile:
for line in infile:
if not [Link]('#'):
[Link](line)
Opens two files simultaneously for reading and writing.
Copies lines from the input to output only if they don’t start with #.
7.7 Directories
Files are organized into directories (folders).
File paths specify directory locations, can be relative or absolute.
Use Python’s [Link] to combine directory and filename to handle operating system-
specific path separators [Link] path on Unix: /usr/share/dict/words
On Windows: c:\\temp\\[Link]
7.8 Fetching from the Web Using urllib
import [Link]
url = "[Link]
destination_filename = "[Link]"
[Link](url, destination_filename)
This downloads a web resource to a local file in one call.
7.8 Fetching from the Web Using requests
import requests
url = "[Link]
response = [Link](url)
print([Link])
Reads web content directly into a string for further processing without saving to disk.