More Python Concepts - Learning Statistics With Python
More Python Concepts - Learning Statistics With Python
Contents
4.1. Using comments
4.2. Installing and importing
Print to
4.3. Listing PDF
the objects in active memory
4.4. Loading and saving data
4.5. Useful things to know about variables
4.6. Variable types
4.7. Regular variables and collection variables
4.8. Regular variables
4.9. Arguments
4.10. Collection variables
4.11. Tuples
4.12. Dictionaries
4.13. Combining variable types
4.14. Sets
4.15. Object methods
4.16. Dynamic typing
4.17. Indexing and slicing
4.18. Slices
4.19. Summary
– Louis Sullivan
In the chapter on Getting started with Python our main goal was to, well, get started with
Skipwe’ll
Python. As we go through the book to main
runcontent
into a lot of new Python concepts, which
I’ll explain alongside the relevant data analysis concepts. However, there’s still quite a
few things that I need to talk about now, otherwise we’ll run into problems when we
start trying to work with data and do statistics. So that’s the goal in this chapter: to build
on the introductory stuff from the last chapter, to get you to the point that we can start
using Python for statistics. Broadly speaking, the chapter comes in two parts. The first
half of the chapter is devoted to the “mechanics” of Python: installing and loading
packages, managing the workspace, navigating the file system, and loading and saving
data. In the second half, I’ll talk more about what kinds of variables exist in Python, and
introduce three new kinds of variables: factors, data frames and formulas. I’ll finish up
by talking a little bit about the help documentation in Python as well as some other
avenues for finding assistance. In general, I’m not trying to be comprehensive in this
chapter, I’m trying to make sure that you’ve got the basic foundations needed to
tackle the content that comes later in the book. However, a lot of the topics are revisited
in more detail later.
8.539539450000001
it’s a lot easier to understand what I’m doing than if I just write this:
seeker = 3.1415
lover = 2.7183
keeper = seeker * lover
print(keeper)
From now on, you’ll start seeing # characters appearing in the code extracts, with
some human-readable explanatory remarks next to them. These are still perfectly
legitimate commands, since Python knows that it should ignore the # character and
everything after it. But hopefully they’ll help make things a little easier to understand.
This two step process might seem a little odd at first, but the designers of Python had
very good reasons to do it this way,[3] and you get the hang of it pretty quickly.
I won’t get into the details of installing libraries here, simply because it is too much for
me to tackle. If you are using Python in an online enviroment, you may already have
access to all the libraries mentioned in this book. If you are working with Python on your
own computer, the exact details of how you install packages may vary. If you want to use
Python on your own computer, and are just getting started, I recommend Anaconda as a
relatively easy way to install Python and get quick access to all the most common and
important libraries.
numbers = [4, 5, 1, 2, 6]
sum(numbers)
18
because the authors of Python felt that adding numbers together was such a basic thing
that there should be a built-in command for it. At least, I assume so. I don’t really know
what the authors of Python thought. But, oddly enough, Python doesn’t have a built-in
command called “mean”. So if I want to know the mean of those same five numbers, I
cannot just write
mean(numbers)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In[4], line 1
----> 1 mean(numbers) Skip to main content
NameError: name 'mean' is not defined
because Python doesn’t know what mean , er, means. Luckily, we don’t have to resort
to first finding the sum and then dividing by the number of numbers, because there are
libraries that do have built-in commands for finding means. The statistics library is
one. To use the commands in this library, we first have to import it. This gives us access
to all the many useful commands in the statistics library, one of which is mean :
import statistics
numbers = [4, 5, 1, 2, 6]
[Link](numbers)
3.6
You probably noticed the . in the code above. This is the way we tell Python that we
want to use a command called mean which is found inside the library statistics .
Without the . , even though we have imported statistics , which has a command
called mean , we still can’t just write mean(numbers) . We have to tell Python where to
look for this command. This all seems very cumbersome, but it’s really not so bad, there
are good reasons for doing it this way[5], and you will get used to it fairly quickly.
One of the ways in which Python is quite flexible is that it gives you some options in terms
of how you import libraries. More precisely, you can:
Let’s say we don’t want to import the entire statistics library — we only want the
mean command. We can achieve this like this:
Why would we want to do this? Well, one good reason is that now we can simply write
mean(numbers) ; we no longer have to write out [Link](numbers) :
3.6
Is this the height of laziness? Maybe. But if you start writing the same thing over and over
again, saving a few characters here and there is pretty sweet. And this brings us to the
other import option: renaming libraries. It is common practice in Python to give libraries
abbreviations when we import them. Many of the most common libraries have
conventional abbreviations, although you could use anything you like. Thus, you will often
see e.g.
import numpy as np
import seaborn as sns
This is very convenient, but be careful: if you e.g. import numpy as np , then Python will
only recognize it as np , at least for the time your code is in Python’s active memory.
Also, although you can use whatever abbreviations you like, I highly recommend sticking
to the conventional ones, for your sake and for the sake of others trying to read your
code. It’s kind of fun the first time to do something like
why_you_gotta_be_so.mean(numbers)
3.6
but good code should be easy to read by yourself and others, and if you start playing too
fast and loose with renaming, it starts to get less clear what’s going on.
An important part of learning to program is to develop the ability to keep a mental model
of what Python knows and doesn’t know at any given time active in your mind. This
sounds very abstract, and it is, but as you become more familar with coding I think you
will see what I mean. I won’t dwell on this here, but it may be useful to take a quick peak
at what I mean. If you are working in e.g. a Jupyter Notebook (and I do suggest you do
this, at least at first), then by typing %who you can see a list of all the variables that Python
is currently aware of. So, in my case, I get the following:
%who
Here we can see variables that we defined, like keeper and lover , and also libraries
that we imported (and renamed), like np and sns , as well as the library statistics
which I then ill-advisedly re-imported and renamed why_you_gotta_be_so . To see more
details on these variables, we can type %whos
%whos
This tells us that e.g. keeper is a floating-point decimal number with the value
8.539539450000001, and shows us the true names of the objects we have renamed on
import. These commands that start with a % sign, by the way, are called “magic”
commands, and can only be used in environments
Skip like Jupyter, which support them. If
to main content
you are not working in such an environment, you can use the command dir() , which
achieves the same thing, but will also show you lots of information you probably aren’t
interested in at this stage.
del(keeper, lover)
%who
If you want to remove all the variables in memory, and you are working in a Jupyter
environment, then %reset is a handy way to do this, although I must say that in practice I
rarely, if ever, have a need to remove variables from memory. There is usually no harm in
them sitting around unused, and if you define a new variable with the same name as an
old one, it will just write over the old variable with the same name.
Comma separated value (CSV) files are those with a .csv file extension. These are
just regular old text files, and they can be opened with almost any software. It’s
quite typical for people to store data in CSV files, precisely because they’re so
simple.
Script files are those with a .py file extension or an .ipynb extension. These aren’t
data files at all; rather, they’re used to save a collection of commands that you want
Python to execute later. They’re just text files, but we won’t make use of them until
later.
In this section I’ll talk about how to import data from a CSV file. First though, we need to
make a quick digression, and talk about file systems. I know this is not a very exciting
topic, but it is absolutely critical to doing data analysis. If you want to work with your data
in Python, you need to be able to tell Python where the data is located.
On a Windows computer the root is the physical drive [6] on which the file is stored, and
for most home computers the name of the hard drive that stores all your files is C: and
therefore most file paths on Windows begin with C:. After that come the folders, and on
Windows the folder names are separated by a \ symbol. So, the complete path to this
book on my Windows computer might be something like this:
C:\Users\ethan\pythonbook\book\[Link]
and what that means is that the book is called [Link], and it’s in a folder called book
which itself is in a folder called ethan which itself is … well, you get the idea. On Linux,
Unix and Mac OS systems, the addresses look a little different, but they’re more or less
identical in spirit. Instead of using the backslash, folders are separated using a forward
Skip to main content
slash, and unlike Windows, they don’t treat the physical drive as being the root of the file
system. So, the path to this book on my Mac might be something like this:
/Users/ethan/pythonbook/book/[Link]
So that’s what we mean by the “path” to a file, and before we move on, it is critical
that you learn how to copy the path to a file on your computer so that you can paste it
into Python. There are (again!) multiple ways to do this on the various operating systems,
and it doesn’t really matter which method you use. A quick search will lead you to many
many online tutorials; just find a method that works for you, on your computer. Even if
you are working in a virtual Python environment in the browser (such as Google Colab),
you will need some concept of file structure to navigate to your data and scripts.
As is often the case, there are many different ways to get the data from a CSV file into
Skipthings
Python so that you can begin doing to main content
with it. Here we will use the pandas library,
which happens to have a handy command called read_csv() which does just what it
says.
We can’t just read the data in willy-nilly, though. We need some place to put it. As you
may have already guessed, we need to define a variable to put our data into. We
haven’t talked about variable types yet, and now is not the time, but let it suffice to say
that there are different kinds of variables, and some of them can store structured data like
the rows and columns in a CSV file. pandas calls this kind of variable a “dataframe”.
You can name your dataframe whatever you like, of course, but by convention they are
often called “df”, so we’ll do that too. Thus:
import pandas as pd
# make a new dataframe variable, and use the "read_csv" command from the pandas
# of the file located at "/Users/ethan/Documents/GitHub/pythonbook/Data/booksal
df = pd.read_csv("/Users/ethan/Documents/GitHub/pythonbook/Data/[Link]"
Here I have put the full path into the parentheses following pd.read_csv , but often I
prefer to save the path as a variable, and put that variable into the parentheses instead,
like this:
import pandas as pd
file = "/Users/ethan/Documents/GitHub/pythonbook/Data/[Link]"
df = pd.read_csv(file)
Either way works, but I think this looks nicer, and it also has an additional advantage: it
makes the code more versatile. Right now we are just loading a single CSV, but if we
want to load many CSV files, it might be useful to write a loop that puts different paths
into the same variable file . But that is a discussion for another time.
df.to_csv('/Users/ethan/Desktop/my_file.csv')
Every pandas dataframe has the built-in ability to be exported as a CSV file. We just
need to tell Python what the new file should be called, and where we want it to go in our
filesystem. Pretty straightforward, really.
x = 1 # a is a number
y = 2 # b is a number
z = x + y
print(z)
If they contain character data (strings), Python will still let you add the variables, but the
outcome might be unexpected:
'12'
To us, there isn’t really that big a difference between 1 and “1”, but to Python, these
are entirely different classes of things.
type(x)
str
import pandas as pd
file = "/Users/ethan/Documents/GitHub/pythonbook/Data/[Link]"
x = pd.read_csv(file) # x is a dataframe
type(x)
[Link]
int
float
Exciting, no? Trust me, you’ll need to do this more often than you might think.
Since Python packages can add their own variable types (like the
[Link] variable we saw above), there can be quite a few
different variable types in Python. But it is worth spending a little time getting to know the
standard variable types in Python, so here I’ll introduce you to a few of the usual
suspects: integers, floats, strings, dictionaries, lists, and sets. These variable types are like
the basic building blocks in Python, and once you become familar with them and get to
know their quirks, foibles, and properties, everything else in Python becomes so much
easier.
1. Integers
2. Floats
3. Strings
As I mentioned, there are also other kinds of collection variables, such as pandas
dataframes provide more structure and allow for more complex, organized data storage
and manipulation. But we’ll get to these in due time, as needed. For now, I want to
introduce you to the basic set of variables in Python.
4.8.1. Intergers
We have already spent some time with one of Python’s most common variable types,
the interger. Integers are pretty straightforward: they are whole numbers, like 2, 7, 4392,
0, and -34. There’s not so much more to say about them. You can do all the usual sorts
of things with integers that you might expect: you can add them, subtract them, multiply
them, etc. To give a some quick examples:
cows = 75
pigs = 21
hens = 7
4.8.2. Floats
On their face, floats are also pretty straightforward: they’re decimal fractions, like 0.234,
0.333, or -0.000003. Most of the time, this level of understanding is all you need.
Occasionally, though, you will run into some confusing behavior with floats, so it is worth
Skip to main content
saying just a few more words about them. There are two issues with floats which we
should at least be aware of:
1. Computers store decimal fractions as base 2 fractions (binary), not base 10, the way
we are used to thinking of them.
2. There are more than one way to round fractions, and Python may not round fractions
the way you expect.
a = 1/10
print(a)
0.1
a = 1/3
print(a)
0.3333333333333333
Nothing odd here. In base 10 math, 1 can be equally divided into ten parts, and the answer
is 0.1
But to our computers, the answer to 1 divided by 10 is stored as a float variable, and
stored in base 2. Which means, that to Python 1 divided by 10 is actually 0.0001 1001 1001
1001 1001 1001 1001 1001 1001 1001 1001 1001 1001 1001 1001 1001 1001 1001 1001 1001 1001
1001 1001 1001 1001 1001….
That is, to the computer, 1/10 is actually an infinitely repeating series, just like the answer
to 1/3 is in the base 10 math we are used to. So when Python shows us that 1/10 = 0.1, it is
really just rounding up, so that the answer looks like what we expect to see. This means
that decimal fractions are not as precise as we might think, and explains why the answer
Python gives us when we add decimals together might be surprising:
.1 + .1 + .1
This answer makes no sense if we think that 0.1 is exactly 0.1, but makes more sense if
we understand that to Python, 0.1 is just an approximation of the answer to 1/10. Now,
most of the time this is not going to matter much, but it could make a difference, so it is
good to be aware of it.
Now, here’s the other thing to be aware of with floats in Python. Often when we have a
number with a decimal, we want to round it. So rather than 1.5, we might want to round to
a whole number:
round(1.5)
We musn’t get off an any further tangents here, because most of the time we don’t
have to think about these bits of Skip to main
esoterica, butcontent
I will mention just one last oddity about
floats that you might conceivably run into at some point. Let’s say we add the following
two numbers:
1 + 1.000000000000001
2.000000000000001
1 + 1.0000000000000001
2.0
Ugh. Really? Yup. If your computer has a 64-bit processor (and these days, it probaby
does), there is a hard limit on the number of decimal places it can store. If you like, play
around with it yourself, and see where these limits lie. Just for “fun”, here are a few last
examples:
print(1 + 1.0000000000000001)
print(1 + 1.000000000000001)
print(1 + 1.0000000000000009)
2.0
2.000000000000001
2.000000000000001
Alright. Enough floats. Mostly they work the way you would expect, but if they don’t,
now you know some of the possible reasons. Time to move on to something more fun:
strings!
4.8.3. Strings
Strings are variables that store characters, like “g”, “h”, “a”, and “p”, but also
“cat”, “apple”, and “supercalifragilisticexpialidocious”. Strings can also be things
like “$”, “,”, “;” and “?”.Skip
Somewhat
to mainconfusingly,
content strings can also be things like
“3”, “42”, and “-8374.87”. Maybe it is helpful to think of strings as being one of the
roles these symbols can play, just like a book could be a gift, a paperweight, a doorstop,
or a weapon. Ok, maybe that’s not so helpful after all. Let’s try some examples
instead:
a = "I am a string"
print(a)
I am a string
a = "I am a "
b = "string"
c = a + b
print(c)
I am a string
12
So we can add strings together, but the result is different than when we add integers
together. To Python, the numbers in the cell above are just the characters 1 and 2, not the
values 1 and 2. So it just sticks them together, just like it would any other strings, if you
add them.
Skip to main content
Now, while we can add two strings together, we can’t multiply two strings by each
other, or subtract, or divide. What would that even mean. We can multiply strings by an
interger, though:
a = 'cat'
print(a*5)
catcatcatcatcat
There are many more fun things we can do with strings; many more than I will go into
here. I will give a few examples, though, just to whet your appetite:
[Link]()
[Link]()
['a', 'long', 'time', 'ago', 'in', 'a', 'galaxy', 'far', 'far', 'away']
[Link]('a')
['', ' long time ', 'go in ', ' g', 'l', 'xy f', 'r f', 'r ', 'w', 'y']
'X'.join(a)
[Link]('galaxy', 'suburb')
[Link]('a', 'A')
'a_long_time_ago_in_a_galaxy_far_far_away'
Python is a fantastic tool for working with strings, which makes it the language of choice
for very many people working with text data of any sort. But now it is time to move on to
collection variables.
4.9. Arguments
This might be as good a place as any to introduce the concept of arguments. Arguments
in Python are not heated discussions about whether or not Die Hard is a Christmas
movie[8], or how to pronounce the first consonant in “gif”[9]. Arguments in Python are
opportunities for us to give Python functions some information. As an example, above I
used the the replace method. Let’s try a different example:
[Link]('a', '@')
[Link]()
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Input In [126], in <cell line: 3>()
1 a = 'a long time ago in a galaxy far far away'
----> 3 [Link]()
Because, as the error message says, replace expects us to at a minimum tell it what to
replace, and what to replace it with. That seems fair enough, actually. How else is it
supposed to know, if we don’t tell it? In fact, replace has another, optional, argument,
“count”. If we use this third arguement, we can also tell replace how many instances
of the old thing we want to replace with the new. The default value is all of them, but if we
wanted, we could e.g. only replace the first two instances of ‘a’ with ‘@’:
[Link]('a', '@', 2)
How did I know that replace has three arguments? How will you know how many
arguments other methods and functions have? Well, I knew because I searched the
internet for “python string replace method arguments”, and that’s my advice to you,
too. Until just now, I had no idea that there was a third argument available for replace ,
because I had never needed it for anything. So I just learned something too!
print(shopping)
print(scores)
print(mixed)
print(combined)
Let’s say I am so enamored with Python that I actually decided to keep my shopping list
in a Python list. Seems unlikely, I know, but bear with me. Later, I realize I have forgotten
what I wrote on the list. This does kind of sound like me, actually. To see the contents of
the entire list, I can use print() , the way I did above. But let’s say I only want to see
the second item on the list. Python has a way to access specific items in lists, but it will
seem strange at first!
Let’s take a look. To access an item in a list, we need to know its index , that is, its
location in the list. We indicate an index with square brackets. In Python, this is known as
slicing . So to find the item with the index 2 in my shopping list, I can write
shopping[2] , like so:
'bananas'
The simple answer for this is that Python uses “zero-based indexing”: basically, Python
starts counting at zero. So “apples” is in the zeroeth position, “pears” is in the first
position, and “bananas” is in the second position. If it helps, maybe think of it like
buildings in Europe that start with the ground floor, and then go up to the first floor, etc.
Now just when you have started to get used to zero-indexing, try negative indexing on for
size. We can also count backwards from the end of the list, by using negative indices
such as shopping[-2] But be careful: when you use negative indexing, Python behaves
the way you might have originally expected it to. Thus, shopping[-1] will return
“bananas”, shopping[-2] will give us “pears”, etc. That’s just how Python is.
# +---+---+---+---+---+---+
# | P | y | t | h | o | n |
# +---+---+---+---+---+---+
# 0 1 2 3 4 5 6
# -6 -5 -4 -3 -2 -1
len(shopping)
While Python uses square brackets ( [ ] ) to indicate lists, it uses regular old parentheses
plus a comma to indicate tuples. Let’s take a look:
We can still get data out of tuples in the same way we get it out of lists, by slicing them,
using square brackets and an index value:
print(shopping_tuple[0])
apples
Now, if I decide I don’t want apples after all, but want grapes instead, I can just re-
assign the new value to the appropriate position in the list, replacing “apples” with
“grapes”
shopping_list[0] = 'grapes'
print(shopping_list)
Don’t try that kind of funny business with a tuple, though. It won’t have any of it:
shopping_tuple[0] = 'grapes'
print(shopping_tuple)
One more detail, before we leave tuples. Remember I said that Python uses parentheses
and a comma to indicate that something is a tuple? I meant it. If we try to make a tuple
that only has one item, you might think we could do it like this:
a = ('pears')
type(a)
str
But, as you can see, Python does not think this is a tuple. It just sees it as a string. To
make a tuple with only item, we still need the comma:
a = ('pears',)
type(a)
tuple
print(a[0])
pears
4.12. Dictionaries
Dictionaries are a very useful variable type in Python, but it may take you a while to
appreciate their usefulness. Trust me, though. Once you begin to appreciate what
Skip to main content
dictionaries can do, you will learn to love them. But, like anything (or anyone), before you
can truly love dictionaries, you have to get to know them.
In essence, dictionaries are very simple. They consist of two parts: keys and values. You
can think of the key as the word you look up in the dictionary, and the value as the
definition. The syntax for defining a dictionary is {key: value} . A simple key-value pair
could be {'name': 'Ethan'} .
me = {'name': 'Ethan'}
print(me['name'])
Ethan
Of course, most dictionaries will have more than one entry. If were to make a dictionary
with information about my pet, for instance, I might write:
Now, if I can’t remember what kind of animal my pet is, I can always consult my
dictionary:
my_pet['species']
'cat'
Dictionary values can be more than just a single string, though. Let’s say I want to plan
out my week using Python (because that would be a normal thing to do!) I can make a
dictionary called my_week , and include entries for shopping and a to-do list.
print(my_week['todo'])
Notice that I have made the value for shopping a list, but the value for todo is a tuple.
That might be because I want to keep open the option to change items in my shopping
list, but want to keep my to-do list fixed. Or it might just be because I wanted to
demonstrate that dictionary values can be all kinds of different variable types.
[['Barred Owl', 'Burrowing Owl', 'Barn Owl', 'Screech Owl', 'Spotted Owl'], ['G
Now, this one is a collection of strange bedfellows, but Python isn’t judgy. So, in
mishmosh , I have two keys, birds and dictionaries . Why not? If we look in the
dictionary, we can see that the value for the key birds is a list of lists:
mishmosh['birds']
[['Barred Owl', 'Burrowing Owl', 'Barn Owl', 'Screech Owl', 'Spotted Owl'],
['Goldfinch', 'Crossbill', 'Redpoll', 'Grosbeak'],
['House Sparrow',
'Song Sparrow',
'White-Throated Sparrow',
'Captain Jack Sparrow']]
mishmosh['dictionaries']
We’ll leave it there for now, but hopefully you can see that by using these basic
elements: regular variables and collection variables, you can store data in a structured
way, so that you can access it when you need it. There is one last type of collection
variable that will come in handy, and that is the set. Sets are the collection of unique items
in a collection of items. As an example, say every student in a class wrote down their
favorite color, and we put them all into a list, like this:
4.14. Sets
Skip to main content
our_colors = ['green', 'blue', 'red', 'blue', 'orange', 'yellow', 'green', 'blu
Since every student wrote down a color, if we find the length of the list, we will also find
the number of students in the class:
len(our_colors)
10
But some students wrote the same colors as other students. So if we want to find the
unique colors chosen, ignoring in repeated colors, we can find the set of our list:
set(our_colors)
And by finding the length of the set, we can see how many unique colors were chosen:
len(set(our_colors))
As another example, we could make a see how many unique characters make up a
sentence:
24
{'r', 'w', 'v', ' ', 'i', 'y', 'P', 'a', 'l', 'u', 'd', 'm', 'n', 'h', 'e', 'I'
Notice that not only does Python count the blank space and the exclamation mark as
characters, it also counts the upper and lower case I/i as separate characters. Just to
keep you on your toes.
Here, we created a string of words separated by spaces, all lowercase, and called it a .
Then, by writing the name of the variable, a , followed by .title() , Python knew to
convert the string to title case. We can also convert our variable a , now in title case, to
upper case, by a similar method:
a = [Link]()
print(a)
title() and upper() are examples of special properties that all string variables
Skip
possess. These special properties aretocalled
main“methods”.
content To see the full list of methods
available for a variable in Python, we can use the function dir() . So, for example, since
a is a string, we can see what methods are available for strings by writing dir(a) :
dir(a)
['__add__',
'__class__',
'__contains__',
'__delattr__',
'__dir__',
'__doc__',
'__eq__',
'__format__',
'__ge__',
'__getattribute__',
'__getitem__',
'__getnewargs__',
'__gt__',
'__hash__',
'__init__',
'__init_subclass__',
'__iter__',
'__le__',
'__len__',
'__lt__',
'__mod__',
'__mul__',
'__ne__',
'__new__',
'__reduce__',
'__reduce_ex__',
'__repr__',
'__rmod__',
'__rmul__',
'__setattr__',
'__sizeof__',
'__str__',
'__subclasshook__',
'capitalize',
'casefold',
'center',
'count',
'encode',
'endswith',
'expandtabs',
'find',
'format',
'format_map',
'index',
'isalnum',
'isalpha',
Skip to main content
'isascii',
'isdecimal',
'isdigit',
'isidentifier',
'islower',
'isnumeric',
'isprintable',
'isspace',
'istitle',
'isupper',
'join',
'ljust',
'lower',
'lstrip',
'maketrans',
'partition',
'removeprefix',
'removesuffix',
'replace',
'rfind',
'rindex',
'rjust',
'rpartition',
'rsplit',
'rstrip',
'split',
'splitlines',
'startswith',
'strip',
'swapcase',
'title',
'translate',
'upper',
'zfill']
Yikes! You can safely ignore all the things with __ in front of and after them, like
__add__ , and __class__ . These aren’t meant for us to meddle with! But even setting
these aside, you can see that for strings the list is quite long, and to be honest with you, I
don’t even know what all of these do. Lists, on the other hand, don’t have as many
methods available to them:
b = [3, 6, 1231]
dir(b)
['__add__',
'__class__',
'__class_getitem__',
'__contains__',
'__delattr__',
'__delitem__', Skip to main content
'__dir__',
'__doc__',
'__eq__',
'__format__',
'__ge__',
'__getattribute__',
'__getitem__',
'__gt__',
'__hash__',
'__iadd__',
'__imul__',
'__init__',
'__init_subclass__',
'__iter__',
'__le__',
'__len__',
'__lt__',
'__mul__',
'__ne__',
'__new__',
'__reduce__',
'__reduce_ex__',
'__repr__',
'__reversed__',
'__rmul__',
'__setattr__',
'__setitem__',
'__sizeof__',
'__str__',
'__subclasshook__',
'append',
'clear',
'copy',
'count',
'extend',
'index',
'insert',
'pop',
'remove',
'reverse',
'sort']
We can access these special methods by writing the name of the variable, and then a .
and then the name of the method. But we can only use the methods that are available for
the type of variable we are working with. Lists, for instance, do not have a title
method, so it’s no good taking our list b and writing
[Link]()
b = 342
type(b)
int
b = 'zebrafish'
type(b)
str
owls = ['Barred Owl', 'Burrowing Owl', 'Barn Owl', 'Screech Owl', 'Spotted Owl
finches = ['Goldfinch', 'Crossbill', 'Redpoll', 'Grosbeak']
sparrows = ['House Sparrow', 'Song Sparrow', 'White-Throated Sparrow', 'Captain
We already know how to find the owl in position 2 (keeping in mind that we start counting
at 0):
owls[2]
'Barn Owl'
owls[-1]
'Spotted Owl'
birds
[['Barred Owl', 'Burrowing Owl', 'Barn Owl', 'Screech Owl', 'Spotted Owl'],
['Goldfinch', 'Crossbill', 'Redpoll', 'Grosbeak'],
['House Sparrow',
'Song Sparrow',
'White-Throated Sparrow',
'Captain Jack Sparrow']]
So to dig the data out of birds , we need to go two levels deep with our slices. To get
‘Redpoll’, for example, we need to find the item in position 2 of list 1:
print(birds[1][2])
Skip to main content
Redpoll
Our variable mishmsoh was even more complicated, but the same principle still applies.
We just need to work our way down through the layers, until we get to the data we want.
So, if I am still having a hard time remembering what kind of pet I have, all I need to do is
find the information in the variable:
My pet is a cat
His name is D2
He likes sleeping
If this seems like an enormous amount of work to go to just to store details about my cat,
well, I don’t blame you. But just file these variable types away somewhere in your mind,
because they will all be very useful later on as you progress with Python. And yes, I did
name my cat D2.
Skip to main content
4.18. Slices
I have mentioned the concept of slicing before, but now it is time to talk about it in a bit
more depth. Sometimes we want to access a portion of our data. For example, we might
want only part of a string, or part of a list. Say I go to the store every day, and every day I
write down how much I spend.
spent = [2,5,345,4,6,12,432,6,575,43,64,32,234,23,6,3,565,345,43,233,12,865,456
We can slice the data, to find any grouping of these numbers that we like. For example,
the first five:
spent[0:5]
[2, 5, 345, 4, 6]
The slice uses square brackets[10] to indicate which data we want. By now, we are
familiar enough with Python to know that Python starts counting at zero, so it makes
sense[11] that if we want the first 5 numbers from our list, then we will need to start with
the number in the zeroeth position. But why does the slice end with the number in the
fifth? Because, wouldn’t that be the fourth number, if we start counting at zero? Yes, it
would, except that Python slices end with the item in the position indicated by the second
number not including that number! So when we write [0:5] , Python starts counting at 0,
then includes everything up to one less than the final number. Sometimes I think the
authors of Python are just messing with us. But if you play around with it a bit, you’ll
soon get the hang of it.
spent[-5:]
Here we are counting backwards to five positions from the end, and then going forwards
to the end. Skip to main content
Of course, we could also take a section from the middle:
spent[4:10]
The general form for slices is [start:end:step] , where step indicates how many
positions to jump each time. If you don’t indicate any value for step , Python assumes it
is one. If you don’t put any value for end , then Python assumes it is the same as the
value for start . We can combine these to find:
[3, 565, 345, 43, 233, 12, 865, 45645, 45, 332, 453, 346, 23, 234, 3, 324, 1]
spent[0:17:7]
[2, 6, 6]
spent[::4]
Now that we have seen how to use the different data types to store data, and indices and
slices to get data out, it is time for a break. Storing data in collections like lists, tuples, and
dictionaries, together with accessing data using indices and slices gives you enormous
flexibility, but it takes practice to get used to it. Slicing in particular, I find, gives people
(including me!) trouble, so I really do recommend spending some time making lists of
data, and slicing them in every way you can think of, until you get the feeling for how it
works.
4.19. Summary
This chapter continued where the previous chapter left off. The focus was still primarily
on introducing basic Python concepts, but this time at least you can see how those
concepts are related to data analysis:
Installing and importing libraries. Knowing how to extend the functionality of Python
by installing and using libraries is critical to becoming an effective Python user
Loading and saving data. Finally, we encountered actual data files. Loading and
saving data is obviously a crucial skill, this was dealt with here.
Filesystem paths. This section dealt very briefly with the concept of filesystems.
Mostly, I just told you what a filesystem path is, and left it up to you to figure out how
Skip to main content
to find paths on your machine.
Useful things to know about variables. In this section, we talked about variable types,
with a special focus on lists and dataframes. There is much more to say about these
topics, but hopefully this is enough to get you started.
Taken together, the chapter on Getting Started with Python and More Python Concepts
provide enough of a background that you can finally get started doing some statistics!
Yes, there’s a lot more Python concepts that you ought to know (and we’ll talk about
some of them in the chapters on Data Wrangling and Basic Programming, but I think that
we’ve talked quite enough about programming for the moment. It’s time to see how
your experience with programming can be used to do some data analysis…
[1] There are some subtle differences between libraries, packages, and modules, but we
don’t need to concern ourselves with these here, and I may well mix up these words
in the text. The key thing is, they are bits of code that we need to import to make stuff
happen in Python.
[2] [Link]
[3] Basically, the reason is that there are thousands of libraries, and probably thousands
of authors of libraries, and no-one really knows what all of them do. Keeping the
installation separate from the loading minimizes the chances that two libraries will
interact with each other in a nasty way, like using the same name for different things.
[4] Well, actually it is an .ipynb file, but let’s not bicker and argue about who killed
who.
[5] Chiefly among them, this helps avoid unpleasant situations where we may have
imported two different libraries that each contain a function with the same name, but
which do different things, or worse yet, do different things but appear to do the same
thing, which could really get us in trouble!