TeachingNote_Python

October 18, 2023

I
@author: Amir Akbari

AR
1 Starting with Python
1.1 Installing Python
Please search for Download Anaconda Python on Google, choose your Operating System (Win-
dows, iOS, Linux), and follow the instructions to install Python version 3 via Anaconda. Anaconda
is a distributor of Python and provides Python Interpreter (engine), Sypder IDE (editor), Jupyter
(notebook), . . . . This makes the Python configuration on your computer seamless, however, some
KB
advanced users may prefer to install Python components separately, and for example work with a
different editor such as PyCharm.

Note: To prevent potential glitches, avoid naming your folders with special characters. Likewise,
it is also recommend to avoid a space in folder and file names. You may use the underscore key,
_, instead. While this is not an issue for installing Python or Anaconda, it is an issue for IPython
which looks in c:\user\username\.ipython for configuration files. The solution is to define the
HOME variable before launching IPython to a path that has only ASCII characters.
A
I use a Windows laptop and present the shortcut keys for Windows. On Mac, often there is a similar
shortcut key and functionality using the command key instead of Ctrl. Of course, Mac users can
search for the shortcut on the web too. The current Python version on my device is 3.8 and I work
with Spyder version 5. If you use a different version (Python 3 and above) you may notice small
visual differences but the functionality would be them same.
A.

1.2 Understanding the Sypder Environment

Spyder is an IDE (Integrated Development Environment) for Python. The IDE makes it easier to
write codes in Python and later debug your code. Below, I introduce the basics of Spyder. You may
find more information about it on its website at [Link] and on its YouTube
channel at Spder IDE.

Note: You may change the background color (aka theme) or font in the Editor and other windows
in Spyder from Tools -> Preferences -> Appearance

1
 Spyder has three windows:

I
AR
KB
Figure 1. Spyder Windows. An example Screenshot

1. Editor: where you write your codes.

This is where you write your codes before evaluating it. In the editor, your codes are color-coded for
better readability. The executable codes are shown in black font. The reserved words (see Section
4. Variable Names and Types below for details) are in blue. The text input values (those between
A

" " or ' ') and notes are in green. The gray font lines are comments. Notes and comments are
similar concepts. These are lines which will not be executed but help to document, add clarification
comments, and organize your code. Use # at the beginning of your text to comment out a line and
use """ to open and close a few lines of notes.

Spyder has the tab completion feature: - After entering 1 or more characters, pressing the TAB
A.

button on your keyboard will bring up a list of functions, packages, and variables that match the
typed text. This will reduce the chance of a typo in the syntax and is the recommended approach
in coding, especially because Python is case sensitive (Python is also whitespace sensitive. So
indentation, either spaces or tabs, affects how Python interprets files. We will talk about these
more later). If the list of matches is large, pressing the TAB button again allows the arrow keys can
be used to browse and select a completion. Also, when you write the function names in Spyder a
window will pop up which shows its syntax, its input variables, and further information. A similar
feature is in MS-Excel.

2
 The editor comes with a line number area, shown in the left side bar (see Figure 1). If Spyder spots
an error in a line of your code it will show warnings and syntax errors, with a cross in a red circle
beside that line. They can help you to detect potential problems before running your code. For
instance, there is an error in line 16 in Figure 1 above. If I hover my mouse on the alert sign (the
cross in a red circle) an window will pop up that reads Code Analysis Undefined name "MIN".
This occurs because Python is case sensitive and differentiates between min and MIN or Min. It does
not know what MIN is so it gives the error.

I
Once you are done with your code, you can save it similar to an MS-Excel file and open it next
time. Your codes will have a .py type. A Python program (aka scripts) is a collection of functions

AR
that can be run stand alone. You can open as many files you want in Spyder. They will be shown
as a new tab beside each other. In Figure 1, I have two files open: TeachingNote_Python.py and
[Link]. In Spyder, if you modify your codes and your file is not saved afterwards, you will
see an asterisk (*) beside its name. In Figure 1, I have modified [Link] file but did not save
it, so Spyder added a * to indicate that.

2. IPython Console: where you run your code and see the outputs

This is where you can run Python code, either from the Editor or interactively. To run your code
KB
you have several options. The most commonly used ones are:

• you can execute your whole script by choosing Run from the Run drop menu in the toolbar (See
Figure 2). Alternatively, you may press the green triangle icon (its shortcut key on Windows
is F5).

• if you want to run one part of your script, first create a cell by adding #%% before and after
that part. Then choose Run Cell from the Run drop menu. This will execute the codes in
the active cell, the one that your mouse has last clicked in. Alternatively, click the icon with
the green triangle in a rectangle (its shortcut key is Ctrl+Enter). If you want to serially run
A

cells, one after another, you can use Run cell and advance or the icon with a double green
triangle in a rectangle (its shortcut is Shift+Enter).

• if you want to run a few lines of your code, you can select them and choose Run selection
or current line from the Run drop menu, or use the icon with a green triangle beside I (its
shortcut key is F9). Similarly, you can write part of your code in the IPython Console window
A.

and press the Enter key to run only that part. For example, you can use it as a calculator.
Try it!

3
 I
AR
Figure 2. Run command

The output of your code will be shown in the IPython Console. For instance, when I execute
print("Hello, World!"), which asks Python to display the text “Hello World!”, we see this text
displayed in the IPython Console window (Figure 1). An input counter number beside each of
your commands will be displayed too, e.g. In [1] before print ("Hello, World!") in Figure 1. The
counter for the next command increases serially. Also, if your code has an error, it will be shown
KB
here. For example, run x2 = MIN(5,-1,100) to see a NameError.

You may access the history of all the codes you executed (ran) in History tab in this window too.

3. Source Window: it has four panes

• VARIABLE EXPLORER: here you can see the variables you generated. In Figure 1, I
executed x1 = min(5,-1,100), and we see x1 in the Variable Explorer window, whose
value is -1. Its type is int (integer) and size is 1 (See Section 4. Variable Names and
Types for more details on these).
A
• FILE EXPLORER: you can see all the files or folders in your working directory (see the
upper right address bar in the Figure 1, which points to the folder C:\Users\akbara23
on my device. Click on the folder icon beside this address bar and choose a folder for
this course on your computers.
• HELP: you can see help and documentation on the functions. To access the help and
documentation of each function, in the IPython console or in the Editor, select a function
A.

(double click on it) and press Ctrl+I. This will call the help on that function and show it
in the Source window. You can also type help(FUNCTION_NAME) in the console or search
for the function in the HELP pane.
• PLOTS: if your code generates a plot, it will be shown here (see Section 5.7. Matplotlib
for details on plotting in Python).

4
 2 Working with Python
Python is an interpreted high-level programming language for general-purpose programming. For
finance related purposes, Python allows us to process data with complex formulas, similar to what
VBA does in MS-Excel, but with fewer limitations and more access to predefined functions. In
addition, it allows us to work with large tables and databases, that are difficult to manage with
MS-Excel. Moreover, it helps us to run our analysis repeatedly on different databases automatically.

Python has an extensive set of built-in functions (similar to those of MS-Excel), but you can also
work with more sophisticated functions that others have written/developed, via importing those

RI
specific Python Libraries (will talk about them shortly). Of course, you can define your own functions
as well.

In this teaching note, for pedagogical reasons, I do my best to draw analogies between Python and
MS-Excel. This undersells the vast capabilities in Python, but I believe will help you to better
adapt to this programming language.

BA
Functions are called similar to MS-Excel: FUNCTION_NAME( input_variables). For example, the
print function prints the value of the input in the IPython Console:

[1]: print("Hello world!")

Hello world!
AK
More generally, functions can have multiple input variables (separated with commas) and can gen-
erate multiple output variables. In general, the syntax is:

out1, out2, out3, . . . = FUNCTION_NAME(in1, in2, in3, . . .).

In the example below, we are giving (parsing) 3 input variables to the min (minimum) function:

[2]: min(5, -1, 100)

[2]: -1
A.

To access the help and documentation of each function, in the IPython console or in the Editor,
select a function (double click on it) and press Ctrl+I. This will call the help on that function and
show it in the Source window. You can also type help(FUNCTION_NAME) in the console. For example
for the min function:

[3]: help(min)

Help on built-in function min in module builtins:

min(...)
min(iterable, *[, default=obj, key=func]) -> value

5
 min(arg1, arg2, *args, *[, key=func]) -> value

With a single iterable argument, return its smallest item. The

default keyword-only argument specifies an object to return if
the provided iterable is empty.
With two or more arguments, return the smallest argument.

I
You can also search for documentation for the function in the search bar in the HELP window.
Lastly, help can also be shown automatically after writing a left parenthesis next to an object. You

AR
can activate this behavior in Preferences > Help.

Note that Python is case sensitive and if you type Min, you will receive a NameError, which means
python cannot recognize this function.

[4]: #MIN(5, -1, 100)

NameError Traceback (most recent call last)

<ipython-input-207-451a873a8a93> in <module> ----> 1 MIN(5, -1, 100) NameError:
KB
name 'MIN' is not defined

2.1 Variable Names and Types

Python, similar to MS-Excel, needs to know the type of the variables it is working with. Recall in
MS-Excel, we can set the data type of each cell to Number, Date, Text, Currency, . . . . A similar
setting exists in Python.

The full details of these are easier to comprehend when we face an application. Below, I just list
some important pointers so that you can start working with Python.
A

2.1.1 Data Names

In MS-Excel, each cell is a variable (e.g. A2 or B17). In Python, we directly give a name to the
variables we use (this can also be done in MS-Excel). For instance:

[5]: A = 3
A.

B = 5
C = A+B
print(A,B,C)

3 5 8

You can also just type the name of the variable and run it. e.g.

6
[6]: C

[6]: 8

In this document, I use both of these approaches to display the output values.

Variable names can take many forms, although

• They can only contain numbers, letters (both upper and lower), and underscores. They cannot

I
have special characters such as $.
• Also variable names cannot have space in them (use underscore instead).

AR
• Names must begin with a letter or an underscore. They cannot start, for instance, with a
number.
• Names are CaSe SeNsItIve. That is, variables A and a are different variables.
• Reserved Words: The following names are Python keywords. They
have special meaning and shouldn’t be used as variable names:
as, or, in, is, if, and, del, not, try, for, def, from, elif, with, else, exec,
pass, while, yield, break, print,class, raise, global, assert, except, import,
lambda, return, continue, finally
KB
If you define/use them in your code as a new variable, you’ll get a syntax error. Therefore, the
following codes will not execute:

x: = 1.0

1X = 1

X-1 = 1

for = 1
A

2.1.2 Data Types:

Some of the most commonly used variables are (we will see the more complex ones later in the
course):

• Integer Numbers (aka INT):

A.

Integers are numbers starting from 1, 2, 3, ..., ∞. For example:

Caution: Notice that large numbers never include commas. We can ask Python to show them with
commas (e.g. 123,456,789) but we do not input them with commas (e.g. 123456789). In MS-Excel,
if you input 123,456,789 in a cell, Excel guesses that this is a number and automatically stores it
as a number without asking the user. See the subsection for printing numbers to learn about how
to ask python to print numbers with commas use as below

7
[7]: a = 123456789
print(a)

123456789

• Real Numbers (aka FLOAT)

Real numbers are numbers with decimal values. You can ask Python to show only a few decimals
of the number (similar to the MS-Excel Data Type). See subsection printing numbers.

I
[8]: a = 13.56798

AR
print(a)

13.56798

• String (aka STR)

Strings are similar to the TEXT data type in MS-Excel. In order to make a string we may use a
single quote sign ' or double quotations ". Either works equally well. But if the string contains
one, we need to use the other. For example

x = "His name is Conan O'Brien"

KB
x = 'My cat is named "Butters" '

If you need both a ' and a " in your string, you can use the escape character which tells Python
that the following character is to be taken as the literal character and is not a quote to delimit the
string. See it in action escaping the ” below:

x = "My cat's name is \"Butters\""

Example: Let’s see an example using string and print function

A

[9]: x = "My cat's name is \"Butters\""

print(x)

My cat's name is "Butters"

Recall, the syntax for each function is FUNCTION_NAME(in1,in2,...). There are no commas be-
A.

tween parentheses. So Python assumes there is one input. The input is not a number (integer or
float) nor a string (it is not in " or '). So, Python concludes that you want to print the values of
variable x.

Note: Going back to our initial example, print("Hello world!"), the input to the print function
is the string “Hello world!”. If we do not put this string in ", we receive a syntax error

[10]: #print(Hello world!)

8
 File "<ipython-input-345-01a3c6a17cb8>", line 1 print(Hello world!)
ˆ SyntaxError: invalid syntax

The error message is pointing to w in world. Why? Because you are not parsing a number or a
string. So, Python concludes that you are parsing a variable to be printed but the name of this
variable has space in it (before world), and spaces are not allowed for names. Ok, let’s substitute
space with an underscore print(Hello_world!)

I
[11]: #print(Hello_world!)

File "<ipython-input-572-a8b0d1e4f18c>", line 1 print(Hello_world!)

AR
ˆ SyntaxError: invalid syntax

Again, we get a syntax error but this time it is pointing to !. Recall, variable names cannot have
special characters such as !. Ok, let’s remove it, print(Hello_world)

[12]: #print(Hello_world)

---------------------------------------------------------------------------
NameError Traceback (most recent call last)
KB
<ipython-input-692-fabb9b810891> in <module> ----> 1 print(Hello_world) NameError:
name 'Hello_world' is not defined

And now we get a Name Error, suggesting that we want to work with (print) a variable that we
have not defined it before.

• List (aka LIST)

List allows you to have a group of variables in one variable. This is similar to selecting a series of
cells (in a row or column) in MS-Excel. For example
A

List of characters: x1 = ['a','b','c']

List of numbers: x2 = [1,5,23]

List of characters and numbers: x3 = ['ball', 3.14, -50, 'university', "course"]

List of Lists: newEngland=[["Massachusetts",6692824],["Connecticut",3596080],["Maine",1328302],["New

A.

Hampshire",1323459],["Rhode Island",1051511],["Vermont",626630]]

• Boolean (aka BOOL)

The Boolean variables are 0/1 or True/False variables. e.g.:

[13]: x1 = True
y1 = False

Boolean variables are very useful in conditional statements, such as IF statements.

9
 2.2 Working with Numbers
2.2.1 Basic arithmetic operations on numbers

• +, -, *, /: These add, subtract, multiply, divide.

• **: This is for exponentiating (to the power).
• //: This is for integer divide (drops fractional part).
• %: This is to compute remainder on division for integers (MOD function in MS-Excel).

I
[14]: a = 5
b = 2

AR
x1 = a + b
x2 = a * b
x3 = a ** b
x4 = a / b
x5 = a // b
x6 = a % b

print("x1=",x1, "; x2=", x2, "; x3=",x3, "; x4=",x4, "; x5=",x5, "; x6=",x6)
KB
x1= 7 ; x2= 10 ; x3= 25 ; x4= 2.5 ; x5= 2 ; x6= 1

2.2.2 Priority of operations:

The usual laws of arithmetic hold with respect to the priority of the operations. For example,

[15]: x7 = x1 + x2*x3**x4/x5
print(x7)
A
15632.0
x
x2 x3 4
x7 = x1 + x5
10 252.5
=7+ 2
10 3125
=7+ 2
A.

= 7 + 15625

x7 = 15632

Of course, you can use parentheses to clarify the priority of operations:

[16]: x7 = x1 + ((x2*(x3**x4))/x5)
print(x7)

15632.0

10
 2.2.3 Printing Numbers

The print() function allows you to modify the format of the printed values. The general syntax is
to write f'{Variable_name:Desired_Format}

For instance, if you want to show numbers with commas use as below:

[17]: a = 123456789
print(a)

I
print(f'{a:,}') # to change the format (f) of the displayed number (a) to␣
comma-separated (,)
,→

AR
123456789
123,456,789

If you want to print numbers in with several decimals, say 3, use as below:

[18]: a = 13.56798
print(a)
print(f'{a: .3f}') # to change the format (f) of the displayed number (a) to␣
float (f) with (rounded) 3 decimals
KB
,→

13.56798
13.568

If you want to print numbers in percentage with the % sign, and no decimals, use as below:

[19]: a = 0.57134545
print(a)
print(f'{a*100:.0f}','%') # multiply a by 100, and add \% sign
A

0.57134545
57 %

If you want to show numbers in accounting format, with 2 decimals and $ sign, use as below:

[20]: a = 8712345.6789
print(a)
A.

print('$',f'{a:,.2f}')

8712345.6789
$ 8,712,345.68

11
 2.3 Working with Lists
2.3.1 Accessing items of a list

Consider the following example, where a list is called Alphabet:

[21]: Alphabet = ["a","b","c","d","e","f"]

To get the ith element of the list, we use the following syntax: List_name[i-1]. In Python, the
index of a list (and array and matrix variables that we will see later) starts from 0. Example:

RI
[22]: print ("The 1st item of Alphabet is: ", Alphabet[0])
print ("The 2nd item of Alphabet is: ", Alphabet[1])
print ("The Last item of Alphabet is: ", Alphabet[-1])

The 1st item of Alphabet is: a

The 2nd item of Alphabet is: b
The Last item of Alphabet is: f

BA
This is equivalent to the INDEX(array, row_num, [column_num]) function in MS-Excel, where
you can access a cell by giving the row and column number of a cell to access it. For example,
INDEX(A4:C10, 2, 3) returns cell C5.

If you are interested in a few items, you can use a range

AK
[23]: print ("Items 2, 3, and 4 of Alphabet are: ", Alphabet[1:4]) # (Index 1,2,3)
print ("3 first items of Alphabet are: ", Alphabet[:3]) # (Index 0,1,2 excluding␣
,→ 3)
print ("Items 4 to end of Alphabet are: ", Alphabet[3:]) # (Index 3, 4, ...)

Items 2, 3, and 4 of Alphabet are: ['b', 'c', 'd']

3 first items of Alphabet are: ['a', 'b', 'c']
Items 4 to end of Alphabet are: ['d', 'e', 'f']

if for any reason you want to access to an empty list:

A.

[24]: x = []

2.3.2 Functions of Lists

• The len() function gives the length of a list variable (i.e. the number of items in the list).
This is similar to the COUNT() function in MS-Excel, which helps us get the length of an array
of cells. e.g.

[25]: x = len(Alphabet)
print("Alphabet has", x, "items.")

12
 Alphabet has 6 items.

• One can insert an item in the List, at a certain place with the .insert feature,
• add an item to the end of the list with the .append feature
• If you want to remove an item from the list, use the .remove feature of the list:

[26]: [Link](2, 'Z') # add character `Z` as the 3rd item of Alphabet
[Link]('B') # add character `B` at the end of Alphabet

I
[Link]('f') # remove character `f` from Alphabet

AR
print(Alphabet)

['a', 'b', 'Z', 'c', 'd', 'e', 'B']

• If you are interested to find the index of an item on the list, use the .index feature of the list:

[27]: print(Alphabet)
[Link]('c')

['a', 'b', 'Z', 'c', 'd', 'e', 'B']

KB
[27]: 3

Similarly in MS-Excel, we can use the ROW() and COLUMN() functions to get the row and column
indexes of a cell.

• You can easily sort a list by .sort feature:

[28]: A = [3,5,-1,2,0,10,11,0]
[Link]()
A
print("After sorting, A is:", A)

After sorting, A is: [-1, 0, 0, 2, 3, 5, 10, 11]

In MS-Excel, we can use Sort & Filter option.

Note, .sort permanently change the order of your list. If you want to keep the original ordering
of the list but you need to sort it, say for just printing, you can use the sorted() function:
A.

[29]: numlist = [67, 54, 39, 47, 38, 23, 99, 91, 91, 70]
print("sorted numlist is:",sorted(numlist))
print("But numlist stays as-is:",numlist)

sorted numlist is: [23, 38, 39, 47, 54, 67, 70, 91, 91, 99]
But numlist stays as-is: [67, 54, 39, 47, 38, 23, 99, 91, 91, 70]

13
 • use in feature to check if an item belongs to a list. For example, running this statement will
return True, because Alphabet includes “a”.

[30]: "a" in Alphabet

[30]: True

[31]: # And running this statement will return False

I
"r" in Alphabet

[31]: False

AR
2.3.3 Copying Lists

Lists are mutable and so assignment does not create a simple copy. After the assignment the changes
to either variable affect both.

[32]: x = [1, 2, 3]
y = x
y[0] = 10
KB
print("y = ", y)
print("x = ", x)

y = [10, 2, 3]
x = [10, 2, 3]

This is because of the memory management of Python. Setting y = x, also sets the memory pointer
A

of y equal to that of the x. Therefore, if you change y, you are also changing x. This makes data
management more efficient by Python (but we don’t care about it in this class).

However, slicing a list creates a copy of the list and any immutable types in the list – but not
mutable elements in the list. This helps you preserve the value of x. In this way, simply copy the
content of x to y:
A.

[33]: x = [1, 2, 3]
y = x[:]
y[0] = 10
print("y = ",y)
print("x = ",x)

y = [10, 2, 3]
x = [1, 2, 3]

14
 2.4 Converting Variable Types
Similar to MS-Excel. Sometimes variables are stored in the wrong format and we need to change
them. You can

• find the type a variable x by entering:

[34]: type(Alphabet)

I
[34]: list

[35]: x1 = 40

AR
type(x1)

[35]: int

• convert a number to a string (character)

[36]: x2 = "30"
print("type of the variable x2 is", type(x2))
KB
type of the variable x2 is <class 'str'>

Similarly, we can convert a previously stored number to a string variable:

[37]: x3 = str(x1)
print("type of the variable x3 is",type(x3))

type of the variable x3 is <class 'str'>

A
In MS-Excel, the function TEXT() does a similar job.

• convert a number which is stored as a string (character) back to a number

[38]: x4 = int(x2)
print("type of the variable x4 is",type(x4))
A.

type of the variable x4 is <class 'int'>

[39]: x5 = float(x2)
print("type of the variable x5 is",type(x5))

type of the variable x5 is <class 'float'>

Knowing the data type of the variables you are working with is important. For example since x2
and x3 are stored as a string (character) then x2+x3 is ‘3040’ (not 70)!

15
[40]: print(x2+x3)

3040

however, int(x2)+int(x3) is 70.

[41]: x = int(x2)+int(x3)
print(x)

I
70

AR
A KB
A.

16
 3 Conditional Statement
3.1 IF statement
The concept of the IF statement is similar to the one in MS-Excel. There are three versions of the
IF conditions: (1) if - then, (2)if-then-else, (3)if-then-elseif-then-else.

In Python, the statement after the condition is considered the “then” statement of the IF statement
and there is no need to include the “then” argument. Simply add 4 spaces or 1 tab to open the

I
“then” arguments.

AR
1. Simple if-then statement: the syntax is as follows

IF CONDITION :

WHAT TO DO IF CONDITION HOLDS

For example:

[42]: x = 5
if x > 0:
print("x equals to", x)
KB
print("x is positive")

x equals to 5
x is positive

2. if- then- else the syntax is as follows

IF CONDITION :

WHAT TO DO IF CONDITION HOLDS

A

ELSE:

WHAT TO DO IF CONDITION DOES NOT HOLD

For example:

[43]: y = 0
A.

if y > 0:
print("y is positive")
else:
print("y is not positive")

y is not positive

3. if- then- else if- then, . . . Nested if statement can be implemented with elif command
(shorten version of else if). The syntax is as follows

17
 IF CONDITION_1 :

WHAT TO DO IF CONDITION_1 HOLDS

ELIF CONDITION_2 :

WHAT TO DO IF CONDITION_2 HOLDS

ELSE:

I
WHAT TO DO IF CONDITION_1 & CONDITION_2 DO NOT HOLD

elif can be repeated as often as necessary. For example:

AR
[44]: z = 0
if z > 0:
print("z is positive")
elif z < 0:
print("z is negative")
else:
print("z must be 0")
KB
z must be 0

3.2 Logical Operators and Devices

Logical operators and devices are useful if you want to create (more complex) conditions. The core
logical operators (besides > and <) are

• >= : Greater than or equal to

• <= : Less than or equal to
A
• == : Equal to
• != : Not equal to

Note: Python uses = for assignment and == for testing equality. Whereas, in MS-Excel, you can
use = in the IF statement for testing equality. There you can use <> to test for the not equal
condition.
A.

If you want to combine several conditions you use logical devices:

• logical_and(): Both logical expressions are True. Alternatively use and or & between con-
ditions
• logical_or() : Either of the logical expressions is True. Alternatively use or between con-
ditions
• logical_not(): Not True. Alternatively, use not or ~ between conditions

18
 There are equivalent logical operators in MS-Excel such as AND(), OR(), and NOT() functions that
can be used in the IF statement.

• Special case: for lists you can also use the in feature to check if an item belongs to it.

[45]: Team = ['T','e','a','m']

if not('i' in Team):
print('There is no "i" in "Team"!')

I
There is no "i" in "Team"!

AR
You can achieve similar functionality in MS-Excel too but it is less straightforward. You can use
the contain search option in a column that is already Filtered.

3.3 FOR statement

A FOR loop, allows you to iterate an operation several times. More specifically, it allows you to
iterate over the members of a sequence in order, say in a list, to execute a block of code each
time. There is no easy alternative in MS-Excel for loops, except dragging a formula in a row. For
instance, you may use an index variable (say row 1 to 5), and copy the equation in multiple rows in
KB
the adjacent column to mimic the loop functionality. This is more flexible and simpler in Python.
The syntax is

FOR DEFINED_RANGE :

WHAT TO DO IF CONDITION_1 HOLDS

For example, below, we ask Python to print “Financial Modeling is FUN with Python!” 5 times.
We use the RANGE function to specify the number of iterations:

[46]: # Example 1
A

for i in range(5):
print("Financial Modeling is FUN with Python!" )

Financial Modeling is FUN with Python!

Financial Modeling is FUN with Python!
Financial Modeling is FUN with Python!
A.

Financial Modeling is FUN with Python!

Financial Modeling is FUN with Python!

The RANGE(a,b,j) function is similar to the SEQUENCE() function in MS Excel and uses a start
number a, a stop number b and a step size j. When not specified, a=0 and j=1. For example,
range(10) is equivalent of range(0,10,1) and range(2,10) is equivalent of range(2,10,1).

19
[47]: # Example 2
for i in range(2,10):
print(i,end=' ')
# the `end=' '` is used to avoid the enter at the end of each print.
# therefore the output is printed in one line:

2 3 4 5 6 7 8 9

I
[48]: # Example 3
for ct in range(2,9,2):

AR
print(ct,end=' ')

2 4 6 8

[49]: # Example 4
for ct in range(22,5,-3):
print(ct,end=' ')

22 19 16 13 10 7
KB
We can define a specific range by using iterating in a list:

[50]: # Example 5
for names in ["Sarah", "Ben","Moe"]:
print(names)

Sarah
Ben
Moe
A

[51]: # a more advanced example of 5

Companies = ['TD','RBC','Goldman Sachs']
CEOs = ['Leo Salom','David I. McKay','David Solomon']

for i in range(len(Companies)):
print('CEO of', Companies[i],'is',CEOs[i],'.' )
A.

# here i gives the index of the loop for each iteration.

CEO of TD is Leo Salom .

CEO of RBC is David I. McKay .
CEO of Goldman Sachs is David Solomon .

Note that this is identical to the example below, where I use the enumerate function to also get the
index number in the CEOs list and assign it to the variable i:

20
[52]: # a more advanced example of 5
CEOs = ['Bharat Masrani','David I. McKay','Lloyd Blankfein']

for i,company_name in enumerate(['TD','RBC','Goldman Sachs']):

print('CEO of', company_name,'is',CEOs[i],'.' )
# here i gives the index of the loop for each iteration.

CEO of TD is Bharat Masrani .

I
CEO of RBC is David I. McKay .
CEO of Goldman Sachs is Lloyd Blankfein .

AR
Loops can also be nested (a FOR loop inside another FOR loop). Pay attention to the spacing at the
beginning of each line, in each loose

[53]: # Example 6
count = 0
x = range(10)
y = range(10)
KB
for i in x:
for j in y:
count = count + j
print(count)

450

Important: The iterable variable (in the above example x and y) should not be reassigned once
inside the loop.
A

A loop can be terminated early using break. The break command is usually used after an if
statement to terminate the loop prematurely if some condition has been met.

[54]: # Example 7
x = range(10)
for i in x:
A.

print(i,end=' ')
if i > 5:
break

0 1 2 3 4 5 6

21
 A more readable implementation of the above example is with the use of while loops:

WHILE CONDITION_HOLDS:

WHAT TO DO IF CONDITION_1 HOLDS

Contrast the for statement with the while loop, we check a condition for each iteration. For
example:

[55]: i = 0

I
while i < 5:
print("Financial Modeling is FUN with Python!" )

AR
i = i+1

Financial Modeling is FUN with Python!

Financial Modeling is FUN with Python!
Financial Modeling is FUN with Python!
Financial Modeling is FUN with Python!
Financial Modeling is FUN with Python!
A KB
A.

22
 4 Define a Function:
In Python, you can define your own function, in a separate file for example, and call it. This will
help you with code-readability and debugging your code. You can test each function separately and
then merge them all.

The syntax is as follows:

DEF FUNCTION_NAME(INPUT_VARIABLE):

I
PROCESS INPUT_VARIABLE

AR
RETURN OUTPUT_VARIABLE (optional)

After defining your function, you should load it to Python. This way, Python learns about your
newly defined function. Then you can parse any input variable you like to the function, as many
times as you like. There are two ways to do so 1. Simple: copy the function you wrote in the
console and run it. 2. Complete: save it in a .py file (e.g. [Link]). Then load it similar to
a library (see the next section).
KB
Example: Write a function that calculates the total grade of a student in Commerce-3FD3 Finan-
cial Modeling, from his/her grade in 5 weekly assignments (10% in total), 2 midterm exams (40%
in total), final exam (40% in total), and class participation (10% in total)

[56]: def Total_Grade_3FD3(Assignments,MidtermExams,FinalExam,ClassParticipation):

# Input values:
# Assignments is a list of 5 marks, each with a 2% weight
# MidtermExams, each with a 20% weight
# FinalExam, each with a 40% weight
A
# ClassParticipation, each with a 10% weight
# Output
# Total_Grade = 10%*Assignments + 40%*MidtermExams + 40%*FinalExam +␣
10%*ClassParticipation
,→
A.

Total_Grade = 0.02 * sum(Assignments) + 0.20 * sum(MidtermExams) + 0.40 *␣

FinalExam + 0.10 * ClassParticipation
,→

return Total_Grade

Now that we have this function, we can run the above function to get the grade of any student. For
example

23
[57]: # Sarah's marks:
Total_Grade_3FD3([100,0,0,100,50],[80, 65],78, 80)

[57]: 73.2

[58]: # Rose's marks:

Total_Grade_3FD3([100,100,100,100,100],[80, 75],88, 100)

I
[58]: 86.2

AR
I can now give the data for the whole class and get their total grade.

[59]: # step 1. create the input data

Section_4_marks = [
["Sarah",[100,0,0,100,50],[80, 65],78, 80],
["Rose",[100,100,100,100,100],[80, 75],88, 100],
["Ben",[100,0,0,100,50],[80, 65],78, 80],
["John",[0,0,0,100,50],[50, 45],38, 20],
]
KB
# step 2. calcualte the marks for wach student and print
for student_i in Section_4_marks:

student_i_totalGrade =␣
Total_Grade_3FD3(student_i[1],student_i[2],student_i[3],student_i[4])
,→

print("The total mark for", student_i[0], "is", student_i_totalGrade)

The total mark for Sarah is 73.2

The total mark for Rose is 86.2
A

The total mark for Ben is 73.2

The total mark for John is 39.2

I can further modify my codes to get the class average.

[60]: # step 1. create the output

Section_4_totalGrades = []
A.

# step 2. calcualte the marks for wach student

for i, student_i in enumerate(Section_4_marks):

student_i_totalGrade =␣
Total_Grade_3FD3(student_i[1],student_i[2],student_i[3],student_i[4])
,→

24
 Section_4_totalGrades.append(student_i_totalGrade)

# step 3. print the class average

classAverage = sum(Section_4_totalGrades)/len(Section_4_totalGrades)
print("The class average is", classAverage)

The class average is 67.95

I
Furthermore, you can call a function inside another function.

AR
Example: Write a function to calculate Effective Annual rate of Return of a multi-period invest-
ment. Your function will receive the each year’s rate of return in a list and the output of your
function is the overall rate of return for the investment, annualized.

From the times value of money equation we have,

F V = P V (1 + R)

Therefore, starting with $ 1 investment, after t years, the wealth of the investor is:
KB
W ealtht = 1 × (1 + r1 ) × (1 + r2 ) × · · · × (1 + rt )

Using the definition of EAR for T years, we have:

F V = P V (1 + EAR)T

Rearranging the two equations, we have:

A

 1
T
EAR = W ealthT −1

[61]: def EAR_function(Returns):

#This function calculates the Effective Annual Rate of a given list of interests
#(1+EAR)^T = (1+r_1)*(1+r_2)*...*(1+r_T)
A.

#Input:
# Returns= n-list of annual rate of returns for n years
#Output:
# EAR = float

# Initialization

25
 Wealth = 1 # This is the initial wealth of $1

# calculate the total return for the whole investment period

for R_t in Returns:
Wealth = Wealth * (1+R_t)

# calculate the EAR

I
EAR = Wealth ** (1/len(Returns)) - 1

AR
return EAR

[62]: #%% Run the function: Example 1

R = [0.25,0.10,-0.12,0.05, 1.65, 2.00, -0.50]
print(EAR_function(R))

0.2602976156686305

[63]: #%% show the EAR in % with 2 decimals

KB
R = [0.05,0.10, -0.05, -0.02, 0.15]
print("{0:.02f}%".format(EAR_function(R)*100) )

4.34%
A
A.

26
 5 Libraries
In Python, many tools (functions) are not automatically loaded and are in modules called libraries.
To be able to use the functions in a library, you first need to load the library. It is recommended to
only load libraries that you need in each of your codes. In this way, you keep your programs smaller
when they aren’t needed.

There are a large number of libraries available for Python. Often you need to search online to find

I
the proper library and the proper functions in that library to get the desired job done. For the
purpose of this class, I shortly introduce a set of widely used libraries and their functions that help us

AR
with financial modeling, such as NumPy, Pandas, MatPlotLib and SciPy. These libraries are already
installed on your devices once you install Python with Anaconda. There are two other libraries
that we use frequently and are not already installed: NumPy-Financial and Pandas_DataReader.
Below, I show how to install a library on Python.

5.1 Install packages (library)

There are several methods to install packages. The most common one is pip install: On your
computer, in Windows program files, search for Anaconda Prompt(Anaconda3). In Mac iOs you
KB
can search within the Terminal search bar. This will open a black command window. Type pip
install numpy-financial and press enter. This will install the NumPy-Financial library on your
Python
A
A.

Figure 3. PIP INSTALL with Anaconda Prompt

Similarly, type pip install pandas_datareader to install this library. Update Recently, Yahoo
made changes to their API that broke compatibility with previous pandas datareader versions.
As a short-term solution, I use functions in yfinance library (for more on this, see the section
related to pandas datareader below). Therefore, we need to also install this library. In Anaconda
Prompt(Anaconda3) type pip install yfinance to install this library.

27
 5.2 Working with libraries
To get the documentation for a library, go to [Link]/3/ and search for the name of that
library (say NumPy). Go to that library and review the functions there. You can alternatively, go to
Help>Python Documentation in Spyder and search NumPy in the index. You see the same thing.

A typical way of loading a library is by using the import LIBRARY_NAME: e.g.

[64]: import numpy_financial # which will load the library named NumPy-Financial. or

I
import numpy_financial as npFin # you can assign a short name for the library␣
,→ too.

AR
Then you can call the functions inside the library (here the NumPy-Financial library) by
LIBRARY_NAME.FUNCTION_NAME(INPUT). This way, Python will not mix two functions with simi-
lar names that are in different libraries.

You can use the dir() command to get the list of functions in each library. For example, after
importing the library (say NumPy-Financial), in the IPython console type dir(NumPy_Financial)
to get the list of functions in the Numpy library. ‘ For example
KB
[65]: import numpy_financial as npFin
dir(npFin)

[65]: ['__builtins__',
'__cached__',
'__doc__',
'__file__',
'__loader__',
'__name__',
A

'__package__',
'__path__',
'__spec__',
'__version__',
'_financial',
'fv',
A.

'ipmt',
'irr',
'mirr',
'nper',
'npv',
'pmt',
'ppmt',

28
 'pv',
'rate']

These functions are equivalent to MS-Excel functions for time value of money. I will discuss them
in the section for the NumPy-Financial library.

Below, I will introduce NumPy (which offers a series of mathematical calculation functions),
NumPy-Financial (for its time value of money functions), Pandas (for data processing in the spread-

I
sheet format), Pandas_DataReader (for downloading financial data from the web), Matplotlib (to
plot graphs), SciPy (for optimization functions), and Statsmodels (for statistical analysis). This

AR
is a basic introduction so that you can start working with these libraries. You may need to consult
their documentation webpages or online forums for a more indepth understanding.
A KB
A.

29
 5.3 NumPy Library
NumPy is a powerful library that allows data processing on large, multi-dimensional arrays and
matrices. Moreover, it provides a large collection of high-level mathematical functions to operate
in these arrays. Most other libraries use NumPy functions in the background. So, I start with it.

5.3.1 Data Types:

NumPy works exclusively with numbers (integer and float). There is also NaN, or Not a Number,
which happens if the input data has missing observations, similar to the blank cells in MS-Excel, or

RI
whenever a function produces a result that cannot be clearly evaluated to produce a number. There
are also inf, which represents infinity, and -inf, which represents the negative infinity. These might
happen, for instance, if you divide by a very very small number, closed to 0.

NumPy provides the core data types for numerical analysis – arrays and matrices, which are both
similar to lists, but arrays and matrices, unlike lists, are always rectangular so that all dimensions

manner. The two important differences are:

BA
have the same number of elements. They resemble the Tables/sheets in MS-Excel. We use mostly
arrays in this class. Matrices are essentially a subset of arrays and behave in a virtually identical

• Matrices always have 2 dimensions, whereas Arrays can have 1, 2, 3 or more dimensions
(although in this course we mostly use 2-dimensional arrays to be consistent with the MS-
Excel examples)
AK
– This means that a 1 by n vector stored as an array has 1 dimension and n elements, while
the same vector stored as a matrix has 2-dimensions where the sizes of the dimensions
are 1 and n (in either order).
• Matrices follow the rules of linear algebra for multiplication.
– Standard mathematical operators on arrays operate element-by-element, similar to MS-
Excel. This is not the case for matrices, where multiplication (*) follows the rules of
linear algebra (this makes coding more concise).
• The best practice is to use arrays and to use the @ symbol for matrix (linear algebra) multi-
plication.
A.

5.3.2 Constructing Arrays

To create an array, from a list, use the array() function:

[66]: import numpy as np

X = [0.0, 1, 2, 3, 4]
Y = [Link](X)
print("An example of a 1-row-5-column vector (1x5):")
print(Y)

30
 An example of a 1-row-5-column vector (1x5):
[0. 1. 2. 3. 4.]

[67]: Y = [Link]([[0.0, 1, 2, 3, 4], [5, 6, 7, 8, 9]])

print("An example of a 2-row-5-column matrix (2x5):")
print(Y)

An example of a 2-row-5-column matrix (2x5):

[[0. 1. 2. 3. 4.]

RI
[5. 6. 7. 8. 9.]]

You can find the dimension of your array/ matrix with the shape() function:

[68]: print([Link](Y))

(2, 5)

Y is an array of 2 by 5

5.3.3 Merging Two arrays (concatenate)

BA
The first item indicates the number of rows and the second item indicates the number of columns.

[69]: print("Y is an array of", [Link](Y)[0], "by",[Link](Y)[1] )

AK
use concatenate() function from the NumPy library can help you merge two arrays. This is similar
to the .append() feature in lists. Use the axis option of the function to define the dimension of
the merger:

[70]: X = [Link]([[1.0,2.5],[3.4,4.6]])
Y = [Link]([[5.0,6.0],[7.1,8.9]])
Z1 = [Link]((X,Y),axis = 0) # stack x on top of y
print("Z1 = ", Z1)

Z1 = [[1. 2.5]
A.

[3.4 4.6]
[5. 6. ]
[7.1 8.9]]

[71]: Z2 = Z1 * 3 # create a new 4 by 2 array from z1

Z3 = Z1 * -2 # create a new 4 by 2 array from z1
Z = [Link]((Z1,Z2,Z3),axis = 1) # stack z2 beside z1
print("Z = ", Z)

31
 Z = [[ 1. 2.5 3. 7.5 -2. -5. ]
[ 3.4 4.6 10.2 13.8 -6.8 -9.2]
[ 5. 6. 15. 18. -10. -12. ]
[ 7.1 8.9 21.3 26.7 -14.2 -17.8]]

In MS-Excel, you cannot automatically concatenate arrays or matrices. You can manually copy and
paste the cells you want. This could be a very time consuming process, if you are working with
several sheets. Also you might make mistakes.

I
5.3.4 Accessing elements of Arrays

AR
Similar to Lists, you can access each item of an array by indexing:

[72]: print(Z[1,2]) # the item in row 2 and column 3

10.2

[73]: print(Z[2:4, 0:2 ]) # rows 3 to 4 (exclusive) and columns 1 to 2 (exclusive)

[[5. 6. ]
KB
[7.1 8.9]]

[74]: print(Z[2, :]) # rows 2 (third row) and all columns

[ 5. 6. 15. 18. -10. -12.]

[75]: print(Z[:,1]) # colum 1 (second column) and all rows

[2.5 4.6 6. 8.9]

You can reshape your array using the reshape() function, as follows:
A

[76]: W = [Link](Z,(2,12))
print("W is a reshaped matrix of Z, which is", [Link](W)[0], \
"by",[Link](W)[1] )
# I used \ sign to break a long line of code into two shorter ones.
print(W)
A.

W is a reshaped matrix of Z, which is 2 by 12

[[ 1. 2.5 3. 7.5 -2. -5. 3.4 4.6 10.2 13.8 -6.8 -9.2]
[ 5. 6. 15. 18. -10. -12. 7.1 8.9 21.3 26.7 -14.2 -17.8]]

5.3.5 Quick look at arithmetic operations

Basic Math Operations These operators in NumPy, listed below, behave similar to the core
Python, when both x and y are scalars (i.e., numbers):

32
 • +, -, *, /, These add, subtract, multiply, divide.

• **, This is for exponentiating (to the power).

• //, This is for integer divide (drops fractional part).

• %, This is to compute remainder on division for integers (MOD function in MS-Excel).

When one of x or y is an array and the other is a scalar the operation is element by scalar. When
one of x or y is a vector array (1 dimentional) and the other is a matrix array (2 dimentional) the
operation is element by element per column. In this case, x and y should be the same number of

RI
columns. When both x and y are arrays the operation is element by element. In this case, x and y
should be the same size. See below for each example:

Example: Multiply an array by a scalar

[77]: X = [Link]([[1,2],[3,4]])
Y = 5
V = X * Y
print("X =",X)
print("Y =",Y)
print("V = X * Y")
print(" = ",V)
BA
AK
X = [[1 2]
[3 4]]
Y = 5
V = X * Y
= [[ 5 10]
[15 20]]

Example: Multiply each element of a vector array by the elements of another array

[78]: X = [Link]([[1,2,3],[5,3,4]])
A.

Y = [Link]([7,8,9])
V = X * Y
print("X =",X)
print("Y =",Y)
print("V = X * Y")
print(" = ",V)

X = [[1 2 3]
[5 3 4]]
Y = [7 8 9]

33
 V = X * Y
= [[ 7 16 27]
[35 24 36]]

Example: Multiply each element of an array by the elements of another array

[79]: X = [Link]([[1,2,3],[5,3,4]])
Y = [Link]([[7,8,9],[0,6,3]])

I
V = X * Y
print("X =",X)

AR
print("Y =",Y)
print("V = X * Y")
print(" = ",V)

X = [[1 2 3]
[5 3 4]]
Y = [[7 8 9]
[0 6 3]]
V = X * Y
KB
= [[ 7 16 27]
[ 0 18 12]]

Matrix Multiplication Matrix Multiplication is accessible with @. If X is a matrix with N rows

and M columns (N by M) and Y is M by L, Z = X @ Y produces an array with V[i,j] = sum X[i,m]
Y[m,j], where m=1 to M and X[i,m] denotes the row i and column m value of the matrix X.

[80]: V = X @ [Link](Y)
print("X =", X)
A

print("Y transposed =", [Link](Y))

print("V = X Matrix-multiply Y")

print(" = ", V)

X = [[1 2 3]
A.

[5 3 4]]
Y transposed = [[7 0]
[8 6]
[9 3]]
V = X Matrix-multiply Y
= [[50 21]
[95 30]]

34
 Recall Matrix multiplication is the sumproduct of each row of X by each column of Y. For example,

V [0, 0] = X[0, 0] × Y [0, 0] + X[0, 1] × Y [1, 0]

= 1×5 + 2×7
= 19

V [0, 1] = X[0, 0] × Y [0, 1] + X[0, 1] × Y [1, 1]

RI
etc

Sum, Product and Difference of an Array Similar to MS-Excel, in NumPy we have sum()
and product() functions that allow us to get the sum/product of all elements of an array. Let’s see
some examples:

Let’s consider the array Z that we created above:

[81]: print("Z = ", Z)

Z =
[
[
[
[[
3.4
5.
7.1
1.
4.6
6.
8.9
2.5
10.2
15.
21.3
3.
13.8
18.
7.5
-6.8
-10.
-2. BA
-9.2]
-12. ]
26.7 -14.2 -17.8]]
-5. ]
AK
[82]: print([Link](Z)) # sum all elements of the array (output = 1 element)

77.0

[83]: print([Link](Z, axis = 0)) # sum the elements of each column, (output = 4␣
,→ elements)

[ 16.5 22. 49.5 66. -33. -44. ]

[84]: print([Link](Z, axis = 1)) # sum the elements of each row (output = 3 elements)
A.

[ 7. 16. 22. 32.]

nansum() is identical to sum(), except that NaNs are ignored.

Numpy also has a function to create the cumulative sum, called cumsum. This is very useful for
financial applications, where each row is added with the above ones. You need to tell NumPy on
which axis you want it to run the summation:

[85]: # cumulative sum, across the rows

print([Link](Z,axis = 0))

35
 [[ 1. 2.5 3. 7.5 -2. -5. ]
[ 4.4 7.1 13.2 21.3 -8.8 -14.2]
[ 9.4 13.1 28.2 39.3 -18.8 -26.2]
[ 16.5 22. 49.5 66. -33. -44. ]]

[86]: # cumulative sum, across the columns

print([Link](Z,axis = 1))

[[ 1. 3.5 6.5 14. 12. 7. ]

RI
[ 3.4 8. 18.2 32. 25.2 16. ]
[ 5. 11. 26. 44. 34. 22. ]
[ 7.1 16. 37.3 64. 49.8 32. ]]

prod() and cumprod() behave similarly to sum() and cumsum() except that the product and cu-
mulative product are returned.

Numpy does not have a similar function to MS-Excel’s sumproduct. There are several alternative

[87]: W = [Link](X * Y, axis=1)

print('X =', X)
print('Y =', Y)
BA
ways to implement this functionality. For instance, you may frist multiply the two arrays and then
sum them over the columns as below:
AK
print('The sumproduct of X and Y = ', W)

X = [[1 2 3]
[5 3 4]]
Y = [[7 8 9]
[0 6 3]]
The sumproduct of X and Y = [50 30]

This is a useful feature for portfolio constructions. See the example after this section

diff() generates the difference between elements of an array. This is useful when you want to get
A.

how much a company has grown from the previous year, for example, or compute the stock returns.
For instance,

Q= [Link](Z, 1, axis=0) will give the 1st difference between elements of array Z. In this case,
first row of Q is equal to the second row of Z minus the first row of Z. i.e.: Q[0, ] = Z[1, ] − Z[0, ]
and so on

[88]: Q= [Link](Z, 1, axis=0)

print(Q)

36
 [[ 2.4 2.1 7.2 6.3 -4.8 -4.2]
[ 1.6 1.4 4.8 4.2 -3.2 -2.8]
[ 2.1 2.9 6.3 8.7 -4.2 -5.8]]

[89]: # Double difference, column by column

print([Link](Z, 2, axis=0))

I
[[-0.8 -0.7 -2.4 -2.1 1.6 1.4]
[ 0.5 1.5 1.5 4.5 -1. -3. ]]

AR
Other Simple Math Functions for Array Similar to MS-Excel:

• exp() returns the element-by-element exponential (e ˆ x ) for an array.

• log() returns the element-by-element natural logarithm (ln(x)) for an array.
• log10() returns the element-by-element base-10 logarithm (log10 (x )) for an array.
• sqrt() returns the element-by-element square root of x
• abs() returns the element-by-element absolute value for an array.
KB
5.3.6 Example: Stock Returns

The stock price of the Coca-Cola company (KO) from 2016 to 2019, from Yahoo Finance, is shown
below. How much is its annual stock return in this period?

Date Open High Low Close Adj Close

2016-12-30 $ 41.69 $ 41.84 $ 41.35 $ 41.46 $ 37.58

2017-12-29 $ 45.70 $ 46.18 $ 45.69 $ 45.88 $ 42.98
2018-12-31 $ 47.49 $ 47.54 $ 46.96 $ 47.35 $ 45.90
A

2019-12-31 $ 55.20 $ 55.38 $ 54.98 $ 55.35 $ 55.35

Recall the rate of return of a stock during a period is calculated from:

Pt + Dt − Pt−1
Rt =
Pt−1
A.

where, Pt and Dt are the closing price and dividends of the stock on period t, respectively. Note that
the above equation is the re-organized form of the time value of money equation F V = P V (1 + R)T ,
where the future value is the sum of the price of stock and its dividends and PV is the last period
price of the stock.

In Yahoo Finance data, the Adj Close column incorporates the effect of dividends and other dis-
tributions. It is also incorporates the effects of share splits. Therefore, we can simply use that
column.

37
 For example, the rate of return in 2017 (i.e., from December 2016 to December 2017), we have

42.98 − 37.58
R2017 =
37.58
= 0.1437

Instead of calculating this, year by year, you can calculate this simply by replicating the diff
functionality in numpy:

I
[90]: Price_KO = [Link]([37.58,42.98,45.90,55.35])

AR
Return_KO = [Link](Price_KO)/Price_KO[:-1]
print("KO returns(in %) = ",[Link](Return_KO*100,2))

KO returns(in %) = [14.37 6.79 20.59]

This approach can be used with multiple stocks. Simply store stock prices in different columns of
a matrix and use the same lines of code.

In finance, we might use log-returns, based on continuous time compounding F V = P V e rT :

KB
P + D 
t t
Rt = ln = ln(Pt + Dt ) − ln(Pt−1 )
Pt−1

Log-returns have some convenient theoretical and computational characteristics but do not precisely
depict period-to-period performance, especially in longer horizons, such as a year. We can calculate
them in python using the diff and logfunctionalities of numpy:

[91]: return_KO = [Link]([Link](Price_KO))

print([Link](return_KO*100,2))
A

[13.43 6.57 18.72]

5.3.7 Example: Portfolio Construction

A portfolio is a collection (basket) of financial investments like stocks, and bonds. It is commonly
identified by its ingredients (what is in the basket) and weight of them (portion of each item in the
A.

basket).

For instance, suppose you construct a portfolio with initial wealth of $ 4967.80 in December 2017.
You decide to buy 5 shares of IBM Corp., 100 shares of Tesla Inc., 30 shares of Apple Inc., and 10
shares of Microsoft Corp. stocks.

The price of these stocks in December 2017 were, respectively $ 153.42, $ 20.76, $ 42.31, $ 85.54.
Meaning your portfolio (P) is:

38
[92]: initial_wealth = 4967.80
prices = [Link]([153.42,20.76,42.31,85.54])
nb_shares = [Link]([5,100,30,10])
weights = nb_shares * prices / initial_wealth
print([Link](weights*100,2))

[15.44 41.79 25.55 17.22]

I
That is, you invested about 15% of your portfolio money in IBM Corp., 41% in Tesla Inc., 25% in
Apple Inc., and 17% in Microsoft Corp. stocks.

AR
n o
P = (IBM ; 15.44%) , (T SLA; 41.79%) , (AAP L; 25.55%) , (M SF T ; 17.22%)

Note that the sum of the portfolio weights is 1 (or 100%)

[93]: print([Link](weights))

1.0

The returns on these stocks, including dividends and other distributions, in years 2018, 2019, 2020,
KB
and 2021 are shown below. What is the return on your portfolio in each year?

Year RIBM RT SLA RAAPL RMSFT

2018 -0.22 0.07 -0.05 0.21

2019 0.24 0.26 0.89 0.58
2020 -0.01 7.43 0.82 0.43
2021 0.17 0.50 0.35 0.52
A

Recall, the return of a portfolio (Rp ) is the weighted average of the returns of the stocks (Ri ) in
that portfolio.
X
Rp = wi × Ri
i

where wi is the weight of stock i in the portfolio.

A.

So, the portfolio return, for example, for year 2018 is:

Rp = 0.1534 × −0.22 + 0.4152 × 0.07 + 0.2539 × −0.05 + 0.1711 × 0.21

= 0.0187

39
 Instead of calculating this, year by year, you can calculate this simply by replicating the sumproduct
functionality:

w = [0.10, 0.50, 0.25, 0.15]

 
−0.22 0.07 −0.05 0.21

I
 
 0.24 0.26 0.89 0.58
R=
−0.01

 7.43 0.82 0.43

AR
0.17 0.50 0.35 0.52

[94]: Returns = [Link]([[-0.22,0.07,-0.05,0.21],[0.24,0.26,0.89,0.58],[-0.01,7.43,0.

82,0.43],[0.17,0.5,0.35,0.52]])
,→

Portfolio_return = [Link](weights * Returns, axis=1)

print("The performance of the portfolio (in %) for years 2018, 2019, 2020, and␣
2021 was")
,→
KB
print([Link](Portfolio_return*100,2))

The performance of the portfolio (in %) for years 2018, 2019, 2020, and 2021 was
[ 1.87 47.3 338.69 41.42]

Of course, an alternative to approach is to calculate the returns of a portfolio from its value. That
is, first calculate the value of the find at the end of each period, then find its rate of return, period
over period.
X
F undV alue = N bSharesi × (P ricei + Dividendi )
A

i
F undV aluet − F undV aluet−1
Rp,t =
F undV aluet−1

5.3.8 Descriptive Statistics

Python has max(), min(), sum() functions to generate some basic descriptive statistics. For other
A.

descriptive statistics, such as mean, median, standard deviation , variance, . . . we can use functions
in NumPy Library:

• mean() and nanmean(). in MS-Excel, we have the AVERAGE() function

• median(), in MS-Excel, we have the MEDIAN() function
• std() and nanstd(). in MS-Excel we have STDEV.P() and STDEV.S()
• var() and nanvar(). in MS-Excel we have VAR.P() and VAR.S()

These functions, similar to the sum() function allow you to choose the Axis or axes along which the

40
 statistics are computed. The default is to compute the flattened array.

These functions can work on LISTs and NUMPY ARRAYs.

Example:

[95]: # input variables:

nlist = [2, 4, 13, 3, 7, 8, 5]
rlist = [3.14, 2.71, -8.43, 5.25, 10.11, -23.78, 44.45]

RI
print("the maximum of nlist is",max(nlist))
print("the minimum of nlist is",min(nlist))
print("sorted nlist is:",sorted(nlist), \
", the min is the first item:",sorted(nlist)[0],\
", and the max is the last item of the list:", sorted(nlist)[-1])
print("the sum of nlist is",sum(nlist))

the maximum of nlist is 13

the minimum of nlist is 2
BA
sorted nlist is: [2, 3, 4, 5, 7, 8, 13] , the min is the first item: 2 , and the
max is the last item of the list: 13
the sum of nlist is 42
AK
[96]: avg_nlist = [Link](nlist)
print("The average of nlist is ",avg_nlist)

avg_rlist = [Link](rlist)
print("The average of rlist is ",avg_rlist)

The average of nlist is 6.0

The average of rlist is 4.778571428571429

[97]: median_nlist = [Link](nlist)

A.

print("The median of nlist is",median_nlist)

# alternative way to find the median is:
print("sorted nlist is :",sorted(nlist), \
"and the median is the middle item:",sorted(nlist)[len(nlist)//2] )

The median of nlist is 5.0

sorted nlist is : [2, 3, 4, 5, 7, 8, 13] and the median is the middle item: 5

[98]: print("The standard deviation of rlist is",[Link](rlist))

print("The variance of rlist is",[Link](rlist))

41
 The standard deviation of rlist is 19.27059523772752
The variance of rlist is 371.3558408163265

System of Equations

• solve() solves the system of linear equations,

Example Consider the following problem where we have 3 equations and 3 unknowns (a, b, c)

I
11 = a1+b2+c4

AR
12.5 = a 3.4 + b 4.4 + c 9.1
3.3 = a 2.2 + b 1.9 − c 0.5

To solve this, first define arrays Y , as vector of left-hand side yi variables, and X as the matrix of
right-hand side xij variables.
y1 = a x11 + b x21 + c x31
y2 = a x21 + b x22 + c x23
y3 = a x31 + b x32 + c x33
KB
where,

 
11
Y = 12.5
 

3.3

 
1 2 4
X = 3.4 4.4 9.1 
 
A
2.2 1.9 −0.5

Then use the solve function from the linear algebra package of NumPy ([Link]). As numerical,
below I solve the above system of equations with the following values for Y and X:

[99]: X = [Link]([[1,2,4],[3.4,4.4,9.1],[2.2,1.9,-0.5]])
Y = [Link]([[11],[12.5],[3.3]])
A.

[Link](X,Y)

[99]: array([[-9.47696477],
[12.42276423],
[-1.09214092]])

42
 5.3.9 Random Variables

In Python, you can generate random values with different distributions using the functions in the
NumPy library ( for more advanced random number generators you may want to use the functions
in the RANDOM library.)

Most often we use 4 types of random variables:

• Uniform Distribution (real numbers)

I
This is the case if you want to generate a random number in a range (between a and b) with equal
probability. The function rand() generates uniform random variables between [0, 1), including 0

AR
and excluding 1, each time you call it. Example:

[100]: [Link]()

[100]: 0.2362809224088045

You may create several random numbers by calling this function once. The function can create an
array of the given shape, for example: a 3-by-2 matrix
KB
[101]: [Link](3,2)

[101]: array([[0.69588315, 0.45621073],

[0.5987936 , 0.79513309],
[0.30860584, 0.70877425]])

• Uniform distribution (integer numbers)

This is the case if you want to generate a random INTEGER number in a range (between a and
b) with equal probability. One way to generate such numbers is to use the randint function. For
A

example, you may simulate rolling a six-faced dice for 10 times:

[102]: [Link](1, high=6+1,size=10)

[102]: array([6, 3, 4, 4, 4, 6, 6, 5, 4, 2])

• Uniform distribution (from a set of options):

A.

For example, if you want to simulate flipping a coin (Heads or Tails):

[103]: coin =["Head","Tail"]

[Link](coin, size=10) # this will replicate 10 coin flipping

[103]: array(['Tail', 'Head', 'Head', 'Head', 'Tail', 'Head', 'Head', 'Head',

'Tail', 'Head'], dtype='<U4')

43
 You can modify the chance (probability) of the Head or Tail events. Below I set the probability of
a Head = 70% (therefore we get more Head outcomes)

[104]: [Link](coin, size=10, p=[0.70, 0.30])

[104]: array(['Head', 'Head', 'Head', 'Tail', 'Head', 'Tail', 'Head', 'Tail',

'Tail', 'Head'], dtype='<U4')

I
• Normal distribution (bell curve)

Most random variables in the world are normally distributed (following central limit theorem). A

AR
normally distributed random variable is defined with its mean (the center of the bell curve) and
its standard deviation (the dispersion around the mean of the bell curve). The randn() function
generates a standard normal random value with mean 0 and standard deviation 1. The output of
this function ranges from -inf. to +inf.

[105]: [Link]()

[105]: -0.4241437027639983
KB
Similar to the rand() function, you can generate an array of random normal variables:

[106]: [Link](3,2)

[106]: array([[-1.28996663, -1.65721063],

[ 0.19596404, 1.12607952],
[ 1.49506861, 0.56814519]])

Example: Let’s experiment with random generating functions, as an example. The following
A

example builds a sentence using various parts of speech. It randomly chooses words from a list by
using [Link](). We have used a method of the string data type to capitalize the first
letter of the sentence.

[107]: verbs =["goes","cooks","shoots","faints","chews","screams"]

nouns =["bear","lion","mother","baby","sister","car","bicycle","book"]
A.

adverbs =["handily","sweetly","sourly","gingerly","forcefully","meekly"]
articles=["a","the","that","this"]

def sentence():
article = [Link](articles)
noun = [Link](nouns)
verb = [Link](verbs)

44
 adverb = [Link](adverbs)

our_sentence = article + " " + noun + " " + verb + " " + adverb + "."
our_sentence = our_sentence.capitalize()

print(our_sentence)

I
[108]: # call the function to print the random sentence:
sentence()

AR
That baby faints handily.

Note: The numbers that these functions generate are from a pseudorandom sequence of numbers.
These numbers appears to be statistically random, despite having been produced by a completely
deterministic and repeatable process. You may set the sequence, using [Link](n) where
n is an integer number of your choice, to get the same number each time you run the random
generator function. This way, the results of your code can be reproduce at a later time.

Example: Without setting the seed generate a random number from a normal distribution. Repeat
KB
this experiment one more time and print the results. The two numbers are not identical (each look
random). Now, set the seed to an arbitrary number, say n = 5. Then draw a random number from
the same distribution for one more times. Repeat this for another time (set the seed and draw the
random number). The two numbers are identical (although each is a random number)

[109]: print([Link]())
print([Link]())

0.26556179713890227
A

-0.7231357148553637

[110]: [Link](5)
print([Link]())
[Link](5)
print([Link]())
A.

0.44122748688504143
0.44122748688504143

45
 5.4 NumPy-Financial
The NumPy-Financial Python package is a collection of Time Value of Money functions. These
functions were copied to this package from version 1.17 of NumPy.

You need to first install this library as it is not included in the main package during the installation.
Launch Anaconda Prompt (from the Start menu of Windows or the Mac equivalent) then type/run:
pip install numpy-financial. It will download and install it on your machine.

I
The functions in this library are:

[111]: import numpy_financial as npFin

AR
dir(npFin)

[111]: ['__builtins__',
'__cached__',
'__doc__',
'__file__',
'__loader__',
'__name__',
KB
'__package__',
'__path__',
'__spec__',
'__version__',
'_financial',
'fv',
'ipmt',
'irr',
A
'mirr',
'nper',
'npv',
'pmt',
'ppmt',
'pv',
A.

'rate']

These are similar to the time value of money functions in MS-Excel. Let’s see some examples:

• npv(rate, values)

This is almost identical to the NPV(rate,value1,[value2],...) function in MS-Excel to calculate

the net present value of a series of future cash flows, with this distinction that the first input cash
flow is for today. In MS-Excel, the first input cash flow is for the next period and the user needs to

46
 manually subtract the initial cash flow to calculate the net present value

Example suppose you are asked to calculate the NPV of a project with initial investment of $ 10
Million. The cash flows in next 5 years are -2,1,3,10,20 million dollars. Your required rate of return
is at least 10%:

[112]: DiscountRate = 0.10

CashFlow = [Link]([-10,-2,1,3,10,20])*1e6

I
NPV = [Link](DiscountRate,CashFlow)

AR
print('Net Prsent Value is ${:,.2f}'.format(NPV))
# Notice that I am using a formatting command
# this changes the formatting of the output to currency, with 2 decimals.

Net Prsent Value is $10,510,769.88

• pv(rate, nper, pmt, fv=0, when='end')

Similarly, pv() calculates the present value of future cash flows. The cash flows could be an annuity
KB
payments (pmt) that are paid at certain dates (such as loan payments), or could be a lump sum
payment (fv) paid at maturity (such as the principal of a zero coupon bond), or could be both (such
as corporate bonds).

Similar to the MS-Excel PV function, the argument when defines if it is an Annuity due, where
the payments are at begins of each period (such as rent and insurance payments), or Ordinary
Annuity, where payments are paid at the end of each period (such as mortgage and loan payments).
Choose when = {'begin', 1} or when = {'end', 0} depending the application. The default value
is end. The default value of fv is 0.
A

Example Suppose you are leasing a car. The cash value of the car is $ 25,000 (after tax and fees).
The car dealer offers you the following: monthly payments of $ 400 for 3 years, with an interest rate
of 1.99% APR. After 3 years, you have the option to pay $ 15,000 and own the car. Is this a good
deal?
A.

[113]: PV = [Link](0.0199/12, 3*12, 400, 15000, 1)

print('Present Value of his offer is ${:,}'.format(round(PV,2)))
# this changes the formatting of the output to currency, with 2 decimals.

Present Value of his offer is $-28,121.92

That is, if you accept the dealer’s offer, including the time value of money, you are paying more
than $ 28,000 for a car which is worth only $ 25,000‘. So this is a bad deal.

47
 • fv(rate, nper, pmt, pv, when='end') calculates the future value of a stream of cash flows.

• rate(nper, pmt, pv, fv, when='end', guess=None, tol=None, maxiter=100), finds the
required rate of return. For this class, you can ignore the following arguments: guess, tol,
and maxiter.

• nper(rate, pmt, pv, fv=0, when='end') computes the number of periodic payments of an
annuity.

I
• pmt(rate, nper, pv, fv=0, when='end') computes the payment against loan principal plus
interest. Similarly, ipmt() computes the interest portion of a payment and ppmt() computes

AR
the payment against loan principal.

• irr(CashFlows) calculates the internal rate of return for a stream of future cash flows (the
first element is the initial investment with a negative sign). mirr() is for modified IRR
calculation. KB
A
A.

48
 5.5 Pandas Library
Pandas library is an increasingly important component of the Python scientific stack. It is a high-
performance package that provides a comprehensive set of data structures for working with data.
pandas also provides high-performance, robust methods for importing from and exporting to a wide
range of formats, such as xlsx and csv files. Data structures in Pandas allow for labeling columns
and rows, which is not available in NumPy. Moreover, it allows for relational-database operations
(such as join), which are provided in MS-Access.

I
[114]: import pandas as pd

AR
5.5.1 Data Structures

The most important data structures in pandas are Series and DataFrames. Series are the equivalent
of 1-dimensional arrays. DataFrames are collections of Series and so are 2-dimensional (like MS-
Excel spreadsheets).

Series Series are the primary building block of the data structures in pandas, and in many ways
a Series behaves similarly to list and can have both number and text.
KB
[115]: # Panda Series
s = [Link]([5.4, 6.6, 53, 'book', 4.5,"apple",1])
s

[115]: 0 5.4
1 6.6
2 53
3 book
A

4 4.5
5 apple
6 1
dtype: object

First, notice that the data type of s is shown as object. This means that the pandas series has
A.

elements with a type of string. Second, as we see in the printed values in this example, pandas has
automatically generated a label (index) for the series (the first column above) using the sequence 0,
1, . . . . This is similar to the row number in MS-Excel. You can use these indexes when printing
or accessing the data. for example, print items with index 0 and 3:

[116]: s[[0,3]]

49
[116]: 0 5.4
3 book
dtype: object

DataFrame DataFrames collect multiple series in the same way that a spreadsheet collects mul-
tiple columns of data. A DataFrame is composed of Series and each Series has its own data type,
and so not all DataFrames are representable as homogeneous NumPy arrays.

I
A number of methods are available to initialize a DataFrame. The simplest method uses a homo-
geneous NumPy array.

AR
[117]: a = [Link]([[7.0,8,9],[3,4,5]])
df = [Link](a)
df

[117]: 0 1 2
0 7.0 8.0 9.0
1 3.0 4.0 5.0
KB
A DataFrame contains column labels and row labels (similar to row number and column headings
in MS-Excel). When none are provided, the numeric sequence 0, 1, . . . is used. In the above
example, we have three columns that are labeled 0, 1 and 2. There are also two rows labeled as 0
and 1. Labels are useful when working with large DataFrames, where you do not need to remember
which column stores what values. We can assign column and row names by columns and index
commands:

Example Going back to our example for the grades in Section 4 of Commerce 3FD3:
A
[118]: Section_4_marks = [
[100,0,0,100,50,80, 65,78, 80],
[100,100,100,100,100,80, 75,88, 100],
[100,0,0,100,50,80, 65,78, 80],
[0,0,0,100,50,50, 45,38, 20]
]
A.

df = [Link](Section_4_marks, columns=['A_1', 'A_2', 'A_3','A_4','A_5',\

'Mid_1','Mid_2','Final','Participation'],\
index=['Sarah', 'Rose', 'Ben','John'])

df

50
[118]: A_1 A_2 A_3 A_4 A_5 Mid_1 Mid_2 Final Participation
Sarah 100 0 0 100 50 80 65 78 80
Rose 100 100 100 100 100 80 75 88 100
Ben 100 0 0 100 50 80 65 78 80
John 0 0 0 100 50 50 45 38 20

Then it is easier to access columns or rows of the DataFrame. You can select a column by giving
its name in [], for example:

I
[119]: df['Mid_1']

AR
[119]: Sarah 80
Rose 80
Ben 80
John 50
Name: Mid_1, dtype: int64

If you want to access a list of columns, say both Mid_1 and Mid_2, put them in a list. e.g:
KB
[120]: df[['Mid_1','Mid_2']]

[120]: Mid_1 Mid_2

Sarah 80 65
Rose 80 75
Ben 80 65
John 50 45

To access a row, we can use the .loc[row name]:

A

[121]: [Link]['Sarah']

[121]: A_1 100

A_2 0
A_3 0
A_4 100
A.

A_5 50
Mid_1 80
Mid_2 65
Final 78
Participation 80
Name: Sarah, dtype: int64

51
[122]: [Link]['Sarah','Final']

[122]: 78

If you don’t know the index of an item, you can also access it via its row or column number via the
.iloc[] feature:

[123]: [Link][1,0]

I
[123]: 100

AR
or alternatively

[124]: [Link][1]['A_1']

[124]: 100

You can define a new column for the DataFrame using ['new column name']. For example, below
I calculate the total grade for each student:

[125]: df['total_grade'] = (df['A_1']+ df['A_2']+ df['A_3']+df['A_4']+df['A_5'])*0.02 \

KB
+ (df['Mid_1'] + df['Mid_2'])*0.20 \
+ df['Final'] * 0.40 \
+ df['Participation']*0.10
df

[125]: A_1 A_2 A_3 A_4 A_5 Mid_1 Mid_2 Final Participation total_grade
Sarah 100 0 0 100 50 80 65 78 80 73.2
Rose 100 100 100 100 100 80 75 88 100 86.2
A
Ben 100 0 0 100 50 80 65 78 80 73.2
John 0 0 0 100 50 50 45 38 20 39.2

As we see above, you may simply add/subtract columns or multiply with a constant number. You
may multiply/divide Series or dataFrame similar to numpy arrays; These operations will be executed
element by element, but considering the index/column names. If you are multiply two pandas Series,
they should have the same indexes. For instance, if we want to multiply grades of assignment 5 and
A.

midterm 2, we can do so like this:

[126]: df['A_5']*df['Mid_2']

[126]: Sarah 3250

Rose 7500
Ben 3250
John 2250

52
 dtype: int64

Here, both df['A_5'] and df['Mid_2'] are pandas Series and have the same index labels (which
are the student names). Now, let’s define a new Series and call it S2, using the same values of
column Mid_2 but without defining the index labels. E.g.,

[127]: S2 = [Link]([65,75,65,45])
print(S2)

I
df['A_5']*S2

AR
0 65
1 75
2 65
3 45
dtype: int64

[127]: 0 NaN
1 NaN
2 NaN
KB
3 NaN
Ben NaN
John NaN
Rose NaN
Sarah NaN
dtype: float64

As we see, this multiplication for Pandas is ambiguous; the two Series do not share index labels.
A
Therefore, the output is not the element by element multiplications but a combination of the index
labels. In this case, you may convert S2 to a numpy array and then multiply.

[128]: df['A_5']*S2.to_numpy()

[128]: Sarah 3250

Rose 7500
A.

Ben 3250
John 2250
Name: A_5, dtype: int64

When multiplying/dividing Pandas DataFrame, they have to have the same column names. For
instance, let’s scale the grade of the class with respect to Rose’s marks:

[129]: df/[Link]['Rose']

53
[129]: A_1 A_2 A_3 A_4 A_5 Mid_1 Mid_2 Final Participation total_grade
Sarah 1.0 0.0 0.0 1.0 0.5 1.000 0.866667 0.886364 0.8 0.849188
Rose 1.0 1.0 1.0 1.0 1.0 1.000 1.000000 1.000000 1.0 1.000000
Ben 1.0 0.0 0.0 1.0 0.5 1.000 0.866667 0.886364 0.8 0.849188
John 0.0 0.0 0.0 1.0 0.5 0.625 0.600000 0.431818 0.2 0.454756

There are also sub(),add(), div(), and mul() functions in pandas, for subtraction, addition, divi-
sion, and multiplication, where you would have more flexibility in executing these operations.

I
If you want to access the names of the columns:

AR
[130]: [Link]

[130]: Index(['A_1', 'A_2', 'A_3', 'A_4', 'A_5', 'Mid_1', 'Mid_2', 'Final',

'Participation', 'total_grade'],
dtype='object')

And if you want to access the names of the rows (i.e. indexes)

[131]: [Link]
KB
[131]: Index(['Sarah', 'Rose', 'Ben', 'John'], dtype='object')
A
A.

54
 Merge and Join DataFrames Pandas’ merge() and join() fucntions provide SQL-like opera-
tions for merging the DataFrames using row labels or the contents of columns. In MS-Excel, you
can use the XLOOKUP function (or other similar lookup functions) for merging.

The primary difference between the two is that merge() by default uses column contents while
join() defaults to using index labels. Both commands take a large number of optional inputs. The
important keyword arguments are:

• how, which must be one of ‘left’, ‘right’, ‘outer’, ‘inner’ describes which set of indices to use
when performing the join.

RI
– ‘left’ uses the indices of the DataFrame that is used to call the method and ‘right’ uses
the DataFrame input into merge or join.

– ‘outer’ uses a union of all indices from both DataFrames and ‘inner’ uses an intersection
from the two DataFrames.

• on is a single column name or list of column names to use in the merge. on assumes the names

BA
are common. If no value is given for on or left_on/right_on, then the common column
names are used.

– left_on and right_on allow for a merge using columns with different names. When
left_on and right_on contain the same column names, the behavior is the same as on.

• left_index and right_index indicate that the index labels are the join key for the left and
AK
right DataFrames.

Below, I try to explain each of these merges. First, I define two arbitrary DataFrames, called
df_left and df_right.

[132]: df_left = [Link]([['Sarah',100],['Rose',50],['Ben',75],['Moe',25]],

columns=['Name','M_1'])
print('df_left:')
print(df_left)
A.

df_right = [Link]([['Sarah',80],['Rose',30],['John',100]],
columns=['Name','M_2'])
print('df_right:')
print(df_right)

df_left:
Name M_1
0 Sarah 100
1 Rose 50
2 Ben 75

55
 3 Moe 25
df_right:
Name M_2
0 Sarah 80
1 Rose 30
2 John 100

Let’s merge the two DataFrames, by putting the values of df_right beside the values of df_left,

I
where the values of items in column Name of the two DataFrame are the same (since column Name
is shared, it will not be repeated). This is called the inner join:

AR
[133]: df_left.merge(df_right,on='Name', how='inner') # the default value for how =␣
'inner'. you may exclude this term too
,→

[133]: Name M_1 M_2

0 Sarah 100 80
1 Rose 50 30

Notice that by the inner join, we lost rows 2 and 3 of df_left and row 2 of df_right, because
KB
their values in column Name are not shared.

left join allows us to keep the left DataFrame as-is and only add the values of df_right where
there is a match. For missing values pandas puts NaN if cannot find the match

[134]: df_left.merge(df_right,on='Name', how='left')

[134]: Name M_1 M_2

0 Sarah 100 80.0
1 Rose 50 30.0
A

2 Ben 75 NaN
3 Moe 25 NaN

Similarly, with the right join, we keep the values of df_right and add the values of df_left if
there is a match.
A.

[135]: df_left.merge(df_right,on='Name', how='right')

[135]: Name M_1 M_2

0 Sarah 100.0 80
1 Rose 50.0 30
2 John NaN 100

Lastly, we have the outer join, which merges all values and puts NaN if cannot find the match

56
[136]: df_left.merge(df_right,on='Name', how='outer')

[136]: Name M_1 M_2

0 Sarah 100.0 80.0
1 Rose 50.0 30.0
2 Ben 75.0 NaN
3 Moe 25.0 NaN
4 John NaN 100.0

I
[Link]() and [Link]() are very similar. With [Link]() function merging is done on index

AR
of the dataFrames. This is very useful for merging timeseries dataFrames.

[137]: df_left = [Link]([100,50,75],

columns=['M_1'],index = ['Sarah','Rose','Ben'])
print('df_left:')
print(df_left)

df_right = [Link]([80,30,100],
columns=['M_2'],index = ['Sarah','Rose','John'])
KB
print('df_right:')
print(df_right)

df_left:
M_1
Sarah 100
Rose 50
Ben 75
A

df_right:
M_2
Sarah 80
Rose 30
John 100
A.

[138]: df_left.join(df_right, how='outer')

[138]: M_1 M_2

Ben 75.0 NaN
John NaN 100.0
Rose 50.0 30.0
Sarah 100.0 80.0

57
 5.5.2 Importing and Exporting Data via Panda

All of the data readers in pandas load data into a pandas DataFrame, which if need be you can
transfer to a NumPy array. In practice, the DataFrame is much more useful since it includes useful
information such as column names read from the data source.

Import Excel files Excel files, both 97/2003 (xls) and 2007/10/13 (xlsx), can be imported
using read_excel(). Two inputs are required to use read_excel(), the FILE_NAME and the

I
SHEET_NAME containing the data. In this example, pandas makes use of the information in the
Excel workbook that the first column contains dates and converts these to datetimes.

AR
[139]: TSLA_prices = pd.read_excel('VOO_TSLA_BTC_prices.xlsx',sheet_name='TSLA')
TSLA_prices

[124]: Date Open High Low Close Adj Close Volume

0 2010-06-29 3.800000 5.000000 3.508000 4.778000 4.778000 93831500
1 2010-06-30 5.158000 6.084000 4.660000 4.766000 4.766000 85935500
2 2010-07-01 5.000000 5.184000 4.054000 4.392000 4.392000 41094000
3 2010-07-02 4.600000 4.620000 3.742000 3.840000 3.840000 25699000
KB
4 2010-07-06 4.000000 4.000000 3.166000 3.222000 3.222000 34334500
... ... ... ... ... ... ... ...
2780 2021-07-15 658.390015 666.140015 637.880005 650.599976 650.599976 20209600
2781 2021-07-16 654.679993 656.700012 642.200012 644.219971 644.219971 16339800
2782 2021-07-19 629.890015 647.200012 621.289978 646.219971 646.219971 21297100
2783 2021-07-20 651.989990 662.390015 640.500000 660.500000 660.500000 15442700
2784 2021-07-21 659.609985 664.859985 650.289978 655.289978 655.289978 13910800

[2785 rows x 7 columns]

A

Other notable keyword arguments while importing include:

• header, an integer indicating which row to use for the column labels. The default is 0 (top)
row, and if skiprows is used, this value is relative.
• skiprows, typically an integer indicating the number of rows at the top of the sheet to skip
before reading the file. The default is 0.
• skip_footer, typically an integer indicating the number of rows at the bottom of the sheet
A.

to skip when reading the file. The default is 0.

• index_col, an integer or column name indicating the column to use as the index. If not
provided, a basic numeric index is generated.
• usecols=[0,1,2,3], only import columns 1 to 4.

Note: If the file that you want to import (read in) to Python is not located in
your working directory, you need to give the full path of the location of the file, e.g.,
C://Users//akbara23//Desktop//VOO_TSLA_BTC_prices.xlsx

58
 Import CSV and other formatted text files Comma-separated value (CSV) files can be read
using read_csv(). CSV files are text-based files and do not have Excel meta data (formatting
embedded in cells; for example, they cannot have several sheets or cell formatting). Because of this,
for many software programs they are easier to connect to, compared to Excel. In Windows, CSV
files can be opened by NotePad or MS-Excel. With the latter, CSV files might look like an excel
file but note that unlike excel files, CSV files cannot have multiple sheets.

[140]: TSLA_csv_data = pd.read_csv('TSLA_prices.csv')

I
Export using Pandas Writing data from a Series or DataFrame is much simpler since the starting

AR
point (the Series or the DataFrame) is well understood by pandas. While the file writing methods
all have a number of options, most can safely be ignored.

[141]: # df.to_excel('FinancialModeling_marks.xlsx')
# df.to_excel('FinancialModeling_marks.xlsx', sheet_name='Section_4')
# df.to_csv('FinancialModeling_marks.csv')

These commands create a new excel file. If you want to write in several sheets of an excel file, you
need to take a few extra steps. First, you need to create an empty excel file and open it in order
KB
to write in it. Then you write your desired DataFrames in each excel sheet, as shown above. Then
you save and close the excel file.

[142]: writer = [Link]('[Link]', engine='xlsxwriter')

df.to_excel(writer, sheet_name='Section_4')
TSLA_csv_data.to_excel(writer, sheet_name='TSLA_csv_data')
A
[Link]()

5.5.3 Manipulating Data in Pandas

Recall, to call a function from NumPy or base Python we type FUNCTION_NAME(in1,in2,...). In

pandas, we first type the input DataFrame (or on a set of its columns/rows), then call the function,
i.e. [Link](). Below, I introduce a few useful functions:
A.

Summary If you want to have a preview of your data you can use head() or tail() functions
which return the first and last 5 rows of a DataFrame, respectively. This gives you a sense of the
data at a glance and usually is the first step:

[143]: TSLA_prices = pd.read_excel('VOO_TSLA_BTC_prices.xlsx',sheet_name = 'TSLA')

TSLA_prices.head()

59
[143]: Date Open High Low Close Adj Close Volume
0 2010-06-29 3.800 5.000 3.508 4.778 4.778 93831500
1 2010-06-30 5.158 6.084 4.660 4.766 4.766 85935500
2 2010-07-01 5.000 5.184 4.054 4.392 4.392 41094000
3 2010-07-02 4.600 4.620 3.742 3.840 3.840 25699000
4 2010-07-06 4.000 4.000 3.166 3.222 3.222 34334500

Similarly, you can use the tail() function to get the last 5 observations of a DataFrame.

I
Another useful function to get familiar with your datasets is the dtype feature; it gives the type of

AR
each column of the DataFrame. For example:

[144]: TSLA_prices.dtypes

[144]: Date datetime64[ns]

Open float64
High float64
Low float64
Close float64
KB
Adj Close float64
Volume int64
dtype: object

Statistical Functions Simple statistical functions such as sum(), mean(), std(), var(), prod(),
median(), quantile(), abs(), cumsum() (cumulative sum), and cumprod() (cumulative product)
are available for Series and DataFrame. DataFrame also supports cov() and corr() – the keyword
argument axis determines the direction of the operation (0 for down columns, 1 for across rows).
A

describe() returns a simple set of summary statistics (number of non-missing observations, average,
standard deviation, . . . ). The output is a series where the index contains the names of the statistics
computed.

[145]: df = TSLA_prices[['Adj Close','Volume']]

[Link]()
A.

[145]: Adj Close Volume

count 2785.000000 2.785000e+03
mean 92.377739 3.173395e+07
std 166.594186 2.868064e+07
min 3.160000 5.925000e+05
25% 8.372000 1.219450e+07
50% 45.402000 2.525950e+07

60
 75% 63.509998 4.081550e+07
max 883.090027 3.046940e+08

Notice that we can get each of the above separately too. For instance, to get the average values,
you can use the mean() function:

[146]: [Link]()

I
[146]: Adj Close 9.237774e+01
Volume 3.173395e+07

AR
dtype: float64

Similarly, use std() to get the standard deviations, min() and max() to get minimum and max-
imum values of your DataFrame. count() returns the number of non-null values – that is, those
which are not NaN or another null value such as None or NaT (not a time, for datetimes). Use
quantile([0.25,0.50,0.75]) to get the 25th , 50th , and 75th percentile values.

Most NumPy functions are available in Pandas, but if needed you can convert a Pandas DataFrame
to a NumPy Array, using to_numpy() function. Then manipulate the data using NumPy functions
KB
[147]: ar = df.to_numpy()
print(ar)

[[4.77800000e+00 9.38315000e+07]
[4.76600000e+00 8.59355000e+07]
[4.39200000e+00 4.10940000e+07]
...
[6.46219971e+02 2.12971000e+07]
A
[6.60500000e+02 1.54427000e+07]
[6.55289978e+02 1.39108000e+07]]

Aggregation The simplest application of groupby is to aggregate statistics such as group means
or extrema. All aggregation functions will return a single value per column for each group. Many
common aggregation functions have optimized versions provided by pandas. For example, mean,
A.

sum, min, std, and count can all be directly used on a DataFrame GroupBy object.

Example let’s find the average price of Tesla’s stocks per year in our sample:

First, create a column for the year (more on the date functions later)

[148]: TSLA_prices['Year'] = TSLA_prices['Date'].[Link]

If the DATE column was stored as a string (str) in your DataFrame, df, you can change it to
Pandas Time-Date series, using pd.to_datetime([Link]). Then you can work with the above

61
 dt() functions (more on dates below).

Second, Select the columns you are interested, and use the groupby function to group the data

[149]: grouped_data = TSLA_prices[['Year','Open','High','Low','Close']].groupby('Year')

Third, use the mean function to give the average values:

[150]: grouped_data.mean()

I
[150]: Open High Low Close

AR
Year
2010 4.681600 4.815446 4.529215 4.668369
2011 5.364397 5.476048 5.238833 5.360952
2012 6.240624 6.361832 6.109176 6.233720
2013 20.883286 21.370603 20.355976 20.880246
2014 44.683079 45.489976 43.841222 44.665817
2015 45.966389 46.676333 45.254127 46.008580
2016 42.011690 42.686699 41.257230 41.953452
2017 62.859243 63.690119 61.937394 62.863259
KB
2018 63.436693 64.738725 62.110462 63.461984
2019 54.605627 55.529960 53.722508 54.706040
2020 289.108428 297.288412 280.697937 289.997067
2021 698.336739 712.457824 681.491378 697.165724

You can combine functions in DataFrame. For example, we can write second and thirds steps above
in one line. Below, I do so, and compute the total volume of traded Tesla stocks in each year:

[151]: TSLA_prices[['Year','Volume']].groupby('Year').sum()
A

[151]: Volume
Year
2010 1026845500
2011 1626175500
2012 1537245000
A.

2013 10593729500
2014 8711606500
2015 5441089000
2016 5811808500
2017 7950157000
2018 10808194000
2019 11540242000

62
 2020 19052912400
2021 4279034100

Delete columns of a DataFrame drop(list of columns,axis=1) will return a DataFrame

with the Series dropped without modifying the original DataFrame.

[152]: TSLA_prices_2 = TSLA_prices.drop(['Low'],axis=1)

I
TSLA_prices_2.tail()

[136]: Date Open High Close Adj Close Volume Year

AR
2780 2021-07-15 658.390015 666.140015 650.599976 650.599976 20209600 2021
2781 2021-07-16 654.679993 656.700012 644.219971 644.219971 16339800 2021
2782 2021-07-19 629.890015 647.200012 646.219971 646.219971 21297100 2021
2783 2021-07-20 651.989990 662.390015 660.500000 660.500000 15442700 2021
2784 2021-07-21 659.609985 664.859985 655.289978 655.289978 13910800 2021

drop(labels) drops rows based on the row labels in a label or list labels. For example, drop row
4 (California):
KB
[153]: TSLA_prices_3 = TSLA_prices_2.drop(2783, axis = 0) # the default axis = 0, so␣
,→ you may exclude this if you want to.
TSLA_prices_3.tail()

[137]: Date Open High Close Adj Close Volume Year

2779 2021-07-14 670.750000 678.609985 653.380005 653.380005 21641200 2021
2780 2021-07-15 658.390015 666.140015 650.599976 650.599976 20209600 2021
2781 2021-07-16 654.679993 656.700012 644.219971 644.219971 16339800 2021
2782 2021-07-19 629.890015 647.200012 646.219971 646.219971 21297100 2021
A
2784 2021-07-21 659.609985 664.859985 655.289978 655.289978 13910800 2021

Similarly, you may use the dropna() function to drop missing observations.

Finally, drop_duplicates() removes rows which are duplicates or other rows, and is used with
the keyword argument drop_duplicates(cols=col_list) to only consider a subset of all columns
when checking for duplicates.
A.

[154]: df = [Link]([[70,80, 5],[90,65,1],[74,95,3],[70,80,4]],

columns = ['Firm_1','Firm_2','Firm_3'],
index = ['Region_1','Region_2','Region_3','Region_4'] )
print('df = ')
df

df =

63
[154]: Firm_1 Firm_2 Firm_3
Region_1 70 80 5
Region_2 90 65 1
Region_3 74 95 3
Region_4 70 80 4

[155]: print('unique values of the first column of df = ',[Link](df['Firm_1']))

I
unique values of the first column of df = [70 90 74]

AR
[156]: df_drop_duplicates = df.drop_duplicates(['Firm_1','Firm_2'])
print('drop rows with duplicate values in the first and second columns: ')
df_drop_duplicates

drop rows with duplicate values in the first and second columns:

[156]: Firm_1 Firm_2 Firm_3

Region_1 70 80 5
Region_2 90 65 1
KB
Region_3 74 95 3

Sort sort_values() function sorts the contents of the DataFrame along either axis using the
contents of a single column or row. Passing a list of columns names or index values implements
lexicographic search. sort_index() function will sort a DataFrame by the values in the index.
Both support the keyword argument ascending to determine the direction of the sort (ascending
by default). ascending can be used with a list to allow sorting in different directions for different
sort variables.
A
Example Sort TSLA_prices by Volume and print the top 5 high volume days. Also, do not print
the following columns: Open, High, Low, Close.

[157]: TSLA_prices.drop(['Open','High','Low','Close'],axis=1).sort_values(by='Volume',␣
,→ascending=False).head()

[157]: Date Adj Close Volume Year

A.

2416 2020-02-04 177.412003 304694000 2020

2417 2020-02-05 146.940002 242119000 2020
2415 2020-02-03 156.000000 235325000 2020
2638 2020-12-18 695.000000 222126200 2020
2418 2020-02-06 149.792007 199404000 2020

Just before the COVID-19 crash, TSLA was trading heavily!

64
 5.5.4 Time-series Data

A TimeSeries data is basically a series where the index contains datetimes index values (more
formally the class TimeSeries inherits from Series), and the Series constructor will automatically
promote a Series with datetime index values to a TimeSeries.

TimeSeries have some useful indexing tricks. For instance, you can easily access all of the data for
a particular year using DataFrame_NAME.loc['yyyy-mm'] syntax where yyyy is the year and mm is

I
the month.

Example Show the closing stock prices and Volume of trades for Tesla in Feb. 2020:

AR
[158]: TSLA_prices.index = TSLA_prices['Date'] # use Date column as the index
TSLA_prices.drop('Date',axis=1,inplace=True)# drop the Date column. we don't␣
need it any more now
,→

print('Tesla Volume in Feb 2020:')

TSLA_prices.loc['2020-02'].drop(['Open','High','Low','Adj Close','Year'],axis=1)

Tesla Volume in Feb 2020:

[158]: Close Volume

KB
Date
2020-02-03 156.000000 235325000
2020-02-04 177.412003 304694000
2020-02-05 146.940002 242119000
2020-02-06 149.792007 199404000
2020-02-07 149.613998 85317500
2020-02-10 154.255997 123446000
2020-02-11 154.876007 58487500
A

2020-02-12 153.457993 60112500

2020-02-13 160.800003 131446500
2020-02-14 160.005997 78468500
2020-02-18 171.679993 81908500
2020-02-19 183.483994 127115000
2020-02-20 179.882004 88174500
A.

2020-02-21 180.199997 71574000

2020-02-24 166.757996 75961000
2020-02-25 159.981995 86452500
2020-02-26 155.759995 70427500
2020-02-27 135.800003 121386000
2020-02-28 133.598007 121114500

Similarly, you can access to a year, month, a date, or a period of time of a TimeSeries, for example:

65
 • The data for year 2020: TSLA_prices.loc['2020']
• The data for the month Feb., 2020:TSLA_prices.loc['2020-02']
• The data for teh day Feb. 27th, 20200:TSLA_prices.loc['2020-02-27']
• The data for the days in between Feb. 25th, 20200 and Mar. 20th, 20200:
TSLA_prices.loc['2020-02-25':'2020-03-20']

Once you have the time-series index, you can get the other date values. For example:

I
[159]: TSLA_prices['Year'] = TSLA_prices.[Link]
TSLA_prices['Quarter'] = TSLA_prices.[Link]

AR
TSLA_prices['Month'] = TSLA_prices.[Link]
TSLA_prices['Day'] = TSLA_prices.[Link]

TSLA_prices.head()

[142]: Open High Low Close Adj Close Volume Year Quarter Month Day
Date
2010-06-29 3.800 5.000 3.508 4.778 4.778 93831500 2010 2 6 29
2010-06-30 5.158 6.084 4.660 4.766 4.766 85935500 2010 2 6 30
KB
2010-07-01 5.000 5.184 4.054 4.392 4.392 41094000 2010 3 7 1
2010-07-02 4.600 4.620 3.742 3.840 3.840 25699000 2010 3 7 2
2010-07-06 4.000 4.000 3.166 3.222 3.222 34334500 2010 3 7 6

If you are importing an excel file into a pandas DataFrame and the DATE column in excel is stored
as a Date variable, the [Link]() function usually correctly detects the type of the DATE
column and stores it as date variable in the output DataFrame. However, if you are importing
from a text or csv files (in which all variables are stored as plain text) the DATE column will
also be stored as text (object) in the output DataFrame. So, if the DATE column was stored
A
as a string (str) in your DataFrame, df, you can change it to Pandas Time-Date series, using
pd.to_datetime([Link]). Then you can work with the above dt() functions.

Pandas’ to_datetime() guesses the date format in df but you may also provide the date format to
this function. For example, if the dates in the [Link] are stored as ‘YYYYMMDD’ you can use
the argument format=‘%Y%m%d’. See below
A.

[160]: pd.to_datetime('20210115', format='%Y%m%d', errors='ignore')

[160]: Timestamp('2021-01-15 00:00:00')

See Python’s webpage on “datetime — Basic date and time types”, here
[Link] for more
information on date formats.

66
 Rate of Returns A more useful application of TimeSeries data is for financial time series,
where we are interested in the growth rates (or rate of return) in a period. For example, in the
dataset above, we can calculate the daily returns of stock price. Growth rates are computed using
pct_change().
Pt + Dt − Pt−1
Rt =
Pt−1
where, Pt and Dt are the closing price and dividends of the stock on period t, respectively. In Yahoo
Finance data, the Adj Close column incorporates the effect of dividends and other distributions.

I
[161]: TSLA_prices['daily_returns'] = TSLA_prices['Adj Close'].pct_change()

AR
TSLA_prices.head().
drop(['Open','High','Low','Close','Volume','Year','Quarter','Month','Day'],axis=1)

[161]: Adj Close daily_returns

Date
2010-06-29 4.778 NaN
2010-06-30 4.766 -0.002512
2010-07-01 4.392 -0.078473
2010-07-02 3.840 -0.125683
KB
2010-07-06 3.222 -0.160937

Notice that our stock price data starts from June 29, 2010. So, we do not have the return data for
that date.

The pct_change() function can also calculate growth rates over a longer period. The keyword
argument periods constructs overlapping growth rates which are useful when using seasonal data.
For example, in our dataset, which is at the daily frequency, you can get the annual growth rate,
with respect to the 252 business days (the default value is 1)
A

[162]: TSLA_prices['Adj Close'].pct_change(periods=252).tail() # a year has about 252␣

business days
,→

[162]: Date
2021-07-15 1.104126
A.

2021-07-16 1.146484
2021-07-19 1.152861
2021-07-20 1.010043
2021-07-21 1.089093
Name: Adj Close, dtype: float64

Alternatively, you can re-sample the daily price data to annual data (end of year) and then compute
the annual returns:

67
[163]: price_annual = TSLA_prices.resample("1y").bfill()
price_annual['Adj Close'].pct_change().tail()

[163]: Date
2017-12-31 0.477165
2018-12-31 0.038280
2019-12-31 0.257001
2020-12-31 7.434370
2021-12-31 0.000000

RI
Freq: A-DEC, Name: Adj Close, dtype: float64

A number of string aliases are given to useful common time series frequencies. We will refer to these
aliases as offset aliases. For instance,

• B: business day frequency

• D: calendar day frequency
• W: weekly frequency
• M: month end frequency
• Q: quarter end frequency
• A, Y: year end frequency
BA
You may resample up or down. For instance, you may resample from daily data to convert them to
AK
annual data or resample from annual data to convert them to daily data. Check the documentation
for this function on Pandas’ website.

For the resample() function to operate correctly we need to specify to backward fill (.bfill())
or forward fill (.ffill()) the missing observations. See the examples below, where the daily data
ends in the middle of September. ffill() function uses the last observation per month (15th of
September) and uses it for the last day of the month. bfill() function does a backward fill and
since there is no observation for last day of the month, it gives NaN for the missing observation.

[164]: df = TSLA_prices.loc['2019-07-15':'2019-09-15']
A.

df1 = df['Close'].resample("1m").bfill()
df2 = df['Close'].resample("1m").ffill()
print('df1 = ', df1)
print('df2 = ', df2)

df1 = Date
2019-07-31 48.321999
2019-08-31 45.001999
2019-09-30 NaN
Freq: M, Name: Close, dtype: float64

68
 df2 = Date
2019-07-31 48.321999
2019-08-31 45.122002
2019-09-30 49.040001
Freq: M, Name: Close, dtype: float64

The pct_change() function is very useful for analyzing the stock returns. Most financial databases
provide a time series for the price of a company and you can use this function to get the holding

I
period returns.

AR
The TimeSeries is also a useful data structure for plotting financial data. In the next sections, I
will explain the basics of plotting with Python.

Construct Portfolios A portfolio is a collection (basket) of financial investments like stocks,

and bonds. It is commonly identified by its ingredients (what is in the basket) and weight of them
(portion of each item in the basket).

The return of a portfolio (Rp ) is the weighted average of the returns of the stocks (Ri ) in that
portfolio.
KB X
Rp = wi × Ri
i

where wi is the weight of stock i in the portfolio.

For instance, suppose you construct a portfolio by investing 80% of your money in VOO and 20% in
TSLA. Let’s find the annual return of your portfolio in 2018, 2019, 2020, and 2021.

[165]: # read daily data

TSLA_prices = pd.read_excel('VOO_TSLA_BTC_prices.xlsx',sheet_name = 'TSLA')
VOO_prices = pd.read_excel('VOO_TSLA_BTC_prices.xlsx',sheet_name = 'VOO')
A

# merge price data

df = [Link](VOO_prices[['Date','Adj Close']],TSLA_prices[['Date','Adj␣
,→ Close']],suffixes=('_VOO', '_TSLA'), on='Date')

# use Date column as the index to convert this dataftame to a timeseries␣

A.

,→ DataFrame
[Link] = df['Date']
# drop the Data columns except adj colose. we don't need them anymore now
[Link](['Date'],axis=1,inplace=True)

# find annual price data and then annual returns and drop the missing␣
,→ observations for the first year

69
 df_annual = [Link]("1y").ffill().pct_change().dropna()

# find portfolio returns

# # first find a time-series of weights.
# # In this example the weights are constant. The multiplication in numpy␣
would be column by column.
,→

weights = [Link]([0.80, 0.20],index=df_annual.columns)

I
# # Then find the portfolio returns:

AR
# # Recall: Rp = sum (w_i * R_i)
df_annual['Rp'] = (df_annual * weights).sum(axis=1)

# # show the returns during a period

df_annual.loc['2018':'2021']

[165]: Adj Close_VOO Adj Close_TSLA Rp

Date
2018-12-31 -0.045007 0.068893 -0.022227
KB
2019-12-31 0.313650 0.257001 0.302321
2020-12-31 0.183244 7.434370 1.633469
2021-12-31 0.170528 -0.071393 0.122144
A
A.

70
 5.6 Pandas_datareader
This library allows you to download data from several websites such as Yahoo Finance (yahoo), US
Federal Reserve Economic Data (fred), Bank of Canada, World Bank, and Kenneth French’s data
library (famafrench). Check the documentation of the library for more details here: [Link]
[Link]/en/latest/remote_data.html

You need to first install this library as it is not included in the main package during the installation.

I
Launch Anaconda Prompt (from the Start menu of Windows or the Mac equivalent) then type/run:
pip install pandas_datareader. It will download and install it on your machine.

AR
The main function in this library is called DataReader() and its general syntax is:
DataReader(Symbol, Data_Source, start_date, end_date). * Symbol identifies what you want
to download, * Data_Source identifies the label (name) of the data vendor. See the library’s docu-
mentation for the full list of vendors that are compatible with Pandas_datareader. * start_date
and end_date identify the data period to be downloaded. If not identified, it will be the last 5
years.

In this class, we mostly use this library to get stock prices from Yahoo Finance website. But for
financial modeling you may also want to download Macroeconomic data (such as GDP) from the
KB
central banks’ website. Below, I list two examples:

Stock Prices We can use Yahoo Finance website to get the stock prices of companies using their
Ticker Symbols. The example below shows how to download the closing price for three companies:

[166]: import pandas_datareader.data as pdr

# Define the instruments to download. We would like to see Apple, Microsoft and␣
the S&P500 index.
A
,→

tickers = ['AAPL', 'MSFT', '^GSPC']

# We would like all available data from 31/12/2022 until 09/01/2023.

start_date = '2022-12-31'
end_date = '2023-09-01'
A.

# User pandas_reader.[Link] to load the

# stock prices from Yahoo Finance.

# panel_data = [Link](tickers, 'yahoo', start_date, end_date)

import yfinance as yfin

yfin.pdr_override()

71
 panel_data = pdr.get_data_yahoo(tickers, start_date, end_date)

# Yahoo Finance gives 'High', 'Low', 'Open', 'Close', 'Volume', 'Adj Close'.

Close_Price = panel_data['Close']
Close_Price.head()

I
[*********************100%%**********************] 3 of 3 completed

[166]: AAPL MSFT ˆGSPC

AR
Date
2023-01-03 125.070000 239.580002 3824.139893
2023-01-04 126.360001 229.100006 3852.969971
2023-01-05 125.019997 222.309998 3808.100098
2023-01-06 129.619995 224.929993 3895.080078
2023-01-09 130.149994 227.119995 3892.090088

This will download data from Yahoo Finance, at the daily frequency. To get monthly data (prices
KB
at the end of each month), you need to re-sample from daily data every 1 month:

[167]: Close_Price_monthly = Close_Price.resample("1m").ffill()

Close_Price_monthly.head()

[167]: AAPL MSFT ˆGSPC

Date
2023-01-31 144.289993 247.809998 4076.600098
2023-02-28 147.410004 249.419998 3970.149902
A
2023-03-31 164.899994 288.299988 4109.310059
2023-04-30 169.679993 307.260010 4169.479980
2023-05-31 177.250000 328.390015 4179.830078

Note: Yahoo Finance recently made changes to their API that broke compatibility with previous
pandas datareader versions. As a solution, here I am using yfinance library to override on the
functions of pandas_datareader. You may directly use yfinance but since the issue might be
A.

resolved soon, here I explain both methods.

[168]: import yfinance as yfin

panel_data = [Link](tickers, start_date, end_date)

Close_Price = panel_data['Close']

72
 Close_Price.head()

[*********************100%%**********************] 3 of 3 completed

[168]: AAPL MSFT ˆGSPC

Date
2023-01-03 125.070000 239.580002 3824.139893
2023-01-04 126.360001 229.100006 3852.969971

I
2023-01-05 125.019997 222.309998 3808.100098
2023-01-06 129.619995 224.929993 3895.080078

AR
2023-01-09 130.149994 227.119995 3892.090088

Economics Data We can use the website of the Federal Reserve Bank of [Link] (FRED), the
Bank of Canada, or World Bank. For example, below we can download US GDP data:

[169]: import pandas_datareader as pdr

GDP_data = pdr.get_data_fred('GDP', start='2020-01-01', end='2021-12-31')
GDP_data
KB
[169]: GDP
DATE
2020-01-01 21706.513
2020-04-01 19913.143
2020-07-01 21647.640
2020-10-01 22024.502
2021-01-01 22600.185
2021-04-01 23292.362
A

2021-07-01 23828.973
2021-10-01 24654.603

Portfolio Data from Fama French We can access the datasets on the Fama/French Data
Library ([Link] via Pan-
das_datareader. There are many useful portfolios and asset pricing factors in this data library
A.

(it has more than 250 datasets). We will work with some of these in this class.

The get the name of all datasets in this website, use .get_available_datasets() function. Below,
we can see the 5 first datasets.

[170]: import pandas_datareader.data as pdr

from pandas_datareader.famafrench import get_available_datasets

73
 get_available_datasets()[:4]

[170]: ['F-F_Research_Data_Factors',
'F-F_Research_Data_Factors_weekly',
'F-F_Research_Data_Factors_daily',
'F-F_Research_Data_5_Factors_2x3']

Let’s dig in more deeply and choose ‘5_Industry_Portfolios’ to get the data for the top 5 sectors

I
in the US: Consumers, Manufacturing, High Tech., Health care, and others.

AR
[171]: Industry_Portfolios = pdr.get_data_famafrench('5_Industry_Portfolios')

In the Variable Explorer tab click on Industry_Portfolios and click on DECR to get the description
of the data that you just downloaded.

This file was created by CMPT_IND_RETS using the 2021-June CRSP database (CRSP dataset
is accessible from the WRDS portal- if you have not done so, you may get your credentials from
[Link] and your MAC emails). It contains the value- and equal-
weighted returns for 5 industry portfolios. The portfolios are constructed at the end of June.
KB
The annual returns are from January to December. Missing data are indicated by -99.99 or -999.
Copyright 2021 Kenneth R. French

0 : Average Value Weighted Returns -- Monthly (96 rows x 5 cols)

1 : Average Equal Weighted Returns -- Monthly (96 rows x 5 cols)
2 : Average Value Weighted Returns -- Annual (8 rows x 5 cols)
3 : Average Equal Weighted Returns -- Annual (8 rows x 5 cols)
4 : Number of Firms in Portfolios (96 rows x 5 cols)
5 : Average Firm Size (96 rows x 5 cols)
A

6 : Sum of BE / Sum of ME (8 rows x 5 cols)

7 : Value-Weighted Average of BE/ME (8 rows x 5 cols)

To compute the value-weighted average portfolio returns (index 0), the dataset uses the market
capitalization of each firm as a weight
A.

X
Rvw = wi Ri

74
[172]: Industry_Portfolios[0].head()

[172]: Cnsmr Manuf HiTec Hlth Other

Date
2018-10 -6.01 -8.65 -7.62 -8.79 -6.74
2018-11 2.08 2.47 -0.93 6.42 2.81
2018-12 -9.88 -9.06 -8.23 -8.25 -10.94
2019-01 8.10 8.96 8.70 5.32 9.80

I
2019-02 1.09 4.16 5.39 3.30 2.96

AR
A KB
A.

75
 5.7 Matplotlib
This is a complete plotting library capable of high-quality graphics. Here I cover the basics of produc-
ing plots. Further information is available on the matplotlib website ([Link]

Another useful graphics library is seaborn, which is a Python data visualization library based on
matplotlib. It provides a high-level interface for drawing attractive and informative statistical
graphics. Refer to its documentation page ([Link] for more information.

I
[173]: import [Link] as plt

AR
5.7.1 Line Plot

The most basic, and often most useful 2D graphic is a line plot. Basic line plots are produced using
plot(INPUT_VARIABLE) using a single input containing a 1-dimensional array.

Example plot the monthly returns for the High-Tech sector (‘HiTec’ in the Kenneth French Data
Library datasets)

[174]: df = Industry_Portfolios[0]
y = df['HiTec'].to_numpy() # for teaching purposes, I convert it first to a␣
KB
NumPy Array object
,→

[Link](y)
[Link]()
A
A.

76
 You may need to also call show() function to see the generated plot, in the last step of your plotting,
in interactive Python editors such as Jupyter.

You can modify the line color, line style or the marker on the graph by adding the Format strings,
which may contain any of the following elements:

Color Line Style Marker

b for Blue - for Solid . for Point (·)

I
g for Green -- for Dashed o for Circle (•)
r for Red -. for Dash-dot s for Square (■)

AR
c for Cyan : for Dotted d for Thin Diamond (♦)
m for Magenta x for Cross (×)
y for Yellow + for Plus (+)
k for Black * for Star (⋆)
w for White

For example:
KB
[175]: # plot y in (r), use square markers (s), in dotted lines (:)
[Link](y,'r*:')

# Add a title
[Link]("High Tech. Sector", color='g')

# Add y-labes
[Link]('HiTec monthly returns', color='b')
A

# Add the x-label

[Link]('Observation', color='k')

# Add gridlines
[Link]()
A.

# plotting is finished. show it

[Link]()

77
 I
AR
KB
Note In Spyder you need to run simultaneously all commends related to one graph, otherwise each
line would generate a separate graph. The purpose of the show() function in this case is to inform
the Python interpreter that you are done editing the graph and you want to observe the output.

You can add multiple lines in a graph like below:

[176]: x = Industry_Portfolios[0].to_numpy()
A

[Link](x[:,0],'b', label = 'Cnsmr')

[Link](x[:,1],'g.', label = 'Manuf')
[Link](x[:,2],'r:',label = 'HiTec')
[Link](x[:,3],'k-.', label = 'Hlth ')
[Link](x[:,4],'y--',label = 'other')
A.

# Add legend
[Link](loc = 'lower right', frameon = True)
[Link]()

78
 I
AR
KB
Alternatively, you can plot each line in separate graphs. See the example below, in which we plot
5 graphs in 2-by-3 row-column panels. The function figure() creates a new empty image. The
function subplot(NMj) identifies how many panels are in each row (N) and in each column (M). The
last number, j, identifies the location of the current panel in the panels.

[177]: [Link]()
[Link]['[Link]'] = [10, 10] # sets the size of the plots
A

[Link](231)
[Link](x[:,0],'b', label = 'Cnsmr')
[Link](loc = 'lower right', frameon = True)

[Link](232)
[Link](x[:,1],'g.', label = 'Manuf')
A.

[Link](loc = 'lower right', frameon = True)

[Link](233)
[Link](x[:,2],'r:',label = 'HiTec')
[Link](loc = 'lower right', frameon = True)

[Link](234)

79
 [Link](x[:,3],'k-.', label = 'Hlth ')
[Link](loc = 'lower right', frameon = True)

[Link](235)
[Link](x[:,4],'y--',label = 'other')
[Link](loc = 'lower right', frameon = True)

I
[Link]()

AR
A KB
You may create different plotting objects for each panel and work with them. See this example,
where we create a 1-by-2 row-column panels. Each item of the object ax identifies each panel (ax[0]
and ax[1]). Then we can add a title or y-label for each panel.

[178]: [Link]['[Link]'] = [10, 5] # sets the size of the plots

A.

fig, ax = [Link](1,2)
ax[0].plot(x[:,0],'b')
ax[0].set_ylabel('Return %')
ax[0].title.set_text('Cnsmr')

ax[1].plot(x[:,3],'k-.')

80
 ax[1].title.set_text('Hlth')

[Link]()

I
AR
KB
5.7.2 Scatter Plots

Similarly, we can draw scatter plots with the Matplotlib library. This is when you want to plot
the variable y in the vertical axis with respect to the variable x in the horizontal axis (you can also
use [Link](x,y)). For instance, below I plot the monthly return of the High-tech sector, in the
A
y-axis, with respect to the Manufacturing sector, in the x-axis

[179]: df = Industry_Portfolios[0]

#plot y versus x, in (r), use square markers (s), in dotted lines (:)
[Link](x=df['Manuf'],y=df['HiTec'])
A.

# Add a title
[Link]("High Tech. vs. Manufacturing Sectors")

# Add y-labes
[Link]('HiTec monthly returns')

# Add the x-label

81
 [Link]('Manuf monthly returns')

# Add gridlines
[Link]()

# plotting is finished. show it

[Link]()

I
AR
A KB
5.7.3 Bar Charts

bar() function produces bar charts using two 1-dimensional arrays. The first specifies the left edge
of the bars and the second the bar heights.

Example Plot the average monthly return of the High Tech. sector per year in the last 5 years.

[180]: df['Year'] = [Link]

A.

df_yr = [Link]('Year').mean()
[Link](df_yr.index,df_yr['HiTec'])
[Link]()

82
 I
5.7.4 Pie Charts

AR
KB
pie() function produces pie charts using a 1-dimensional array of data. The data can have any
values, and does not need to sum to 1. However, most often we use the pie chart to show how
different portions of the data are grouped. In these cases the numbers are summed to 1.

Example: For the High Tech. sector, what percentage of the monthly returns are above 5% (large
gain), between 0 and 5% (moderate gain), between 5% and 0 (moderate loss), and less than -5%
(large loss)? Show with a pie chart

First, construct a column, indicating the observations with returns in each category:
A

[181]: df['HiTec_Return_category'] = 'large gain'

[Link][df['HiTec']< -5 ,'HiTec_Return_category'] = 'large loss'
[Link][(df['HiTec']>= -5) & (df['HiTec']<0),'HiTec_Return_category'] = 'moderate␣
,→ loss'
[Link][(df['HiTec']>= 0) & (df['HiTec']<5),'HiTec_Return_category'] = 'moderate␣
A.

,→ gain'
[Link]()

[181]: Cnsmr Manuf HiTec Hlth Other Year HiTec_Return_category

Date
2023-03 2.78 0.85 9.52 2.49 -5.00 2023 large gain
2023-04 -0.10 0.71 0.31 4.12 1.83 2023 moderate gain
2023-05 0.64 -6.67 6.98 -3.67 -2.65 2023 large gain

83
 2023-06 8.98 7.96 5.83 4.66 7.44 2023 large gain
2023-07 2.36 4.16 4.10 -0.12 5.24 2023 moderate gain

Then count the number of observations in a year with positive returns:

[182]: df_yr = df[['HiTec','HiTec_Return_category']].groupby('HiTec_Return_category').

,→ count()
df_yr

I
[182]: HiTec

AR
HiTec_Return_category
large gain 20
large loss 12
moderate gain 18
moderate loss 8

Lastly, plot the series. Also, add the relative size of each year’s observation as a auto_percentage

[183]: [Link](df_yr['HiTec'], labels = df_yr.index, autopct='%1.1f%%')

KB
[Link]()
A
A.

84
 Pie charts can be modified using a large number of keyword arguments, including labels and custom
colors. Exploded views of a pie chart can be produced by providing a vector of distances to the
keyword argument explode. Note that autopct = '%2.0f' is using an old style format string to
format the numeric labels.

5.7.5 Histograms

Histograms can be produced using hist() function. Histograms show the frequency of repeated

I
values in your data. See the example below, where, we plot the histogram for the monthly returns
of the High Tech. sector. The height of each bin shows the number of months (observations) where

AR
the return falls in that bin. We can set the number of bins used in producing the histogram using
the keyword argument bins.

[184]: [Link](df['HiTec'], bins = 20)

[Link]()
A KB

Histograms can be further modified using keyword arguments. In the next example,
A.

cumulative=True produces the cumulative histogram.

[185]: [Link](df['HiTec'], bins = 30, cumulative=True, color='r')

[Link]()

85
 I
5.7.6 Plots with Pandas

AR
KB
matplotlib also sits on pandas library and you can directly call it from a DataFrame object.

[186]: df = Industry_Portfolios[0]
df['HiTec'].plot()
[Link]()
A
A.

86
 Note Above, I’m plotting a pandas DataFrame data (not NumPy Array), therefore I am able to
use the time index as the x-axis.

You can have primary and secondary y-axises. This is a useful feature especially when the variables
do not have the same scale and their order of magnitudes differ significantly.

Example Plot Apple’s stock prices and monthly returns for the last 5 years. For this, we first
download stock price data with the pandas_datareader library at the daily frequency. Then we
re-sample at the monthly frequency to calculate monthly returns. In order to have primary and

I
secondary y-axises, we need to define two (sub)plots, and tell python to share their x-axis. See

AR
below, where I define a plot and call it ax1 and then create a twin copy of it and call it ax2. Once
we have these, we can plot our graphs in each plot and Python will merge them.

[187]: # 1. Download Price Data for Apple (AAPL).

import pandas_datareader.data as pdr

import yfinance as yfin

yfin.pdr_override()
panel_data = pdr.get_data_yahoo('AAPL','2018-01-01','2022-12-31')
KB
APPL_Price = panel_data['Close']
APPL_Return = panel_data['Adj Close'].resample("1m").ffill().pct_change()

# 2. In order to have two y-axis,

# 2.1. share the x axis between the two plots
fig, ax1 = [Link]()
A
ax2 = [Link]()
# 2.2. plot the series
[Link](APPL_Price,'g', label = 'Price')
[Link](APPL_Return,'b*:', label = 'Return')
A.

# 2.3. Add y-labes

ax1.set_ylabel('Price', color='g')
ax2.set_ylabel('Monthly Return', color='b')

# 2.4. Add a title and the x-label

[Link]('APPL Stock Price and Return')
ax1.set_xlabel('Year')

87
 #2.5. Add legends
[Link](loc = 'upper left') # you can type the location
[Link](loc = 1) # or you can use the index number for the place␣
of the legend location
,→

# 2.5. Save the plot in a file

I
[Link]('AAPL_DailyClosePrice_MonthlyReturns.pdf')
[Link]('AAPL_DailyClosePrice_MonthlyReturns.jpg')

AR
[Link]()

[*********************100%%**********************] 1 of 1 completed
A KB
Example It is a common practice to also impose the 25, 50, 75 percentile on the histogram (see
below). Here, I use the plot function with kind = 'hist' for histogram and I used histype =
'step' in order not to have filled bins. The function axvlines help us to add lines on the graph,
A.

which I use it to mark Mean, Median, 25 and 75 percentiles on the graph.

[188]: df['HiTec'].plot(kind='hist',histtype='step',bins=20)

[Link](df['HiTec'].mean(),c='C1')
[Link](df['HiTec'].median(),c='C1',linestyle='--')
[Link](df['HiTec'].quantile(0.25),c='C1',linestyle=':')
[Link](df['HiTec'].quantile(0.75),c='C1',linestyle=':')

88
 [Link]('Histogram for monthly returns of the HiTec Sector')
[Link]()

I
AR
KB
You may also draw the probability distribution function (pdf) on the histogram, which is the smooth
histogram, scaled to have an integral of 1.

[189]: df['HiTec'].plot(kind='hist',histtype='step',bins=20, density=True)

df['HiTec'].[Link]()
A
[Link]('HiTec')
[Link]()
A.

89
 A.
A

90
KB
AR
I
 5.8 Statsmodels
statsmodels library provides classes and functions for the estimation of many different statistical
models, tests, data exploration. In this class, we use this library for linear regression applica-
tion, using the ordinary least-square, or OLS, estimator. The online documentation is hosted at
[Link].

Example 1 In the example below, we use this library to find the relationship between monthly

I
returns of the High-Tech. and Manufacturing sectors. Let’s first plot these series in a scatter graph,
where for each month, on the y-axis we show the High-Tech sector’s return and the x-axis we show

AR
the Manufacturing sector’s return on the same date. We also fit a linear trendline on the graph,
using the first order polynomial function (polyfit) from the numpy library.

[190]: import numpy as np

import pandas as pd
import [Link] as sm
import pandas_datareader.data as pdr
import [Link] as plt
KB
# data download
start_date = '2018-01-01'
end_date = '2023-01-01'

Industry_Portfolios = pdr.get_data_famafrench('5_Industry_Portfolios',␣
,→ start_date,end_date)

df = Industry_Portfolios[0]
A
x=df['Manuf']
y=df['HiTec']
#plot y versus x, in (r), use square markers (s), in dotted lines (:)
[Link](x,y)

# Add a title, y-label, and x-labels

A.

[Link]("High Tech. vs. Manufacturing Sectors")

[Link]('HiTec monthly returns')
[Link]('Manuf monthly returns')

# fitted line with polyfit

beta,alpha = [Link](x, y, 1)

91
 Formula_text = 'y = ' + str(round(alpha,2))+ ' + ' + str(round(beta,2)) + 'x'

# plot the fitted line

[Link](x, alpha + beta * x , 'k:')
[Link](-20, 6, Formula_text , fontsize=20) # write the formula on the plot

# plotting is finished. show it

I
[Link]()

AR
A KB
The function polyfit() is the equivalent to the TRENDLINE in MS-Excel, in which you can choose
the order of the polynomial you fit to the data. Polynomial of order 1 is a linear line.

Now, let estimate the beta and alpha with the OLS() function from the statmodel library. This
function requires y and x inputs. If you want to also include an intercept in the regression you can
do so with the add_constant() function. By default, the fitted line is assumed to go through the
A.

0,0 point (i.e. no intercept).

[191]: x2 = sm.add_constant(x)
results = [Link](y,x2 ).fit()

alpha = [Link][0]
beta = [Link][1]

92
 print('The intercept of the fitted line = ', round(alpha,2))
print('The slope of the fitted line = ', round(beta,2))

The intercept of the fitted line = 0.57

The slope of the fitted line = 0.81

Example 2 In the example below, we use this library to find the three-year monthly market beta
for the High Tech. sector. In the first step, we import the libraries we need and then download the

I
data:

[192]: # initialize

AR
import pandas as pd
import [Link] as sm
import pandas_datareader.data as pdr

# data download
start_date = '2018-01-01'
end_date = '2021-01-01'
KB
Industry_Portfolios = pdr.
get_data_famafrench('5_Industry_Portfolios',start_date,end_date)
,→

Market_Factors = pdr.
get_data_famafrench('F-F_Research_Data_Factors',start_date,end_date)
,→

Industry_Portfolios dataset from famafrench data library contains several forms of the five sector
portfolios. The value-weighted portfolios are stored in item [0]. Market_Factors dataset includes
both monthly and annual data for the market, as well as the size and value factors. It also provides
A
data for the risk free rate of returns. The monthly data is stored in item [0].

In the next step, we merge the two datasets:

[193]: # merge datasets

df = [Link](Market_Factors[0], Industry_Portfolios[0], on='Date')
[Link]()
A.

[193]: Mkt-RF SMB HML RF Cnsmr Manuf HiTec Hlth Other

Date
2018-01 5.57 -3.13 -1.29 0.12 6.35 2.46 7.06 6.38 5.78
2018-02 -3.65 0.25 -1.04 0.11 -5.26 -6.00 -1.21 -3.49 -3.28
2018-03 -2.35 4.06 -0.21 0.11 -2.31 0.11 -3.07 -2.41 -2.63
2018-04 0.28 1.13 0.54 0.14 1.09 1.38 -0.53 -0.04 0.62
2018-05 2.65 5.25 -3.20 0.14 0.59 2.36 5.67 2.37 1.34

93
 According to the capital asset pricing model (CAPM), the expected excess asset return is linearly
related to the market premium:

E[Ri − Rf ] = βi E[Rm − Rf ]

Therefore, we can estimate market beta for asset i, denoted by βi , using linear regressions:

Ri,t − Rf,t = αi + βi (Rm,t − Rf,t ) + ei,t

RI
where, we include an intercept, αi to capture the time-invariant effect of other (missing) factors. If
CAPM holds in the data, the estimated αi will be insignificant. The term ei,t denotes the asset-
specific noise, which is zero on average.

In this model, the left-hand side variable (Y-axis variable) is the monthly returns of the Hight Tech.
sector (HiTec) in excess of the risk free rate. The righ-hand side variables (the X-axis variable) are
a constant (const) and the excess market return (Mkt-RF).

[194]: Y = df['HiTec'] - df['RF']

X = sm.add_constant(df['Mkt-RF'])
BA
Now, we can estimate the model using the OLS() function. Simply, give the left- and right-hand
side variables and find the fitted values as follows:
AK
[195]: results = [Link](Y, X).fit()
print([Link]())

OLS Regression Results

==============================================================================
Dep. Variable: y R-squared: 0.943
Model: OLS Adj. R-squared: 0.941
Method: Least Squares F-statistic: 578.8
Date: Mon, 16 Oct 2023 Prob (F-statistic): 2.34e-23
A.

Time: 13:39:47 Log-Likelihood: -64.326

No. Observations: 37 AIC: 132.7
Df Residuals: 35 BIC: 135.9
Df Model: 1
Covariance Type: nonrobust
==============================================================================
coef std err t P>|t| [0.025 0.975]
------------------------------------------------------------------------------
const 0.7465 0.238 3.137 0.003 0.263 1.230
Mkt-RF 0.9971 0.041 24.058 0.000 0.913 1.081

94
 ==============================================================================
Omnibus: 0.656 Durbin-Watson: 1.994
Prob(Omnibus): 0.720 Jarque-Bera (JB): 0.754
Skew: -0.257 Prob(JB): 0.686
Kurtosis: 2.525 Cond. No. 5.88
==============================================================================

I
Notes:
[1] Standard Errors assume that the covariance matrix of the errors is correctly

AR
specified.

The above table, presents the analysis of variance information too. It reports the R-squared,
estimated slope and intercept values (coef) and their 5% confidence intervals ([0.025 0.975]).
If you want to access each of these separately you can do so as follows:

[196]: alpha_i = [Link][0]

beta_i = [Link][1]
R2 = [Link]
KB
We observe that the High Tech. sector has a large beta (almost 1). Notice that in this example,
the intercept, αi is estimated non-zero. It is positive and statistically significant, suggesting that
CAPM cannot explain the whole variation in these series.
A
A.

95
 5.9 SciPy
SciPy library contains a number of routines to the find extremum (minimum or maximum) of a
user-supplied objective function located in [Link]. SciPy has more useful functions but
for the sake of this class we only focus on the optimization and solver functions.

[197]: import [Link] as opt

The steps are similar to the Solver add-on in MS-Excel. First we need to define an (objective)

I
function that we want to minimize. For example, consider finding the minimum of the function
f (x) = x2 . A function which allows the optimizer to work correctly has the form

AR
[198]: def optim_target1(x):
return x**2

Once an optimization target has been specified, the next step is to use one of the optimizers functions
in SciPy to find the minimum values of this objective function. Optimizers are categorized into two
main types: unconstrained and constrained optimizers. If the parameters in mind are limited in a
range (for instance, only positive numbers), we use constrained optimizers. On the other hand, if
the parameters of the objective function can take any value then use unconstrained optimizers.
KB
An important application of optimizers in financial modeling is to find the optimal portfolio weights
for an investor.

5.9.1 Unconstrained Optimizers

The basic structure of all of the unconstrained optimizers is optimizer(f, x0) where the optimizer
is one of the fmin_bfgs, fmin_cg, fmin_ncg or fmin_powell functions in the SciPy library. f is
your objective function and x0 is a set of initial values used to start the algorithm. For this course,
A

we mostly use fmin_bfgs, which is the simplest optimizer. Back to our example above:

[199]: opt.fmin_bfgs(optim_target1, x0=1)

Optimization terminated successfully.

Current function value: 0.000000
Iterations: 2
A.

Function evaluations: 6
Gradient evaluations: 3

[199]: array([-7.45036449e-09])

The optimizer fmin_bfgs finds that at x = -7.45036449e-09 our objective function (f (x) = x2 )
reached to its minimum value, which is Current function value: 0.000000. Notice that the

96
 correct theoretical answer is x=0, but numerically Python finds x = -0.000000000745036449, is
accurate to 9 decimal places.

We can modify our objective function to solve equations. Say we want to know at what value of
x the objective function becomes 4. The idea is to define a new objective function and find the
minimum distance of the initial objective function: i.e. f (x) = x2 − 4

[200]: def optim_target2(x):

I
return abs(x**2 - 4)

AR
X_star = opt.fmin_bfgs(optim_target2, x0=1)
X_star

Warning: Desired error not necessarily achieved due to precision loss.

Current function value: 0.000000
Iterations: 1
Function evaluations: 78
Gradient evaluations: 33
KB
[200]: array([1.99999999])

The optimizer finds that at X_star = 1.99999999 our objective function ($f(x) = xˆ2-4 $) equals
to its minimum value (i.e.0.000000). Notice that the correct theoretical answer is x=2, and the
numerical solution is accurate to 8 decimal places. Notice that x = -2 is also a correct theoretical
answer. The choice of the initial value (x0=1) in case of several minimum values becomes important.

[201]: def optim_target2(x):

return abs(x**2 - 4)
A

X_star = opt.fmin_bfgs(optim_target2, x0=-1)

X_star

Optimization terminated successfully.

Current function value: 0.000000
Iterations: 2
A.

Function evaluations: 50
Gradient evaluations: 25

[201]: array([-2.00000001])

Setting x0 = -1 results in X_star = -2.

More Complex Cases: Imagine the case where the objective function has multiple parameters (a
parameter vector). In that case, define the objective function in the following form

97
[202]: def optim_target3(params):
x, y = params
return x**2 - 3*x + 3 + y*x -3*y + y**2

Another case, would be to have an objective function that requires additional inputs, which are not
parameters of interest. In this case, define the objective function in the following form:

[203]: def optim_target4(params, hyperparams):

I
x, y = params
c1, c2, c3 = hyperparams

AR
return x**2 + c1*x + c2 + y*x + c3*y + y**2

This form is especially useful when optimization targets require both parameters and data. Ad-
ditional inputs can be passed to the optimization target using the keyword argument args and a
tuple containing the input arguments in the correct order. In the above example, we have a single
additional input, therefore the comma is necessary in (hyperparams,) to let Python know that this
is a tuple. E.g:

[204]: # the syntax would be to include arg = (extra inputs,)

KB
param0 = [1,2]
hyperparams = [5,0.1,3]
opt.fmin_bfgs(optim_target4, param0, args=(hyperparams,) )

Optimization terminated successfully.

Current function value: -6.233333
Iterations: 3
Function evaluations: 12
A
Gradient evaluations: 4

[204]: array([-2.33333173, -0.33333519])

Example: Find the optimal portfolio Assume that there are only five eligible stocks (avail-
able/approved for investment by your company): The Walt Disney Company (DIS), International
Business Machines Corporation (IBM), The Coca-Cola Company (KO), Ford Motor Company (F),
A.

The Procter & Gamble Company (PG). You want to find the weights of these stocks in a portfolio
with maximum Sharpe ratio. You use the last 10 years of data, at the monthly frequency to infer
assets expected return, as well as their volatility and cross-correlations. As usual, let’s import the
libraries we need

[205]: #%% initialize

import pandas as pd
import numpy as np

98
 import pandas_datareader.data as pdr # for downloading data directly from Python

from [Link] import MonthEnd # for transfering to End of Month

import [Link] as opt

To do this unconstrained optimization, we first write a function to calculate the Sharpe ratio of a
portfolio (call it SharpeRatio(). This function receives stocks’ weights in the portfolio and their

I
monthly returns, as well the monthly risk free rate. In the next step, we find the portfolio weights
that would result in the maximum Sharpe ratio of our portfolio. We do this by minimizing the

AR
negative value of the portfolio’s Sharpe ratio (hence the negative sign in SR_portfolio formula in
the function below). Notice that there is only one constraint here, which is that the sum of the
weights is 100%. We manually make sure this will hold. That is, in the optimization, we find the
weights for the first 4 stocks and calculate the weight for the last one.

[206]: def SharpeRatio(weights, Ret,Rf):

w = [Link](weights, 1-sum(weights) )
Ret_portfolio = (w * Ret).sum(axis=1)
SR_portfolio = -1 * (Ret_portfolio.mean() - Rf)*12/Ret_portfolio.std()/np.
KB
sqrt(12) #I minimize the negative value of the Sharpe Ratio
,→

return SR_portfolio

Then we will use the fmin_bfgs() function from the SciPy library to find the optimal weights. This
function will require an initial guess for the weights (weights0 ). Other inputs to this function are
the rest of the inputs to the SharpeRatio() function (other than portfolio weights).

optimalweights = opt.f min_bf gs(SharpeRatio, weights0 , args)

A

Let’s prepare these information, before running the [Link]() function.

[207]: #%% read data

"""
I use the pandas_datareader library to download data from Yahoo Finance and␣
Kenneth French Website
A.

,→

"""
# # # First Download Stock Price data
tickers = ['DIS', 'IBM', 'KO','F','PG']
start_date = '2011-01-01'
end_date = '2020-12-31'
panel_data = pdr.get_data_yahoo(tickers, start_date, end_date)

99
 Returns = panel_data['Adj Close'].resample("1m").ffill().pct_change().
,→ dropna()# first observation is NaN, so I drop it

#%% Download Market Portfolio and RF from Fama French database

# **** first find the risk free rate and market return from Kenneth French␣
,→ website ***
FF_3Factor_All = pdr.get_data_famafrench('F-F_Research_Data_Factors',␣

I
,→ start_date, end_date)
# # the first part of the output (index 0) is for the monthly factors,

AR
# # these values are reported
# # in percentage so we change it to decimal values, and
# # the time index is period so we need convert it to days
FF_3Factor = FF_3Factor_All[0]/100
FF_3Factor.index = FF_3Factor.index.to_timestamp()+ MonthEnd(1) # make the␣
,→ periodIndex to end of month date

FF_3Factor = FF_3Factor[1:] # drop the first observation for␣

KB
,→ FF_3Factor too
Rf = FF_3Factor['RF'].mean() # avg monthly risk free rate

# # initial values
# # we use equal weights as the inital values. This will also make sure that the␣
,→ sum of weights = 1
weights_0 = [Link](len(tickers)-1)/len(tickers)
A
[*********************100%%**********************] 5 of 5 completed

Let’s run the optimization and print the output.

[208]: # # unconstrained minimization

optimal_weights = opt.fmin_bfgs(SharpeRatio, weights_0, args =␣
,→ (Returns[tickers],Rf))
A.

# # output
optimal_weights_all = [Link](optimal_weights, 1-sum(optimal_weights) )
optimal_weights_all = [Link](optimal_weights_all, index = tickers)
optimal_SharpeRatio = -1 * SharpeRatio(optimal_weights, Returns[tickers],Rf)

print('For the optimal Sharpe Ratio portfolio includes investing:')

print(optimal_weights_all.round(4)*100)

100
 print('The Sharpe Ratio of the optimal portfolio is: {:.2f}'.
,→format(optimal_SharpeRatio))

Optimization terminated successfully.

Current function value: -1.154536
Iterations: 12
Function evaluations: 70

I
Gradient evaluations: 14
For the optimal Sharpe Ratio portfolio includes investing:

AR
DIS 57.68
IBM -17.95
KO 15.09
F -29.98
PG 75.15
dtype: float64
The Sharpe Ratio of the optimal portfolio is: 1.15

5.9.2 Constrained Optimization

KB
Constrained optimization is frequently encountered in financial problems where parameters are only
meaningful in some particular range – for example, the weights in an investment portfolio must be
positive (no short selling) or the weights cannot be larger than 10% for each stock.

More formally the constrained optimization problems can be formulated:

minθ f (θ) subject to

A

g(θ) = 0 (equality)
h(θ) ≥ 0 (inequality)
θL ≤ θ ≤ θH (bounds)
A.

where the f is the objective function, g and h are the set of equations for the equality and inequality
constraints on the parameters.

In portfolio optimization, f could be the Sharpe ratio of the portfolio or expected returns adjusted for
risk. The equality constraint (θ) is usually used to ensure the sum of the weights of the ingredients
of a portfolio equals to one. The inequality or bounds constraints are used to set a limit on portfolio
weights. For instance, to impose the no short selling or to impose a balance weight across industries.

The function fmin_slsqp is the most general constrained optimizer and allows for equality, inequal-

101
 ity and bounds constraints. Constraints (here the functions g and h) are provided either as a list of
callable functions or as a single function which returns an array. The latter is simpler if there are
multiple constraints, especially if the constraints can be easily calculated using linear algebra.

Example: Consider the same portfolio optimization problem, but with some constraints. Say you
do not intend to short sell a stock (no negative portfolio weight. Also, you do not intend to over
invest in a stock (no more than 49% in one stock). As before, the sum of the portfolio weights needs
to be 100%

I
We first need to specify the objective function (SharpeRatio()). This is going to be very similar

AR
to the one we had in the unconstrained optimization, but we will remove the manual check for the
sum of the portfolios.

[209]: # # objective function: minimize the negative value of the Sharpe Ratio
def SharpeRatio(weights, Ret,Rf):
Ret_portfolio = (weights * Ret).sum(axis=1)
SR_portfolio = -1 * (Ret_portfolio.mean() - Rf)*12/Ret_portfolio.std()/np.
sqrt(12) #I minimize the negative value of the Sharpe Ratio
,→

return SR_portfolio
KB
Then we need to define the 3 constraints (sum of portfolio weights and two limits on portfolio
weights)

[210]: # # # Nb. 1
def equal_condition(x, *args):
return sum(x) - 1.0

weights_0 = [Link](len(tickers))/len(tickers)
A

print(weights_0)

# # # Nb. 2. and 3.
lowerBound = 0.00
upperBound = 0.49
weights_bounds = [(lowerBound, upperBound) for n in tickers]
A.

print(weights_bounds)

[0.2 0.2 0.2 0.2 0.2]

[(0.0, 0.49), (0.0, 0.49), (0.0, 0.49), (0.0, 0.49), (0.0, 0.49)]

We will use the same input data and initial weights as before. Let’s find the

102
[211]: # # constrained minimization
optimal_weights = opt.fmin_slsqp(func = SharpeRatio,
x0 = weights_0,
bounds = weights_bounds,
eqcons = [equal_condition],
args = (Returns[tickers],Rf))

Optimization terminated successfully (Exit mode 0)

I
Current function value: -1.008872112324576
Iterations: 5

AR
Function evaluations: 30
Gradient evaluations: 5

[212]: # # output
optimal_weights_all = [Link](optimal_weights, index = tickers)

optimal_SharpeRatio = -1 * SharpeRatio(optimal_weights, Returns[tickers],Rf)

KB
print('For the optimal Sharpe Ratio portfolio includes investing:')
print(optimal_weights_all.round(4)*100)
print('The Sharpe Ratio of the optimal portfolio is: {:.2f}'.
,→format(optimal_SharpeRatio))

For the optimal Sharpe Ratio portfolio includes investing:

DIS 33.97
IBM 0.00
KO 17.03
A

F 0.00
PG 49.00
dtype: float64
The Sharpe Ratio of the optimal portfolio is: 1.01

That means the optimal portfolio invests 24% less in DIS (from 57% to 33%) and 26% less in PG
A.

(75% to 49%). It does not invest in IBM and F (previously it was shorting them). But slightly
invests more in KO by 2% (from 15% to 17%).

Notice that the Sharpe ratio of the unconstrained optimization is higher (1.16 vs 1.01). By imposing
constraints we deviate from the optimal allocation.

103
 6 Python Coding Conventions
There are a number of common practices which can be adopted to produce Python code which looks
more like code found in other modules:

1. Use self-explanatory names for the variables and DataFrame. Something that you would
quickly understand when you look at your codes

2. Try to add sufficient comments and annotations to your codes so that you can understand

I
your work later.

AR
3. Use 4 spaces to indent blocks – avoid using tab, except when an editor automatically converts
tabs to 4 spaces

4. Avoid more than 4 levels of nesting, if possible

5. Limit lines to 79 characters. The \ symbol can be used to break long lines

6. Use two blank lines to separate functions, and one to separate logical sections in a function.

7. Use ASCII mode in text editors, not UTF-8

8. One module per import line

KB
9. Avoid from module import * (for any module). Use either from module import func1, func2
or import module as shortname.

10. Follow the NumPy guidelines for documenting functions More suggestions can be found in
PEP8 Style Guide for Python Code ([Link]
A
A.

104
 7 References and Python Resources
Python is an increasingly popular programming language. You can find numerous useful tutorials
on it on the web. For this document, I personally consulted the following materials frequently:

• On [Link], the “Python Programming: A Concise Introduction” online course by

Bill Boyd

• On [Link], the “Introduction to Python for Econometrics, Statistics and

I
Data Analysis” course by Kevin Sheppard

AR
• On [Link]/learning, the “Python: Data Analysis” and “Python Statistics Essential
Training” courses, by Michele Valisneri

• Python forum on [Link]

• and, many tutorials on [Link]

7.1 Best Python Libraries/Packages for Finance and Financial Data Scientists
An article from Majid AliAkbar ([Link]
KB
finance-financial-data-majid-aliakbar/), Published on March 28, 2017.

Finance professionals involved in data analytics and data science make use of R, Python and other
programming languages to perform analysis on a variety of data sets. Python has been gathering
a lot of interest and is becoming a language of choice for data analysis. Python also has a very
active community which does not shy from contributing to the growth of python libraries. If you
search on Github, a popular code hosting platform, you will see that there is a python package to
do almost anything you want. This article provides a list of the best python packages and libraries
used by finance professionals, quants, and financial data scientists.
A

• Numerical, Statistical & Data Structures

– numpy – NumPy is the fundamental package for scientific computing with Python. It is
a first-rate library for numerical programming and is widely used in academia, finance,
and industry. NumPy specializes in basic array operations.
– scipy – SciPy supplements the popular Numeric module, Numpy. It is a Python-based
A.

ecosystem of open-source software for mathematics, science, and engineering. It is also

used intensively for scientific and financial computation based on Python
– pandas – The pandas library provides high-performance, easy-to-use data structures and
data analysis tools for the Python programming language. Pandas focus is on the funda-
mental data types and their methods, leaving other packages to add more sophisticated
statistical functionality
– quantdsl – Quand DSL is domain specific language for quantitative analytics in finance
and trading. Quant DSL is a functional programming language for modeling derivative

105
 instruments.
– statistics – This is a built-in Python library for all basic statistical calculations

• Financial Instruments

– pyfin – Pyfin is a python library for performing basic options pricing in python
– vollib – vollib is a python library for calculating option prices, implied volatility and
greeks using Black, Black-Scholes, and Black-Scholes-Merton. vollib implements both
analytical and numerical greeks for each of the three pricing formulae.

I
– QuantPy – A framework for quantitative finance In python. Some current capabilities:

AR
Portfolio class that can import daily returns from Yahoo, Calculation of optimal weights
for Sharpe ratio and efficient frontier, and event profiler
– ffn – A financial function library for Python. ffn is a library that contains many useful
functions for those who work in quantitative finance. It stands on the shoulders of giants
(Pandas, Numpy, Scipy, etc.) and provides a vast array of utilities, from performance
measurement and evaluation to graphing and common data transformations.
– pynance – PyNance is open-source software for retrieving, analyzing and visualizing data
from stock and derivatives markets. It includes tools for generating features and labels
for machine learning algorithms.
KB
– tia – TIA is a toolkit that provides Bloomberg data access, easier pdf generation, back-
testing functionality, technical analysis functionality, return analysis and few windows
utils.

• Trading & Backtesting

– TA-Lib – TA-Lib is widely used by trading software developers requiring to perform

technical analysis of financial market data. It has an open-source API for python.
– trade – trade is a Python framework for the development of financial applications. A
A

trade app works like a service. The user informs the items he has in stock and a series of
subsequent occurrences (purchases, sales, whatsoever) with those or other items. trade
then calculates the effects of those occurrences and gives back the new amounts and costs
of the items in stock.
– zipline – Zipline is a Pythonic algorithmic trading library. It is an event-driven system
that supports both backtesting and live trading.
A.

– QuantSoftware Toolkit – Python-based open source software framework designed to

support portfolio construction and management. It is built the QSToolKit primarily
for finance students, computing students, and quantitative analysts with programming
experience.
– quantitative – Quantitative finance, and backtesting library. Quantitative is an event
driven and versatile backtesting library.
– analyzer – Python framework for real-time financial and backtesting trading strategies
– bt – bt is a flexible backtesting framework for Python used to test quantitative trading

106
 strategies.
– backtrader – Python Backtesting library for trading strategies
– pybacktest – Vectorized backtesting framework in Python / pandas, designed to make
your backtesting easier. It allows users to specify trading strategies using full power of
pandas, at the same time hiding all boring things like manually calculating trades, equity,
performance statistics and creating visualizations. Resulting strategy code is usable both
in research and production setting.

I
– pyalgotrade – PyAlgoTrade is an event driven algorithmic trading Python library. Al-
though the initial focus was on backtesting, paper trading is now possible

AR
– tradingWithPython – A collection of functions and classes for Quantitative trading
– pandas_talib – A Python Pandas implementation of technical analysis indicators
– algobroker – This is an execution engine for algo trading. The idea is that this python
server gets requests from clients and then forwards them to the broker API.
– finmarketpy – finmarketpy is a Python based library that enables you to analyze market
data and also to backtest trading strategies using a simple to use API, which has prebuilt
templates for you to define backtest.

• Risk Analysis
KB
– pyfolio – pyfolio is a Python library for performance and risk analysis of financial
portfolios. It works well with the Zipline open source backtesting library.
– empyrical – Common financial risk and performance metrics. Used by zipline and
pyfolio.
– finance – Financial Risk Calculations. Optimized for ease of use through class construc-
tion and operator overload.
– qfrm – Quantitative Financial Risk Management: awesome OOP tools for measuring,
managing and visualizing risk of financial instruments and portfolios.
A

– visualize-wealth – A library built in Python to construct, backtest, analyze, and

evaluate portfolios and their benchmarks.
– VisualPortfolio – This tool is used to visualize the performance of a portfolio

• Time Series

– ARCH – ARCH and other tools for financial econometrics in Python

A.

– statsmodels – Python module that allows users to explore data, estimate statistical
models, and perform statistical tests.
– dynts – A statistic package for python with emphasis on time series analysis. Built
around numpy, it provides several back-end time series classes including R-based objects
via rpy2.

107