Python Get started

Introduction to Python Programming

Tran Giang Son, [Link]@[Link]

ICT Department, USTH

Introduction to Python Programming Tran Giang Son, [Link]@[Link] 1 / 25

Python Get started

Python

Introduction to Python Programming Tran Giang Son, [Link]@[Link] 2 / 25

Python Get started

What

Introduction to Python Programming Tran Giang Son, [Link]@[Link] 3 / 25

Python Get started

What

• Programming language
• Open-source
• Interpreted
• High-level
• General-purpose

• Code readability
• Object Oriented Programming

Introduction to Python Programming Tran Giang Son, [Link]@[Link] 4 / 25

Python Get started

What

Introduction to Python Programming Tran Giang Son, [Link]@[Link] 5 / 25

Python Get started

What

Source

Introduction to Python Programming Tran Giang Son, [Link]@[Link] 6 / 25

Python Get started

Why

• Easy
• Flexible
• Readability
• Projects
• Jobs

Introduction to Python Programming Tran Giang Son, [Link]@[Link] 7 / 25

Python Get started

Why: Easy
• Syntax
• Natural
• Intuitive
• Python:
print("Hello world.")
• Java:
public class Test {
public static void main(String args[]) {
[Link]("Hello world.");
}
}

Introduction to Python Programming Tran Giang Son, [Link]@[Link] 8 / 25

Python Get started

Why: Flexible

• Script
• Backend
• Machine Learning, Deep Learning
• Apps
• Mobile
• Desktop
• Web

Introduction to Python Programming Tran Giang Son, [Link]@[Link] 9 / 25

Python Get started

Why: Readability

• Indents ftw!
• Line breaks ftw!

Introduction to Python Programming Tran Giang Son, [Link]@[Link] 10 / 25

Python Get started

Why: Projects

• Full list
• Browser
• Youtube-dl
• Music Player
• Video Editor
• Bittorent client
• Text Editor

Introduction to Python Programming Tran Giang Son, [Link]@[Link] 11 / 25

Python Get started

Why: Jobs!1!!

Introduction to Python Programming Tran Giang Son, [Link]@[Link] 12 / 25

Python Get started

Why: Jobs!1!!

Introduction to Python Programming Tran Giang Son, [Link]@[Link] 13 / 25

Python Get started

Why: Jobs!1!!

Introduction to Python Programming Tran Giang Son, [Link]@[Link] 14 / 25

Python Get started

Why: Jobs!1!!

Introduction to Python Programming Tran Giang Son, [Link]@[Link] 15 / 25

Python Get started

Why

That’s 4 ECTS btw. . .

Introduction to Python Programming Tran Giang Son, [Link]@[Link] 16 / 25

Python Get started

Why NOT?

• Intepretation, not compilation

• Slow
• GIL. . .

• Not optimized for

• Mobile
• Web client

Introduction to Python Programming Tran Giang Son, [Link]@[Link] 17 / 25

Python Get started

Get started

Introduction to Python Programming Tran Giang Son, [Link]@[Link] 18 / 25

Python Get started

Python

• Python
• Python 2.x: discontinued
• Python 3.x

• Download

Introduction to Python Programming Tran Giang Son, [Link]@[Link] 19 / 25

Python Get started

Versions, Environments, Packages

Introduction to Python Programming Tran Giang Son, [Link]@[Link] 20 / 25

Python Get started

Versions, Environments, Packages

• pyenv: manage Python versions
• virtualenv: isolate Python environments
• pip:
• install Python packages
• from Python Package Index

• conda:
• isolate environments
• install packages
• miniconda: minimal installer for conda
• anaconda: miniconda + bunch of pre-installed packages

Introduction to Python Programming Tran Giang Son, [Link]@[Link] 21 / 25

Python Get started

IDE
• Full fledged IDEs
• PyCharm
• Visual Studio Code

• Better live code

• Jupyter notebooks
• Spyder

• Simplicity: any text editor

• Atom
• Sublime Text
• Notepad
• vi/vim/nano. . .
Introduction to Python Programming Tran Giang Son, [Link]@[Link] 22 / 25
Python Get started

Jupyter Notebook

• Written in Python
• Fork of IPython
• Open-source Web app
• live code
• equations
• Computational output - visualizations
• explanatory text

• Popular for Data Science

• JupyterLab

Introduction to Python Programming Tran Giang Son, [Link]@[Link] 23 / 25

Python Get started

Jupyter Notebook

• Main components
• IPython
• ØMQ
• Tornado (web server)
• jQuery
• Bootstrap (front-end framework)
• MathJax

Introduction to Python Programming Tran Giang Son, [Link]@[Link] 24 / 25

Python Get started

Jupyter Notebook

• Install
• pip

pip install notebook

* conda
conda install -c conda-forge notebook
• Launch
jupyter notebook

Introduction to Python Programming Tran Giang Son, [Link]@[Link] 25 / 25

Expressions Data Types Conditions Functions Collections Loops Practice!

The Python Language

Tran Giang Son, [Link]@[Link]

ICT Department, USTH

The Python Language Tran Giang Son, [Link]@[Link] 1 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Expressions

The Python Language Tran Giang Son, [Link]@[Link] 2 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Interactive vs Script

• Interactive
• Type command
• Execute
• Wait for response

• Script
• All-in-one long sequences of statements
• python [Link]
• Shebang #! works

The Python Language Tran Giang Son, [Link]@[Link] 3 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Constants

• What
• Fixed values
• Value does not change over time

• Examples
• Numeric constants
• String constants
• Single quotes '
• Double quotes "

• Why: everywhere

The Python Language Tran Giang Son, [Link]@[Link] 4 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Constants

• How
>>> print(123)
123
>>> print(98.6)
98.6
>>> print('Hello world')
Hello world

The Python Language Tran Giang Son, [Link]@[Link] 5 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Variables

• What
• Named place in the memory to store data
• Access it later using name
• Modifiable at runtime

• Why: store temporary changable values

The Python Language Tran Giang Son, [Link]@[Link] 6 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Variables

• Variable name rules

• Letters, numbers, or underscores
• CaSe sEnSiTiVe
• Not allowed: starting with number

• Examples
• Good: spam, eggs, spam23, _speed
• Bad: 23spam, #sign, var.12
• Different: spam, Spam, SPAM

The Python Language Tran Giang Son, [Link]@[Link] 7 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Variables

• Reserved words
and del for is raise assert elif
from lambda return break else
global not try class except if or while
continue exec import pass
yield def finally in print

The Python Language Tran Giang Son, [Link]@[Link] 8 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Statements

• What: combination of operator and its operand(s)

• Operator: symbol indicating a calculation
• One or more operands

• Numeric expression
• + Addition
• - Subtraction
• * Multiplication
• / Division
• ** Power
• % Remainder

The Python Language Tran Giang Son, [Link]@[Link] 9 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Statements

• Numeric expression
>>> x = 2
>>> x = x + 2
>>> print(x) >>> j = 23
4 >>> k = j % 5
>>> y = 440 * 12 >>> print(k)
>>> print(y) 3
5280 >>> print(4 ** 3)
>>> z = y / 1000 64
>>> print(z)
5

The Python Language Tran Giang Son, [Link]@[Link] 10 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Statements

• Mixing Integer and Floats: convert everything to float.

>>> print(99 / 100)
0
>>> print(99 / 100.0)
0.99
>>> print(99.0 / 100)
0.99
>>> print(1 + 2 * 3 / 4.0 - 5)
-2.5
>>>

The Python Language Tran Giang Son, [Link]@[Link] 11 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Data Types

The Python Language Tran Giang Son, [Link]@[Link] 12 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

What

• Variables, literals, and constants have a “data type”

Type Examples
Integer 0, 12, 5, -5
Float 4.5, 3.99, 0.1
String “Hi”, “Hello”, “Hi there!”"
Boolean True, False
List [ “hi”, “there”, “you” ]
Tuple (4, 2, 7, 3)

The Python Language Tran Giang Son, [Link]@[Link] 13 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

What: Boolean

• bool
• 2 possible values: True, False

The Python Language Tran Giang Son, [Link]@[Link] 14 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

What: Integer

• int
• Unbounded.
>>> i=10**100
>>> type(i)
<class 'int'>
>>> i
1000000000000000000000000000000000000000000000000000000

The Python Language Tran Giang Son, [Link]@[Link] 15 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

What: Float

• float
• Digits and Exponents
>>> 2.5
>>> 2e4
>>> 0.00001
>>> 1000020000300004

The Python Language Tran Giang Son, [Link]@[Link] 16 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

What: Strings
• str
• Series of Unicode characters
• Character: String of length 1
• Enclosed by a pair of single or double quotes
• Multiline: triple quote
• '''
• """

>>> s="""This is
... a Multiline string
... for example"""
>>> s
'This is \na Multiline string \nfor example'
The Python Language Tran Giang Son, [Link]@[Link] 17 / 66
Expressions Data Types Conditions Functions Collections Loops Practice!

Dynamically typing

• Dynamically typed variables

• Types are automatically managed
C, Java
Python
int a;
a = 5
float b;
a = 0.43
a = 5;
a = "Hello"
b = 0.43;

The Python Language Tran Giang Son, [Link]@[Link] 18 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Number Conversion

>>> print(float(99) / 100)

0.99
>>> i = 42
>>> type(i)
<class 'int'>
• int() >>> f = float(i)
• float() >>> print(f)
42.0
>>> type(f)
<class 'float'>
>>> print(1 + 2 * float(3) / 4 - 5)
-2.5
>>>

The Python Language Tran Giang Son, [Link]@[Link] 19 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Number Conversion
>>> sval = '123'
>>> type(sval)
<class 'str'>
>>> print(sval + 1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module
TypeError: can only concatenate str
• Also works with >>> ival = int(sval)
>>> type(ival)
strings!
<class 'int'>
>>> print(ival + 1)
124
>>> nsv = 'hello bob'
>>> niv = int(nsv)
Traceback (most recent call last):
File "<stdin>", line 1, in <module
The Python Language ValueError: invalid literal for
Tran Giang Son, [Link]@[Link] 20 /int(
66
Expressions Data Types Conditions Functions Collections Loops Practice!

String Operators

• Some operators apply to strings

• + concatenation
• * multiple concatenation
• in, not in contains/not contains

>>> print('abc' + '123')

abc123
>>> print('Hi' * 5)
HiHiHiHiHi
>>> "US" in "AmongUS"
True
>>> "us" not in "AmongUS"
True

The Python Language Tran Giang Son, [Link]@[Link] 21 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

String Operators
• Substring: string[index:end:step]
• index, end
• >=0: start from beginning of string
• <: start from end of string
• Can be omitted

+---+---+---+---+---+---+
| P | y | t | h | o | n |
+---+---+---+---+---+---+
| 0 | 1 | 2 | 3 | 4 | 5 |
+---+---+---+---+---+---+
|-6 |-5 |-4 |-3 |-2 |-1 |
+---+---+---+---+---+---+
• step: How many letters to skip
The Python Language Tran Giang Son, [Link]@[Link] 22 / 66
Expressions Data Types Conditions Functions Collections Loops Practice!

String Operators

• string[index:end:step]
>>> s = "Advanced Programming with Python"
>>> s[:20]
>>> s[9] 'Advanced Programming'
'P' >>> s[9:]
>>> s[9:20] 'Programming with Python'
'Programming' >>> s[-6:-4]
>>> s[9:20:2] 'Py'
'Pormig' >>> s[-6:]
'Python'

The Python Language Tran Giang Son, [Link]@[Link] 23 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

String Formats

• Similar to C’s printf()

• Previously, in pre-3.6 Python
print("Greeting, {}. You are {}".format(name, age))
• From Python 3.6 onward: f-string, or formatted string
literals
print(f"Greeting, {name}. You are {age}")

The Python Language Tran Giang Son, [Link]@[Link] 24 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Comments

• What? # starts a line comment

• Why?
• Description of code block
• Document some extra info
• Turn off a line of code

The Python Language Tran Giang Son, [Link]@[Link] 25 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Comments

>>> s = "USTH"
>>> # print("nobody cares")
>>> print(s)
USTH

The Python Language Tran Giang Son, [Link]@[Link] 26 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Conditions

The Python Language Tran Giang Son, [Link]@[Link] 27 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Indentation Rules
• Increase indent after an if statement or for statement (after :
)
• Equivalent to C, Java’s {

• Maintain indent to indicate the scope of the block

• Which lines are affected by the if/for

• Reduce indent to back to the level of the if statement or for

statement to indicate the end of the block
• Equivalent to C, Java’s }

• Blank lines are ignored - they do not affect indentation

• Comments on a line by themselves are ignored w.r.t.
indentation

The Python Language Tran Giang Son, [Link]@[Link] 28 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Indentation Rules

• Python cares a lot about how far line is indented

• Don’t mix tabs and spaces
• “indentation errors” even if everything looks fine

• Use one only

• Most text editors can turn tabs into spaces - make sure to
enable this feature

The Python Language Tran Giang Son, [Link]@[Link] 29 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

if - else

x = 5
if x < 10:
print('Smaller than 10')
else:
print('Bigger than 10')
print('End')

The Python Language Tran Giang Son, [Link]@[Link] 30 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Nested if - else

x = 5
if x < 10:
print('Smaller than 10')
if x > 5:
print(' Still bigger than 5')
else:
print('Bigger than 10')
print('End')

The Python Language Tran Giang Son, [Link]@[Link] 31 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

if - else - if - else

x = 21
if x < 10:
print('Smaller than 10')
elif x < 20:
print('Smaller than 20')
else:
print('Bigger than 20')
print('End')

The Python Language Tran Giang Son, [Link]@[Link] 32 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Functions

The Python Language Tran Giang Son, [Link]@[Link] 33 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

What & Why

• Group of related statements performing a specific task

• Break programs into small chunks
• Better code organization
• Code reusable

The Python Language Tran Giang Son, [Link]@[Link] 34 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

How

• Definition
• Function Name
• Parentheses
• Arguments

def function_name(arguments):
"""docstring"""
statement1
statement2
...
• Call
function_name("a value")

The Python Language Tran Giang Son, [Link]@[Link] 35 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Examples

def greet(name):
"""
This function greets to
the person passed in as
a parameter
"""
print("Hello, " + name + ". Good morning!")

greet("Emmanuel Macron")

The Python Language Tran Giang Son, [Link]@[Link] 36 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Examples

• len(arg): number of elements in arg

• print(args): write args to stdout
• input(prompt): print(prompt), wait and read user input
from stdin, return the entered string

The Python Language Tran Giang Son, [Link]@[Link] 37 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Collections

The Python Language Tran Giang Son, [Link]@[Link] 38 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

What

• Multiple objects are grouped together

• Main types
• Sets
• Sequences
• Maps
• Streams

The Python Language Tran Giang Son, [Link]@[Link] 39 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Set

• Unordered collection of items

• No duplication
• Operators
• [Link](s2): no common element
• s1 <= s2, [Link](s2): s1 ⊆ s2
• s1 >= s2, [Link](s2): s1 ⊇ s2
• s3 = s1 | s2, s3 = [Link](s2): s3 = s1 ∪ s2
• s3 = s1 & s2, s3 = [Link](s2): s3 = s1 ∩ s2
• s3 = s1 - s2, s3 = [Link](s2): s3 = s1 \ s2

The Python Language Tran Giang Son, [Link]@[Link] 40 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Sequences
• Ordered collection of items
• Can have duplications
• Positioned access
• Slicing similar to strings
• seq[start:end:step]

• Implementations
• list
• tuple
• range

• Others:
• str
•
The Python Language Tran Giang Son, [Link]@[Link] 41 / 66
Expressions Data Types Conditions Functions Collections Loops Practice!

Lists

• Mutable sequence
• Values can be changed later

• Flexible, widely used

• Comma separated declaration
>>> names = [ "ICT", "ict" ]

The Python Language Tran Giang Son, [Link]@[Link] 42 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Lists

>>> names = [ "ICT", "ict" ]

The Python Language Tran Giang Son, [Link]@[Link] 43 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Lists

>>> names = [ "ICT", "ict" ]

• + append elements at the end, same or .extend()
>>> names += ["Ict"]
>>> names
['ICT', 'ict', 'Ict']

The Python Language Tran Giang Son, [Link]@[Link] 43 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Lists

>>> names = [ "ICT", "ict" ]

• + append elements at the end, same or .extend()
>>> names += ["Ict"]
>>> names
['ICT', 'ict', 'Ict']
• = replaces single value
>>> names[1] = "I See Tea"
>>> names
['ICT', 'I See Tea', 'Ict']

The Python Language Tran Giang Son, [Link]@[Link] 43 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Lists

>>> names
['ICT', 'I See Tea', 'Ict']

The Python Language Tran Giang Son, [Link]@[Link] 44 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Lists

>>> names
['ICT', 'I See Tea', 'Ict']
• = replaces bunch of values
>>> names[1:3] = [ "Icy Tea", "I See Tea" ]
>>> names
['ICT', 'Icy Tea', 'I See Tea']

The Python Language Tran Giang Son, [Link]@[Link] 44 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Lists

>>> names
['ICT', 'I See Tea', 'Ict']
• = replaces bunch of values
>>> names[1:3] = [ "Icy Tea", "I See Tea" ]
>>> names
['ICT', 'Icy Tea', 'I See Tea']
• += append elements at middle, same as .insert()
>>> names[1:1] += [ "Ice City" ]
>>> names
['ICT', 'Ice City', 'Icy Tea', 'I See Tea']

The Python Language Tran Giang Son, [Link]@[Link] 44 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Lists

>>> names
['ICT', 'Ice City', 'Icy Tea', 'I See Tea']

The Python Language Tran Giang Son, [Link]@[Link] 45 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Lists

>>> names
['ICT', 'Ice City', 'Icy Tea', 'I See Tea']
• sort() elements
>>> [Link]()
>>> names
['I See Tea', 'ICT', 'Ice City', 'Icy Tea']

The Python Language Tran Giang Son, [Link]@[Link] 45 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Lists

>>> names
['ICT', 'Ice City', 'Icy Tea', 'I See Tea']
• sort() elements
>>> [Link]()
>>> names
['I See Tea', 'ICT', 'Ice City', 'Icy Tea']
• del delete elements
>>> del names[1]
>>> names
['I See Tea', 'Ice City', 'Icy Tea']

The Python Language Tran Giang Son, [Link]@[Link] 45 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Lists

>>> names
['I See Tea', 'Ice City', 'Icy Tea']

The Python Language Tran Giang Son, [Link]@[Link] 46 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Lists

>>> names
['I See Tea', 'Ice City', 'Icy Tea']
• .remove() occurrences
>>> [Link]("Icy Tea")
>>> names
['Ice City', 'I See Tea']

The Python Language Tran Giang Son, [Link]@[Link] 46 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Range

• Generates a series of integers

• Very popular, widely used
• range(end) $ = [0..end-1]$.
• range(start, end) $ = [start..end-1]$.
• range(start, end, step) $ = {x | x = start + k * step, x
< end}$

The Python Language Tran Giang Son, [Link]@[Link] 47 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Range

>>> nums = range(10,15)

>>> print(nums)
range(10, 15)
>>> [x for x in nums]
[10, 11, 12, 13, 14]

The Python Language Tran Giang Son, [Link]@[Link] 48 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Tuples

• Immutable sequence
• Contain any type of element.
• A very common use of tuples is a simple representation of
pairs
• Positition (x, y)
• Size (w, h)
• ...

The Python Language Tran Giang Son, [Link]@[Link] 49 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Tuples

• Comma generated expression

>>> p = 10, 20
>>> p
(10, 20)
>>> p = (20, 40)
>>> p
(20, 40)
>>> type(p)
<class 'tuple'>
>>> p[1]=1
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignm

The Python Language Tran Giang Son, [Link]@[Link] 50 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Maps

• Key/value pairs
• Key must be unique
• Similar to JSON objects

• Unordered, mutable
• Implemented by dict

The Python Language Tran Giang Son, [Link]@[Link] 51 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Maps

• Initialization
info = {"name": "USTH", "age": 10, \
"depts": [ "ict", "ged"] }
• Key operations
• in, not in: check key presence

>>> "name" in info

True
• max, min of key

>>> max(info)
'name'

The Python Language Tran Giang Son, [Link]@[Link] 52 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Maps: Operations

>>> info["name"]
'USTH'
• Value operations >>> info["age"] = 11
• d[k]: get value by key >>> info["age"]
• d[k] = v: set value to
11
key
• del d[k] remove key >>> del info["depts"]
from dict >>> info
>>> info
{'name': 'USTH', 'age': 11}

The Python Language Tran Giang Son, [Link]@[Link] 53 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Maps: Methods

• [Link](k[, default]): same as d[k], fallback to default

if key not found
• [Link](k[, default]): del d[k] and return previously
deleted d[k], fallback to default if key not found
• [Link](d2): for each key in d2, sets d1[key] to
d2[key], replacing the existing value if there was one
• [Link](): returns list of keys
• [Link](): returns list of values
• [Link](): returns list of (key,value) tuples.

The Python Language Tran Giang Son, [Link]@[Link] 54 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Maps: Methods
>>> info = {"name": "USTH", "age": 10, \
"depts": [ "ict", "ged"] }
>>> [Link]("name")
'USTH'
>>> [Link]("address", "Earth")
'Earth'
>>> [Link]("depts")
['ict', 'ged']
>>> [Link]()
dict_keys(['name', 'age'])
>>> [Link]()
dict_values(['USTH', 10])
>>> [Link]()
dict_items([('name', 'USTH'), ('age', 10)])

The Python Language Tran Giang Son, [Link]@[Link] 55 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Loops

The Python Language Tran Giang Son, [Link]@[Link] 56 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

What

• Loops (repeated steps) have iteration variables

• Iteration variable changes each time through a loop
• Often these iteration variables go through a sequence of
numbers.

The Python Language Tran Giang Son, [Link]@[Link] 57 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

What

5
n = 5
4
while n > 0 :
3
print(n)
2
n = n – 1
1
print('Blastoff!')
Blastoff!

The Python Language Tran Giang Son, [Link]@[Link] 58 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

break

• The break statement ends the current loop

• Jumps to the statement immediately following the loop
while True:
line = input('> ')
if line == 'done':
break
print(line)
print('Done!')

The Python Language Tran Giang Son, [Link]@[Link] 59 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

continue

• The continue statement ends the current iteration

• Jumps to the top of the loop and starts the next iteration
while True:
> hello there
line = input('> ')
hello there
if line[0] == '#' :
> # don't print this
continue
> print this!
if line == 'done' :
print this!
break
> done
print(line)
Done!
print('Done!')

The Python Language Tran Giang Son, [Link]@[Link] 60 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

range()

x = range(5)
print(x)
[0, 1, 2, 3, 4]
• range()
• built-in function
• returns sequence of x = range(3, 7)
numbers in a range print(x)
• Very useful in “for” loops [3, 4, 5, 6]
• 1, 2, or 3 arguments
x = range(10, 1, -2)
print(x)
[10, 8, 6, 4, 2]

The Python Language Tran Giang Son, [Link]@[Link] 61 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

range()

• for statement
• Iterates over the members of a sequence in order
• Executes the block each time

for i in <collection>
<loop body>
• Examples
n = 5
while n > 0: for n in range(5, 0, -1):
print(n) print(n)
n = n – 1 print('Blastoff!')
print('Blastoff!')

The Python Language Tran Giang Son, [Link]@[Link] 62 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Practice!

The Python Language Tran Giang Son, [Link]@[Link] 63 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Practical Work 0: git/github

• Fork the course’s git repository to your github account

• [Link]

• Clone your forked repository to your home directory

• git@[Link]:<YourAccount>/[Link]

• Edit «[Link]», write your name as instructed.

• Make a new commit with a message “First student commit”
• Push your new commit to your forked github repository

The Python Language Tran Giang Son, [Link]@[Link] 64 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Practical work 1: student mark management

• Make a new Python program

• Name it «[Link]»
• Use tuples, dicts, lists, NO objects/classes
• Build a student mark management system

The Python Language Tran Giang Son, [Link]@[Link] 65 / 66

Expressions Data Types Conditions Functions Collections Loops Practice!

Practical work 1: student mark management

• Functions
• Input functions:
• Input number of students in a class
• Input student information: id, name, DoB
• Input number of courses
• Input course information: id, name
• Select a course, input marks for student in this course

• Listing functions:
• List courses
• List students
• Show student marks for a given course

• Push your work to corresponding forked Github repository

The Python Language Tran Giang Son, [Link]@[Link] 66 / 66
Review Object and Class Inheritance Polymorphism Encapsulation

OOP in Python

Tran Giang Son, [Link]@[Link]

ICT Department, USTH

OOP in Python Tran Giang Son, [Link]@[Link] 1 / 34

Review Object and Class Inheritance Polymorphism Encapsulation

Review

OOP in Python Tran Giang Son, [Link]@[Link] 2 / 34

Review Object and Class Inheritance Polymorphism Encapsulation

Questions!?!!!1

• What is a class? What is an object?

• What is the difference between an object and a class?
• Where do objects come from (how do you create one)?
• How many objects of a given class can you have at a given
time?
• Each data type in Java (and in many other languages) can
be classified as one of two kinds. What are they, and how
are they different?

1
Or exam?!
OOP in Python Tran Giang Son, [Link]@[Link] 3 / 34
Review Object and Class Inheritance Polymorphism Encapsulation

Questions!?!!!

• What is a primitive type? What is a reference type (or

object type)?
• What is a method?
• Can a class have more than one method with the same
name? If so, are there any restrictions?
• What is a parameter? What is a return value?
• Can parameters and return values be primitive types?
Reference types?

OOP in Python Tran Giang Son, [Link]@[Link] 4 / 34

Review Object and Class Inheritance Polymorphism Encapsulation

Questions!?!!!

• What is type matching or type conformance?

• What is a constructor?
• What is assignment? How is it different for reference types
versus primitive types?
• What are accessor methods and mutator methods?
• What is abstraction and why is it a good thing?

OOP in Python Tran Giang Son, [Link]@[Link] 5 / 34

Review Object and Class Inheritance Polymorphism Encapsulation

Questions!?!!!

• What is inheritance?
• What is an inheritance hierarchy?
• What is a subclass? A superclass?
• What are the advantages of using inheritance?
• What is the difference between an is-a and a has-a
relationships?

OOP in Python Tran Giang Son, [Link]@[Link] 6 / 34

Review Object and Class Inheritance Polymorphism Encapsulation

Questions!?!!!

• What is polymorphism?
• What are overriding and overloading? Are they the same?
Give examples.
• What does the keyword super mean? When is it used?
• What does the keyword protected mean? When is it used,
and what does it do?
• What is meant by the static and dynamic types of a
variable?

OOP in Python Tran Giang Son, [Link]@[Link] 7 / 34

Review Object and Class Inheritance Polymorphism Encapsulation

Questions!?!!!

• What is an abstract class? When are they useful? Give an

example.
• What is multiple inheritance? Why is it useful? Can it be
done in Java? Is there a substitute for it?
• What is an interface? Why are they useful?

OOP in Python Tran Giang Son, [Link]@[Link] 8 / 34

Review Object and Class Inheritance Polymorphism Encapsulation

Object and Class

OOP in Python Tran Giang Son, [Link]@[Link] 9 / 34

Review Object and Class Inheritance Polymorphism Encapsulation

Previously, on PW #1
• Functions
• Input functions:
• Input number of students in a class
• Input student information: id, name, DoB
• Input number of courses
• Input course information: id, name
• Select a course, input marks for student in this course

• Listing functions:
• List courses
• List students
• Show student marks for a given course

OOP in Python Tran Giang Son, [Link]@[Link] 10 / 34

Review Object and Class Inheritance Polymorphism Encapsulation

Why

• Lists, dicts, tuples could be OOP’ed

• Student
• Course
• StudentMark

• Easier to manage
• Close to real-world management

OOP in Python Tran Giang Son, [Link]@[Link] 11 / 34

Review Object and Class Inheritance Polymorphism Encapsulation

Why

Object-Oriented Programming
Procedural Programming • Variables and related
• Variables and related
functions are bound
functions are separated
together
• Programs is divided into
• Programs are divided into
functions
objects

OOP in Python Tran Giang Son, [Link]@[Link] 12 / 34

Review Object and Class Inheritance Polymorphism Encapsulation

How

• Define a class
class <ClassName>
• Define a method
def <methodName>([args])
• Define a constructor
def __init__([args])
• Create an object from class
<obj> = <ClassName>([args])
• self: current object

OOP in Python Tran Giang Son, [Link]@[Link] 13 / 34

Review Object and Class Inheritance Polymorphism Encapsulation

How

class Person:
def print(self):
print("Name:", [Link])
print("Age:", [Link])

def __init__(self, n, a):

[Link] = n
[Link] = a

macron = Person("Emmanuel Macron")

OOP in Python Tran Giang Son, [Link]@[Link] 14 / 34

Review Object and Class Inheritance Polymorphism Encapsulation

How

• Call an object’s method

<obj>.<methodName>([args])
• Accessing an object’s attribute
<obj>.<attr> = "values"

OOP in Python Tran Giang Son, [Link]@[Link] 15 / 34

Review Object and Class Inheritance Polymorphism Encapsulation

How

• Object comparison: __lt__ method

• Compares current object with another instance
• Return True if less than2 the other instance

def __lt__(self, other):

return [Link] < [Link]

2
Hence its name is __lt__
OOP in Python Tran Giang Son, [Link]@[Link] 16 / 34
Review Object and Class Inheritance Polymorphism Encapsulation

How

• Object’s string representation: __str__ method

• Defines how an object will be stringified
• Mostly when using with print()

def __str__(self):
return f"My name is {[Link]}. I am {[Link]}."

OOP in Python Tran Giang Son, [Link]@[Link] 17 / 34

Review Object and Class Inheritance Polymorphism Encapsulation

How: complete example [Link]

#!/usr/bin/env python3
class Person:
def __init__(self, n, a):
[Link] = n
[Link] = a

def describe(self):
print("Name:", [Link])
print("Age:", [Link])

def __lt__(self, other):

return [Link] < [Link]

def __str__(self):
return f"My name is {[Link]}. I am {[Link]}."

macron = Person("Emmanuel Macron", 43)

[Link]()
print(macron)

biden = Person("Joe Biden", 78)

print(f"Macron is younger: {macron < biden}")
OOP in Python Tran Giang Son, [Link]@[Link] 18 / 34
Review Object and Class Inheritance Polymorphism Encapsulation

How: complete example

$ ./[Link]

Name: Emmanuel Macron

Age: 43
My name is Emmanuel Macron. I am 43.
Macron is younger: True

OOP in Python Tran Giang Son, [Link]@[Link] 19 / 34

Review Object and Class Inheritance Polymorphism Encapsulation

Inheritance

OOP in Python Tran Giang Son, [Link]@[Link] 20 / 34

Review Object and Class Inheritance Polymorphism Encapsulation

Defining Inheritance

• Define a child class with a superclass in parentheses

• All methods and attributes from superclass will be inherited
to subclass
class Person:
# already defined before...

class President(Person):
def set_term(self, term):
print(f"Setting term to {term}")
[Link] = term

OOP in Python Tran Giang Son, [Link]@[Link] 21 / 34

Review Object and Class Inheritance Polymorphism Encapsulation

Defining Inheritance

• Using the newly defined class and method

print("Macron is now President")
macron = President("Emmanuel Macron", 43)
macron.set_term(25) # from President
[Link]() # from Person
$ ./[Link]
Macron is now President
Setting term to 25
Name: Emmanuel Macron
Age: 43

OOP in Python Tran Giang Son, [Link]@[Link] 22 / 34

Review Object and Class Inheritance Polymorphism Encapsulation

Checking inheritance

• Built-in functions isinstance() and issubclass()

• isinstance() returns True if the object is an instance of
the class or other classes derived from it
• issubclass() is used to check for class inheritance.

OOP in Python Tran Giang Son, [Link]@[Link] 23 / 34

Review Object and Class Inheritance Polymorphism Encapsulation

Checking inheritance

print(f"Macron is President: {isinstance(macron, President)}");

print(f"Macron is Person: {isinstance(macron, Person)}");
print(f"President is Person: {issubclass(President, Person)}")
print(f"Person is President: {issubclass(Person, President)}")

$ ./[Link]

Macron is President: True

Macron is Person: True
President is Person: True
Person is President: False

OOP in Python Tran Giang Son, [Link]@[Link] 24 / 34

Review Object and Class Inheritance Polymorphism Encapsulation

Multiple Inheritance

• Python supports multiple inheritance

• Simply by adding more base class into the parentheses
class Person:
# already defined before...

class Employee:
def work(self):
print("I should be paid...")

class President(Person, Employee):

# already defined before...

OOP in Python Tran Giang Son, [Link]@[Link] 25 / 34

Review Object and Class Inheritance Polymorphism Encapsulation

Multiple Inheritance

print("Macron is now President")

macron = President("Emmanuel Macron", 43)
macron.set_term(25) # from President
[Link]() # from Person
[Link]() # from Employee
$ ./[Link]
Setting term to 25
Name: Emmanuel Macron
Age: 43
I should be paid...

OOP in Python Tran Giang Son, [Link]@[Link] 26 / 34

Review Object and Class Inheritance Polymorphism Encapsulation

Polymorphism

OOP in Python Tran Giang Son, [Link]@[Link] 27 / 34

Review Object and Class Inheritance Polymorphism Encapsulation

Method overrides
• A superclass’s method can be overridden, simply by deffing
the same method name in the subclass
• A superclass instance can be accessed using super() in the
subclass
class Person:
# already defined before...

class President(Person, Employee):

# something before
def describe(self):
super().describe()
print("Term:", [Link])

def work(self):
super().work() # from Employee
OOP in Python Tran Giang Son, [Link]@[Link] 28 / 34
Review Object and Class Inheritance Polymorphism Encapsulation

Method overrides

• Using the overridden method

print("Macron is now President")
macron = President("Emmanuel Macron", 43)
macron.set_term(25) # from President
[Link]() # from Person
[Link]() # from President
$ ./[Link]
Setting term to 25
Name: Emmanuel Macron
Age: 43
I am well paid!
President is well paid!

OOP in Python Tran Giang Son, [Link]@[Link] 29 / 34

Review Object and Class Inheritance Polymorphism Encapsulation

Encapsulation

OOP in Python Tran Giang Son, [Link]@[Link] 30 / 34

Review Object and Class Inheritance Polymorphism Encapsulation

Private / Public access

• public by default
• No specified keyword
• Use underscore prefixes
• name: public
• _name: protected
• __name: private

OOP in Python Tran Giang Son, [Link]@[Link] 31 / 34

Review Object and Class Inheritance Polymorphism Encapsulation

Private / Public access

• Accessor methods / Mutator methods

• Getter / Setter
class Employee:
def __init__(self):
self.__salary = 0

def _get_salary(self):
return self.__salary

def set_salary(self, salary):

self.__salary = salary

def work(self):
if self.__salary == 0:
print("I should be paid...")
else:
print("I am well paid!")

OOP in Python Tran Giang Son, [Link]@[Link] 32 / 34

Review Object and Class Inheritance Polymorphism Encapsulation

Private / Public access

print("Macron is now President")
macron = President("Emmanuel Macron", 43)
macron.set_term(25) # from President
[Link]() # from Person
macron.set_salary(1000) # from Employee
[Link]() # from President
print(f"Macron salary is {macron._get_salary()}")
print(f"Macron salary is {macron.__salary}")

$ ./[Link]
Macron is now President
Setting term to 25
Name: Emmanuel Macron
Age: 43
President is well paid!
Macron salary is 1000
Traceback (most recent call last):
File ".../[Link]", line 59, in <module>
print(f"Macron's salary is {macron.__salary}")
AttributeError: 'President' object has no attribute '__salary'

OOP in Python Tran Giang Son, [Link]@[Link] 33 / 34

Review Object and Class Inheritance Polymorphism Encapsulation

Practical work 2: OOP’ed student mark management

• Copy your practical work 1 to [Link]

• Make it OOP’ed
• Same functions
• Proper attributes and methods
• Proper encapsulation
• Proper polymorphism
• e.g. .input(), .list() methods

• Push your work to corresponding forked Github repository

OOP in Python Tran Giang Son, [Link]@[Link] 34 / 34

Modules Packages Practice!

Modules and Packages

Tran Giang Son, [Link]@[Link]

ICT Department, USTH

Modules and Packages Tran Giang Son, [Link]@[Link] 1 / 19

Modules Packages Practice!

Intro

Modules and Packages Tran Giang Son, [Link]@[Link] 2 / 19

Modules Packages Practice!

Modules

Modules and Packages Tran Giang Son, [Link]@[Link] 3 / 19

Modules Packages Practice!

What

• Module: reusable piece of code

• Can be from external sources
• Regular .py file with defs and classes

Modules and Packages Tran Giang Son, [Link]@[Link] 4 / 19

Modules Packages Practice!

What

• Similar to .ko, .so, .dll, . . .

Modules and Packages Tran Giang Son, [Link]@[Link] 5 / 19

Modules Packages Practice!

Why

• Modularity
• Reusability
• Shareibility
• Maintainability

Modules and Packages Tran Giang Son, [Link]@[Link] 6 / 19

Modules Packages Practice!

How: Making a module

• Create a separated .py file

• define functions, classes as usual
• define __init__(), as needed

Modules and Packages Tran Giang Son, [Link]@[Link] 7 / 19

Modules Packages Practice!

How: Using a module

• import a module to the global namespace

• No path, no extension

• Use functions, classes, constants provided by module

• <module>.<method/class/const>

>>> import math

>>> print([Link])
3.141592653589793

Modules and Packages Tran Giang Son, [Link]@[Link] 8 / 19

Modules Packages Practice!

How: Using a module

• Shortcut to the global namespace

• from <module> import <func/class/const>
• Use <func/class/const> directly

>>> from math import pi

>>> print(pi)
3.141592653589793
>>> from math import *
>>> print(e)
2.718281828459045

Modules and Packages Tran Giang Son, [Link]@[Link] 9 / 19

Modules Packages Practice!

How: Using a module

• Aliasing
• from <module> import <func/class/const> as <alias>
• Use <alias>

>>> import numpy as np

>>> [Link]
3.141592653589793

Modules and Packages Tran Giang Son, [Link]@[Link] 10 / 19

Modules Packages Practice!

Packages

Modules and Packages Tran Giang Son, [Link]@[Link] 11 / 19

Modules Packages Practice!

What

• A bunch of related modules

• [optional] A bunch of sub-packages
• [optional] A bunch of sub-sub-packages

Modules and Packages Tran Giang Son, [Link]@[Link] 12 / 19

Modules Packages Practice!

Why

• Higher level of modularity

• Less import modules from the same packages
• Module name A.B designates a submodule named B in a
package named A.

Modules and Packages Tran Giang Son, [Link]@[Link] 13 / 19

Modules Packages Practice!

How

• Install from Python Package Index (PyPI)

• pip install <packageName>

• Write your own package

• Add your modules
• Add a dedicated file __init__.py for initialization

Modules and Packages Tran Giang Son, [Link]@[Link] 14 / 19

Modules Packages Practice!

How

• import a module in a specific package

import Package1.PackageModule1
• import a whole package
import Package1
• Should also import modules in package __init__.py for
automatic module imports
import Module1
import Module2
import Module3

Modules and Packages Tran Giang Son, [Link]@[Link] 15 / 19

Modules Packages Practice!

How

• Package can be nested

• Sub-package inside a package
• Sub-sub-package inside a sub-package

• Just make an __init__.py, even empty one

Modules and Packages Tran Giang Son, [Link]@[Link] 16 / 19

Modules Packages Practice!

Practice!

Modules and Packages Tran Giang Son, [Link]@[Link] 17 / 19

Modules Packages Practice!

Practical work 3: some maths and decorations

• Copy your practical work 2 to [Link]

• Use math module to round-down student scores to 1-digit
decimal upon input, floor()
• Use numpy module and its array to
• Add function to calculate average GPA for a given student
• Weighted sum of credits and marks

• Sort student list by GPA descending

• Decorate your UI with curses module

• Push your work to corresponding forked Github repository

Modules and Packages Tran Giang Son, [Link]@[Link] 18 / 19

Modules Packages Practice!

Practical work 4: modularization

• Split your program [Link] to modules

and packages in a new pw4 directory
• [Link]: module for input
• [Link]: module for curses output
• domains: package for classes
• [Link]: main script for coordination

• Push your work to corresponding forked Github repository

Modules and Packages Tran Giang Son, [Link]@[Link] 19 / 19

Files Directories Practice!

Files and Directories

Tran Giang Son, [Link]@[Link]

ICT Department, USTH

Files and Directories Tran Giang Son, [Link]@[Link] 1 / 33

Files Directories Practice!

Reviews

• What is a file? What is a directory?

• What is a symlink?
• Shell commands:
• How to list files in a directory?
• How to show a file’s content?
• How to print all lines of a file containing a specific string?

Files and Directories Tran Giang Son, [Link]@[Link] 2 / 33

Files Directories Practice!

Files

Files and Directories Tran Giang Son, [Link]@[Link] 3 / 33

Files Directories Practice!

What

• Everything in UNIX is a file

• Named locations on disk to store information
• Text file
• Binary file

Files and Directories Tran Giang Son, [Link]@[Link] 4 / 33

Files Directories Practice!

Why

Files and Directories Tran Giang Son, [Link]@[Link] 5 / 33

Files Directories Practice!

Why

• RAM is volatile
• Variables are lost after process finishes

• File is persistent
• Data is saved

Files and Directories Tran Giang Son, [Link]@[Link] 5 / 33

Files Directories Practice!

How

1. Open a file
2. Read or write
3. Close the file

Files and Directories Tran Giang Son, [Link]@[Link] 6 / 33

Files Directories Practice!

How: Open a file 1. Open a file

2. Read or write
3. Close the file

• Indicates that the program wants to work with a given file

• What file?
• What operation to work with

• open(fileName, mode)
• fileName: what file
• mode: what operations
• returns a File object representing an opened file

Files and Directories Tran Giang Son, [Link]@[Link] 7 / 33

Files Directories Practice!

How: Open a file 1. Open a file

2. Read or write
f = open("[Link]", "r") 3. Close the file

Mode Meaning

r Reading (default)
w Writing. Creates or clears a file.
x Exclusive creation. Fails if file exists.
a Appending. Creates if file does not exist.
t Opens in text mode. (default)
b Opens in binary mode.
+ Opens a file for updating (rw)

Files and Directories Tran Giang Son, [Link]@[Link] 8 / 33

Files Directories Practice!

How: Read/write 1. Open a file

2. Read or write
3. Close the file

• [Link](size) reads and returns size bytes

• size is optional
• Reads all file content by default
• Updates current file pointer after .read()
• Be careful for large files!

• [Link](offset) sets current file pointer to a specific offset

• [Link]() writes into file

Files and Directories Tran Giang Son, [Link]@[Link] 9 / 33

Files Directories Practice!

How: Read/write

>>> f = open("[Link]", "r+")

>>> [Link](19)
"The language's core"
>>> [Link](0)
>>> [Link]()
"The language's core philosophy is summarized in \
the document The Zen of Python: \n* Beautiful \
is better than ugly. \n* Explicit is better \
than implicit. \n* Simple is better than \
complex. \n* Complex is better than \
complicated. \n* Readability counts. \n"
>>> [Link]("That's all\n")

Files and Directories Tran Giang Son, [Link]@[Link] 10 / 33

Files Directories Practice!

How: Read/write
• Text files
• .readline(): reads until a new line.
• There’s a \n at the end of file

• .readlines(): reads all lines

>>> f = open("[Link]", "r+")

>>> [Link]()
"The language's core philosophy is summarized in the documen
>>> [Link]()
['* Beautiful is better than ugly.\n',
'* Explicit is better than implicit.\n',
'* Simple is better than complex.\n',
'* Complex is better than complicated.\n',
'* Readability counts.\n', "That's all\n"]

Files and Directories Tran Giang Son, [Link]@[Link] 11 / 33

Files Directories Practice!

How: Buffering
• Buffer: in-memory cache of file content
• Speeding up IO accesses1
• Reading/writing blocks is faster than individual bytes

No buffering vs Single buffering

1
Even stdout. . .
Files and Directories Tran Giang Son, [Link]@[Link] 12 / 33
Files Directories Practice!

How: Buffering

• open(fileName, mode, buffering = -1)

• buffering is optional
• -1, same as io.DEFAULT_BUFFER_SIZE
• 0: disable buffering
• 1: line buffering for text files
• >1: fixed size buffer

• Flushing buffer: write buffer to disk, if any

• Manually [Link]()

Files and Directories Tran Giang Son, [Link]@[Link] 13 / 33

Files Directories Practice!

How: Close a file 1. Open a file

2. Read or write
3. Close the file

• Close a file after using

• Clean up OS caches, buffers
• Without closing, there may be data loss with power outage
[Link]()

Files and Directories Tran Giang Son, [Link]@[Link] 14 / 33

Files Directories Practice!

How: Close a file

• Automatically .close() using with

with open('[Link]', 'r+') as f:
data = [Link]()

# other stuffs here, f is closed.

Files and Directories Tran Giang Son, [Link]@[Link] 15 / 33

Files Directories Practice!

How: Extras

• Exceptions
• Temporary files
• Compression
• Objects

Files and Directories Tran Giang Son, [Link]@[Link] 16 / 33

Files Directories Practice!

How: Extras 1. Exceptions

2. Temporary files
3. Compression
• Exceptions? Remind.
4. Objects

Files and Directories Tran Giang Son, [Link]@[Link] 17 / 33

Files Directories Practice!

How: Extras 1. Exceptions

2. Temporary files
3. Compression
• Exceptions? Remind.
4. Objects
• Exception:
• Errors at runtime
• Python: try... except...

• For handling IO errors:

filename = input("Enter file name: ")
try:
f = open(filename, "r")
except IOError:
print(f"Error missing file {filename}")

Files and Directories Tran Giang Son, [Link]@[Link] 17 / 33

Files Directories Practice!

How: Extras 1. Exceptions

2. Temporary files
• Temporary files?
3. Compression
4. Objects

Files and Directories Tran Giang Son, [Link]@[Link] 18 / 33

Files Directories Practice!

How: Extras 1. Exceptions

2. Temporary files
• Temporary files?
3. Compression
• Don’t care about name, location 4. Objects
• Just somewhere to store temp contents
• Automatically cleaned up after close()

• Module tempfile
import [Link]

# gimme a file, whenever it is

f = [Link]('w+t')
[Link]("3.1415926...")
[Link]()

# closed means deleted

Files and Directories Tran Giang Son, [Link]@[Link] 18 / 33
Files Directories Practice!

How: Extras 1. Exceptions

2. Temporary files
• Compression? 3. Compression
4. Objects
• What:

Files and Directories Tran Giang Son, [Link]@[Link] 19 / 33

Files Directories Practice!

How: Extras 1. Exceptions

2. Temporary files
• Compression? 3. Compression
4. Objects
• What: Use less storage to represent data

Files and Directories Tran Giang Son, [Link]@[Link] 19 / 33

Files Directories Practice!

How: Extras 1. Exceptions

2. Temporary files
• Compression? 3. Compression
4. Objects
• What: Use less storage to represent data
• Why:

Files and Directories Tran Giang Son, [Link]@[Link] 19 / 33

Files Directories Practice!

How: Extras 1. Exceptions

2. Temporary files
• Compression? 3. Compression
4. Objects
• What: Use less storage to represent data
• Why:
• Smaller disk storage
• Easier for transmission over network
• Encryption with passwords

Files and Directories Tran Giang Son, [Link]@[Link] 19 / 33

Files Directories Practice!

How: Extras 1. Exceptions

2. Temporary files
• Compression? 3. Compression
4. Objects
• What: Use less storage to represent data
• Why:
• Smaller disk storage
• Easier for transmission over network
• Encryption with passwords

• Plenty of existing modules

• zlib, gzip, bz2, lzma, tarfile, zipfile

• Each module would have different advantages/disadvantages

and usage.

Files and Directories Tran Giang Son, [Link]@[Link] 19 / 33

Files Directories Practice!

How: Extras 1. Exceptions

2. Temporary files
3. Compression
Module Compression In-memory Extension4. Objects
Files Directory

zlib Yes Yes No No No

gzip Yes Yes .gz No No
bz2 Yes Yes .bz2 No No
lzma Yes Yes .xz No No
tarfile No No .tar Yes Yes
zipfile Yes Yes .zip Yes Yes

Files and Directories Tran Giang Son, [Link]@[Link] 20 / 33

Files Directories Practice!

How: Extras 1. Exceptions

2. Temporary files
3. Compression
4. Objects

• Serialize objects into byte array

• Save state to disk, optionally compressed (!)
• Load state later
• Transmit object to a remote machine

Files and Directories Tran Giang Son, [Link]@[Link] 21 / 33

Files Directories Practice!

How: Extras 1. Exceptions

2. Temporary files
3. Compression
4. Objects

• pickle module
• [Link](obj, f): save object obj into
already-opened-for-binary-write file f
• obj = [Link](f): load object from
already-opened-for-binary-read file f

Files and Directories Tran Giang Son, [Link]@[Link] 22 / 33

Files Directories Practice!

How: Extras 1. Exceptions

2. Temporary files
• What can be pickled? 3. Compression
4. Objects
• None, True, and False
• Integers, floating point numbers, complex numbers
• Strings, bytes, bytearrays
• Tuples, lists, sets, and dictionaries containing only picklable
objects
• Can also pickled behaviors:
• Functions defined at the top level of a module
• Built-in functions defined at the top level of a module
• Classes that are defined at the top level of a module

Files and Directories Tran Giang Son, [Link]@[Link] 23 / 33

Files Directories Practice!

How: Extras 1. Exceptions

2. Temporary files
3. Compression
• pickle vs json 4. Objects

Feature pickle json

Compatibility Python-only Open

Format Binary Text
Readability Nah Yay
Data types Many Limited

Files and Directories Tran Giang Son, [Link]@[Link] 24 / 33

Files Directories Practice!

Directories

Files and Directories Tran Giang Son, [Link]@[Link] 25 / 33

Files Directories Practice!

What

Files and Directories Tran Giang Son, [Link]@[Link] 26 / 33

Files Directories Practice!

What

• Hierachical structure
• A bunch of files
• A bunch of sub-directories

• Looks like a tree

• Path indicates a location inside a directory

Files and Directories Tran Giang Son, [Link]@[Link] 26 / 33

Files Directories Practice!

Why

• For organization of data

• Easier traversing and browsing

Files and Directories Tran Giang Son, [Link]@[Link] 27 / 33

Files Directories Practice!

How

• Listing: [Link](), [Link]() (recursive)

• Creating: [Link](), [Link]()
• Deleting: [Link](), [Link]() (recursive)

Files and Directories Tran Giang Son, [Link]@[Link] 28 / 33

Files Directories Practice!

How

>>> import os
>>> e=[Link](".")
>>> [f for f in e]
[<DirEntry '0. [Link]'>, \
<DirEntry '1. course [Link]'>, \
<DirEntry '2. [Link]'>, \
<DirEntry '3. [Link]'>, \
<DirEntry '4. [Link]'>, \
<DirEntry '5. [Link]'>]
>>> [Link]("figs/intro")

Files and Directories Tran Giang Son, [Link]@[Link] 29 / 33

Files Directories Practice!

Practice!

Files and Directories Tran Giang Son, [Link]@[Link] 30 / 33

Files Directories Practice!

Practical work 5: persistent info

• Copy your pw4 directory into pw5 directory

• Update your input functions
• Write student info to [Link] after finishing input
• Write course info to [Link] after finishing input
• Write marks to [Link] after finishing input

Files and Directories Tran Giang Son, [Link]@[Link] 31 / 33

Files Directories Practice!

Practical work 5: persistent info

• Before closing your program

• Select a compression method
• Compress all files aboves into [Link]

• Upon starting your program,

• Check if [Link] exists
• If yes, decompress and load data from it

• Push your work to corresponding forked Github repository

Files and Directories Tran Giang Son, [Link]@[Link] 32 / 33

Files Directories Practice!

Practical work 6: pickled management system

• Copy your pw5 directory into pw6 directory

• Upgrade the persistence feature of your system to use
pickle instead, still with compression
• Push your work to corresponding forked Github repository

Files and Directories Tran Giang Son, [Link]@[Link] 33 / 33

Review Processes Practice!

Multi Processing

Tran Giang Son, [Link]@[Link]

ICT Department, USTH

Multi Processing Tran Giang Son, [Link]@[Link] 1 / 39

Review Processes Practice!

Review

Multi Processing Tran Giang Son, [Link]@[Link] 2 / 39

Review Processes Practice!

Review

• Process
• Scheduling
• IO Redirection

Multi Processing Tran Giang Son, [Link]@[Link] 3 / 39

Review Processes Practice!

Process

• What is process?
• Process vs program?

Multi Processing Tran Giang Son, [Link]@[Link] 4 / 39

Review Processes Practice!

Process
• Process is a program in execution state

Multi Processing Tran Giang Son, [Link]@[Link] 5 / 39

Review Processes Practice!

Process
• Process is a program in execution state (active)

Multi Processing Tran Giang Son, [Link]@[Link] 5 / 39

Review Processes Practice!

Process
• Process is a program in execution state (active)
• Why process?
• Program is passive
• No execution → what’s running?

Multi Processing Tran Giang Son, [Link]@[Link] 5 / 39

Review Processes Practice!

Process
• Process is a program in execution state (active)
• Why process?
• Program is passive
• No execution → what’s running?

• A process execution state contains

Multi Processing Tran Giang Son, [Link]@[Link] 5 / 39

Review Processes Practice!

Process
• Process is a program in execution state (active)
• Why process?
• Program is passive
• No execution → what’s running?

• A process execution state contains

• Processor state (context)

Multi Processing Tran Giang Son, [Link]@[Link] 5 / 39

Review Processes Practice!

Process
• Process is a program in execution state (active)
• Why process?
• Program is passive
• No execution → what’s running?

• A process execution state contains

• Processor state (context)
• File descriptors

Multi Processing Tran Giang Son, [Link]@[Link] 5 / 39

Review Processes Practice!

Process
• Process is a program in execution state (active)
• Why process?
• Program is passive
• No execution → what’s running?

• A process execution state contains

• Processor state (context)
• File descriptors
• Memory allocation

Multi Processing Tran Giang Son, [Link]@[Link] 5 / 39

Review Processes Practice!

Process
• Process is a program in execution state (active)
• Why process?
• Program is passive
• No execution → what’s running?

• A process execution state contains

• Processor state (context)
• File descriptors
• Memory allocation
• Process stack

Multi Processing Tran Giang Son, [Link]@[Link] 5 / 39

Review Processes Practice!

Process
• Process is a program in execution state (active)
• Why process?
• Program is passive
• No execution → what’s running?

• A process execution state contains

• Processor state (context)
• File descriptors
• Memory allocation
• Process stack
• Data section

Multi Processing Tran Giang Son, [Link]@[Link] 5 / 39

Review Processes Practice!

Process
• Process is a program in execution state (active)
• Why process?
• Program is passive
• No execution → what’s running?

• A process execution state contains

• Processor state (context)
• File descriptors
• Memory allocation
• Process stack
• Data section
• Heap

Multi Processing Tran Giang Son, [Link]@[Link] 5 / 39

Review Processes Practice!

Process States
waiting

event finish event wait

interrupt

admitted exit
new ready running terminated

scheduled

Multi Processing Tran Giang Son, [Link]@[Link] 6 / 39

Review Processes Practice!

Process States
waiting

event finish event wait

interrupt

admitted exit
new ready running terminated

scheduled

• new: process has just been created

• ready: waiting to be assigned (scheduled) to a processor
• running: it’s executing instructions
• waiting: waiting for some events to occur
• terminated: finished execution
Multi Processing Tran Giang Son, [Link]@[Link] 6 / 39
Review Processes Practice!

Process Creation

• Start a new process == Create a new process

• Create new child process
• Can create child process → grand child process

• Dependent on OS, parent and child can share

• All resources: opened files, devices, etc. . .
• Some resources: opened files only
• No resource

• A fully loaded system will have a process tree

Multi Processing Tran Giang Son, [Link]@[Link] 7 / 39

Review Processes Practice!

Process Creation
$ pstree -A
init-+-acpid
|-cron
|-daemon---mpt-statusd---sleep
|-dbus-daemon
|-dovecot-+-anvil
| |-config
| `-log
|-master-+-pickup
| |-qmgr
| `-tlsmgr
|-mysqld_safe---mysqld---23*[{mysqld}]
|-php5-fpm---2*[php5-fpm]
|-proftpd
|-screen---bash---python2---{python2}
|-sshd-+-sshd---sshd---bash---pstree
| `-sshd---sshd
|-udevd---2*[udevd]
`-znc---{znc}
Multi Processing Tran Giang Son, [Link]@[Link] 8 / 39
Review Processes Practice!

Process Creation on Windows

BOOL WINAPI CreateProcess(

_In_opt_ LPCTSTR lpApplicationName,
_Inout_opt_ LPTSTR lpCommandLine,
_In_opt_ LPSECURITY_ATTRIBUTES lpProcessAttributes,
_In_opt_ LPSECURITY_ATTRIBUTES lpThreadAttributes,
_In_ BOOL bInheritHandles,
_In_ DWORD dwCreationFlags,
_In_opt_ LPVOID lpEnvironment,
_In_opt_ LPCTSTR lpCurrentDirectory,
_In_ LPSTARTUPINFO lpStartupInfo,
_Out_ LPPROCESS_INFORMATION lpProcessInformation
);
Source: MSDN

Multi Processing Tran Giang Son, [Link]@[Link] 9 / 39

Review Processes Practice!

Process Creation on Windows

• A simplified WinAPI function:

UINT WINAPI WinExec(
_In_ LPCSTR lpCmdLine,
_In_ UINT uCmdShow
);

Multi Processing Tran Giang Son, [Link]@[Link] 10 / 39

Review Processes Practice!

Process Creation on Windows

• A simplified WinAPI function:

UINT WINAPI WinExec(
_In_ LPCSTR lpCmdLine,
_In_ UINT uCmdShow
);
• It’s deprecated.
Source: MSDN

Multi Processing Tran Giang Son, [Link]@[Link] 10 / 39

Review Processes Practice!

Process Creation on UNIX/Linux

• New processes are not created from scratch

• Two steps
• fork()
• exec()

Multi Processing Tran Giang Son, [Link]@[Link] 11 / 39

Review Processes Practice!

Process Creation on UNIX/Linux

• New processes are not created from scratch

• Two steps
• fork()
• exec()
parent
wait()

fork()

exec() exit()
child

Multi Processing Tran Giang Son, [Link]@[Link] 11 / 39

Review Processes Practice!

Process Creation on UNIX/Linux

• fork()
• Perfectly «clone» current process to a new process

Multi Processing Tran Giang Son, [Link]@[Link] 12 / 39

Review Processes Practice!

Process Creation on UNIX/Linux

• fork()
• Perfectly «clone» current process to a new process
• Open files
• Register states
• Memory allocations
• Except process id

• Who’s who?
• Parent?
• Child?

Multi Processing Tran Giang Son, [Link]@[Link] 12 / 39

Review Processes Practice!

Process Creation on UNIX/Linux

• fork()
• Perfectly «clone» current process to a new process
• Open files
• Register states
• Memory allocations
• Except process id

• Who’s who?
• Parent?
• Child?

pid_t fork(void);

Multi Processing Tran Giang Son, [Link]@[Link] 12 / 39

Review Processes Practice!

Process Creation on UNIX/Linux

• Parent: fork() returns process id of child
• Child: fork() returns 0
• Example
#include <unistd.h>
#include <stdio.h>
int main() {
printf("Main before fork()\n");
int pid = fork();
if (pid == 0) printf("I am child after fork()\n");
else printf("I am parent after fork(), child is %d\n", pid);
return 0;
}

$ ./dofork
Main before fork()
I am parent after fork(), child is 2378
I am child after fork()
Multi Processing Tran Giang Son, [Link]@[Link] 13 / 39
Review Processes Practice!

Process Creation on UNIX/Linux

• exec()
• Load an executable binary to replace current process image
• A family of functions.
• Ask man

int execl(...);
int execle(...);
int execlp(...);
int execv(...);
int execvp(const char *file, char *const argv[]);
int execvP(...);

Multi Processing Tran Giang Son, [Link]@[Link] 14 / 39

Review Processes Practice!

Process Creation on UNIX/Linux

• exec() example

#include <stdio.h>
#include <unistd.h>
int main() {
printf("Going to launch ps -ef\n");
char *args[]= { "/bin/ps", "-ef" , NULL};
execvp("/bin/ps", args);
return 0;
}

Multi Processing Tran Giang Son, [Link]@[Link] 15 / 39

Review Processes Practice!

Scheduling

• Multiple processes running at the same time

• Process scheduler is a part that decides which processes to
be executed at a certain time.

Multi Processing Tran Giang Son, [Link]@[Link] 16 / 39

Review Processes Practice!

Scheduling

• Maximize CPU usage

• Responsiveness for User interface
• Provide computational power for heavy-workload processes
• «Multitasking»
• Different characteristics of processes
• CPU bound: spends more time on computation
• I/O bound: spends more time on I/O devices
(reading/writing disk, printing. . . )

Multi Processing Tran Giang Son, [Link]@[Link] 17 / 39

Review Processes Practice!

Scheduling

• By the ability to pause running processes

• Preemption: OS forcely pauses running processes
• Non-preemption (also cooperation): processes willing to
pause itself

Multi Processing Tran Giang Son, [Link]@[Link] 18 / 39

Review Processes Practice!

Scheduling

• By the ability to pause running processes

• Preemption: OS forcely pauses running processes
• Non-preemption (also cooperation): processes willing to
pause itself
• By duration between each «switch»
• Short term scheduler: milliseconds (fast, responsive)
• Long term scheduler: seconds/minutes (batch jobs)

Multi Processing Tran Giang Son, [Link]@[Link] 18 / 39

Review Processes Practice!

Scheduling with Context Switch

Process A Operating System Process B

Executing

Interrupt save state into PCB0

reload state from PCB1

Executing

save state into PCB1

Interrupt
reload state from PCB0

Executing
Multi Processing Tran Giang Son, [Link]@[Link] 19 / 39
Review Processes Practice!

Scheduling with Context Switch

• Switch between processes

• Save data of old process
• Load previously saved data of new process

• Context switch is overhead

• No work done for processes during context switch
• Time slice (time between each switch) is hardware-limited

Multi Processing Tran Giang Son, [Link]@[Link] 20 / 39

Review Processes Practice!

Scheduling with Context Switch

Operating System Process A Process B Process C Process D

Multi Processing Tran Giang Son, [Link]@[Link] 21 / 39

Review Processes Practice!

Scheduler

• Knowns
• List of processes
• Process states
• Accounting information

Multi Processing Tran Giang Son, [Link]@[Link] 22 / 39

Review Processes Practice!

Scheduler

• Knowns
• List of processes
• Process states
• Accounting information

• Constraints
• Process priority (if any)
• Processes have scheduling priority
• Indicates the importance of each process
• Higher priority: more likely to be scheduled

Multi Processing Tran Giang Son, [Link]@[Link] 22 / 39

Review Processes Practice!

Scheduler

• Problems
• P1: What processes to run next?

Multi Processing Tran Giang Son, [Link]@[Link] 23 / 39

Review Processes Practice!

Scheduler

• Problems
• P1: What processes to run next?
• P2: How long should it run?

Multi Processing Tran Giang Son, [Link]@[Link] 23 / 39

Review Processes Practice!

Scheduler

• Problem 1: What processes to run next?

• Job queue - set of all processes entering the system, stored
on disk
• Ready queue - set of all processes residing in main
memory, ready and waiting to execute
• Device queues - set of processes waiting for an I/O device
• Lists of PCBs
• Processes change state → they migrate among the various
queues

Multi Processing Tran Giang Son, [Link]@[Link] 24 / 39

Review Processes Practice!

Scheduler

• Problem 2: How long should it run?

• First In First Served
• Earliest Deadline First
• Shortest Remaining Time
• Round Robin
• ...

Multi Processing Tran Giang Son, [Link]@[Link] 25 / 39

Review Processes Practice!

Scheduler

Algorithm Preempt? Priority? Note

First Come, First Served No No Depends on arrival time

Shortest-Job-First No Yes Low waiting time ω
Shortest-Remaining-Time-First Yes Yes Preemptive SJF, low ω
Round Robin Yes No Low response time ρ
Multilevel Queue Depends Depends Several subqueues, permanent
Multilevel Feedback Queue Depends Depends Several subqueues, migrate

Multi Processing Tran Giang Son, [Link]@[Link] 26 / 39

Review Processes Practice!

IO Redirection

esc
F1 F2 F3 F4 F5 F6 F7 F8 F9 F10 F11 F12

~
`
!
1
@
2
#
3
$
4
%
5
^
6
&
7
*
8
(
9
)
0
_
-
+
= delete
stdin (0) Process stdout (1)
{ } |
Q W E R T Y U I O P [ ] \
tab

: enter
“
caps lock
A S D F G H J K L ; ‘ return

< > ?
Z X C V B N M , . /
shift shift

alt ⌘ ⌘ alt

fn control option command command option

stderr (2)

Default: input from keyboard and output to terminal

Multi Processing Tran Giang Son, [Link]@[Link] 27 / 39

Review Processes Practice!

IO Redirection

esc
F1 F2 F3 F4 F5 F6 F7 F8 F9 F10 F11 F12

~
`
!
1
@
2
#
3
$
4
%
5
^
6
&
7
*
8
(
9
)
0
_
-
+
= delete
stdin (0) Process stdout (1) file
{ } |
Q W E R T Y U I O P [ ] \
tab

: enter
“
caps lock
A S D F G H J K L ; ‘ return

< > ?
Z X C V B N M , . /
shift shift

alt ⌘ ⌘ alt

fn control option command command option

stderr (2)

Input from keyboard and output to file

Multi Processing Tran Giang Son, [Link]@[Link] 28 / 39

Review Processes Practice!

IO Redirection

file stdin (0) Process stdout (1)

stderr (2)

Input from file and output to terminal

Multi Processing Tran Giang Son, [Link]@[Link] 29 / 39

Review Processes Practice!

IO Redirection

file stdin (0) Process stdout (1) file

stderr (2)

Input from file and output to another file

Multi Processing Tran Giang Son, [Link]@[Link] 30 / 39

Review Processes Practice!

IO Redirection

file stdin (0) Process 1 stdout (1) stdin (0) Process 2 stdout (1) file

stderr (2) stderr (2)

Input from file, pipe output of Process 1 to Process 2, output to

another file

Multi Processing Tran Giang Son, [Link]@[Link] 31 / 39

Review Processes Practice!

Processes

Multi Processing Tran Giang Son, [Link]@[Link] 32 / 39

Review Processes Practice!

Modules

• os
• subprocess

Multi Processing Tran Giang Son, [Link]@[Link] 33 / 39

Review Processes Practice!

Task

• Create a process
• Run and wait for finish
• Run in background
• Run with timeout

• IO redirection
• Redirect input
• Redirect output
• Redirect with pipe

• Terminate
• Get return code

Multi Processing Tran Giang Son, [Link]@[Link] 34 / 39

Review Processes Practice!

os module
• os module is deprecated in Python 3
• This is for references only.

Task How

Run and wait [Link]("ps aux")

Run in background [Link]("long_command.sh &")
Timeout N/A
Redirect input [Link]("bc", "w").write("1+2")
Redirect output print([Link]("ps aux", "r").readlines())
Redirect with pipe [Link](), [Link]()
Terminate [Link](pid, [Link])
Get return code return value of [Link]()

Multi Processing Tran Giang Son, [Link]@[Link] 35 / 39

Review Processes Practice!

subprocess module

Task How

Run and wait [Link](["ps", "aux"])

Run in background [Link]("long_command.sh")

Timeout [Link]("long_command.sh", timeout = 10)

Redirect input [Link]("bc", stdin=[Link]).communicate(b"3+4\n")

Redirect output [Link](["ps", "aux"], stdout=[Link]).communicate()

Redirect with pipe [Link]("bc", stdin=[Link])

Terminate [Link](), [Link]()

Get return code subprocess.check_output(), catch CalledProcessError

Multi Processing Tran Giang Son, [Link]@[Link] 36 / 39

Review Processes Practice!

Practice!

Multi Processing Tran Giang Son, [Link]@[Link] 37 / 39

Review Processes Practice!

Practical work 7: Python shell

• Create a new python program, name it «[Link]»

• Make a shell
• User inputs command
• Shell executes the command, print output
• Support IO redirection
• input from file to process
• output from process to file
• e.g. input from one process being output of another

Multi Processing Tran Giang Son, [Link]@[Link] 38 / 39

Review Processes Practice!

Practical work 7: Python shell

• Run it and test some commands

• ls -la
• ls -la > [Link]
• bc < [Link]
• ps aux | grep term

• Push your work to corresponding forked Github repository

Multi Processing Tran Giang Son, [Link]@[Link] 39 / 39

Review Multithreading Practice!

Multithreading

Tran Giang Son, [Link]@[Link]

ICT Department, USTH

Multithreading Tran Giang Son, [Link]@[Link] 1 / 38

Review Multithreading Practice!

Review

Multithreading Tran Giang Son, [Link]@[Link] 2 / 38

Review Multithreading Practice!

Remind PCB

• Process Control Block

Multithreading Tran Giang Son, [Link]@[Link] 3 / 38

Review Multithreading Practice!

Remind PCB

• Process Control Block

• Contains
• Process ID
• Process state (new/ready/running/waiting/terminated)
• Processor state (program counter, registers)
• File descriptors
• Scheduling information (next section)
• Accounting information (limits)

Multithreading Tran Giang Son, [Link]@[Link] 3 / 38

Review Multithreading Practice!

Thread & Single-threaded process

code data file descs

stack registers heap

• Thread
• a single flow of execution
• belongs to a process
• can be considered as lightweight
process thread

• Single-threaded process
• Default
• Only one thread per process

Multithreading Tran Giang Son, [Link]@[Link] 4 / 38

Review Multithreading Practice!

Single-threaded process

max
stack

free memory
• Single stack
• Single text section (code)
• Single data section (global data) heap
• Single heap (dynamic allocation)
data

text
0

Multithreading Tran Giang Son, [Link]@[Link] 5 / 38

Review Multithreading Practice!

Multi-threaded process

• More than one thread per process

• Share the same PCB among threads
• Process state
• Memory allocation (heap, global data)
• File descriptors (files, sockets, etc.)
• Scheduling information
• Accounting information

• Different processor state (program counter, registers)

• Different stack

Multithreading Tran Giang Son, [Link]@[Link] 6 / 38

Review Multithreading Practice!

Multi-threaded process
code data file descs

heap

stack stack stack

registers registers registers

thread thread thread

Multithreading Tran Giang Son, [Link]@[Link] 7 / 38

Review Multithreading Practice!

Multi-threaded process

• Each thread has: max

• stack (thread 1) SP1
Private stack
• Private stack pointer stack (thread 2) SP2
• Private program counter stack (thread 3) SP3
• Private register values free memory
• Private scheduling policies
• Share: heap

• Common text section (code)

data
• Common data section (global PC1
data) text PC2
• Common heap (dynamic 0 PC3
allocation)
• File descriptors (opened files) Process memory space
• Signals. . .

Multithreading Tran Giang Son, [Link]@[Link] 8 / 38

Review Multithreading Practice!

Multi-threaded process vs Multi process

• Same goals

Multithreading Tran Giang Son, [Link]@[Link] 9 / 38

Review Multithreading Practice!

Multi-threaded process vs Multi process

• Same goals
• Do several things at the same time

Multithreading Tran Giang Son, [Link]@[Link] 9 / 38

Review Multithreading Practice!

Multi-threaded process vs Multi process

• Same goals
• Do several things at the same time
• Increase CPU utilization

Multithreading Tran Giang Son, [Link]@[Link] 9 / 38

Review Multithreading Practice!

Multi-threaded process vs Multi process

• Same goals
• Do several things at the same time
• Increase CPU utilization
• Increase responsiveness

Multithreading Tran Giang Son, [Link]@[Link] 9 / 38

Review Multithreading Practice!

Multi-threaded process vs Multi process

• Same goals
• Do several things at the same time
• Increase CPU utilization
• Increase responsiveness

• What is the principal difference between these two types of

process?

Multithreading Tran Giang Son, [Link]@[Link] 9 / 38

Review Multithreading Practice!

Multi-threaded process vs Multi process

• Same goals
• Do several things at the same time
• Increase CPU utilization
• Increase responsiveness

• What is the principal difference between these two types of

process?
• Multi-process with fork(): «resource cloning»

Multithreading Tran Giang Son, [Link]@[Link] 9 / 38

Review Multithreading Practice!

Multi-threaded process vs Multi process

• Same goals
• Do several things at the same time
• Increase CPU utilization
• Increase responsiveness

• What is the principal difference between these two types of

process?
• Multi-process with fork(): «resource cloning»
• Multi-thread process: «resource sharing»

Multithreading Tran Giang Son, [Link]@[Link] 9 / 38

Review Multithreading Practice!

Why?

• Responsiveness
• Performance
• Resource Sharing
• Scalability

Multithreading Tran Giang Son, [Link]@[Link] 10 / 38

Review Multithreading Practice!

Responsiveness

• Perform different tasks at the same time

Multithreading Tran Giang Son, [Link]@[Link] 11 / 38

Review Multithreading Practice!

Responsiveness

• Perform different tasks at the same time

• Several operations can block (e.g. network, disk I/O)

Multithreading Tran Giang Son, [Link]@[Link] 11 / 38

Review Multithreading Practice!

Responsiveness

• Perform different tasks at the same time

• Several operations can block (e.g. network, disk I/O)
• UI needs responsiveness

Multithreading Tran Giang Son, [Link]@[Link] 11 / 38

Review Multithreading Practice!

Responsiveness

• Perform different tasks at the same time

• Several operations can block (e.g. network, disk I/O)
• UI needs responsiveness

→ one thread for UI, other threads for background tasks

Multithreading Tran Giang Son, [Link]@[Link] 11 / 38

Review Multithreading Practice!

Performance

• Creating (fork()) a new process is slower than a thread

• Terminating a process is also slower than a thread
• Switching between processes is slower than between threads

Multithreading Tran Giang Son, [Link]@[Link] 12 / 38

Review Multithreading Practice!

Resource Sharing

• Memory is always shared

• Heap
• Global data

• All file descriptors are also shared

• Open files
• TCP sockets
• UNIX sockets
• Devices

• No need to use shm*()

Multithreading Tran Giang Son, [Link]@[Link] 13 / 38

Review Multithreading Practice!

Scalability

• More CPU cores: simply increase number of threads

• Don’t create too many threads
• Overhead
• Synchronization

Multithreading Tran Giang Son, [Link]@[Link] 14 / 38

Review Multithreading Practice!

Why NOT multi-thread?

• Threads are evil

• Nondeterministic
• Synchronization
• Deadlocks

• Complication

Multithreading Tran Giang Son, [Link]@[Link] 15 / 38

Review Multithreading Practice!

Multi-process real world app

Apache HTTPD Prefork Model1

1
Image courtesy of Toni Miu’s blog
Multithreading Tran Giang Son, [Link]@[Link] 16 / 38
Review Multithreading Practice!

Multi-thread, multi-process, real world app

Apache HTTPD Worker Model2

2
Image courtesy of Toni Miu’s blog
Multithreading Tran Giang Son, [Link]@[Link] 17 / 38
Review Multithreading Practice!

Multi-thread, multi-process, real world app

Multithreading Tran Giang Son, [Link]@[Link] 18 / 38

Review Multithreading Practice!

Multithreading

Multithreading Tran Giang Son, [Link]@[Link] 19 / 38

Review Multithreading Practice!

Python threading

• Global Interpreter Lock

• Implemented in CPython
• Mutex
• Only 1 thread can control the Python intepreter
• Only one thread can be executed at any given time
• Bottleneck in Python CPU-bound code
• Not a problem in wrapper-to-native-code3
• Not a problem in IO-bound programs

3
e.g. numpy uses native libraries, so no GIL problem
Multithreading Tran Giang Son, [Link]@[Link] 20 / 38
Review Multithreading Practice!

Python threading

• Why GIL?
• Memory management
• Reference counting
• Garbage collector

• Simplification of thread-safety
• Only 1 mutex on the intepreter
• No multiple mutexes on each object
• No deadlock

• That’s not a bug

Multithreading Tran Giang Son, [Link]@[Link] 21 / 38

Review Multithreading Practice!

Python threading

• Why GIL?
• Memory management
• Reference counting
• Garbage collector

• Simplification of thread-safety
• Only 1 mutex on the intepreter
• No multiple mutexes on each object
• No deadlock

• That’s not a bug but a feature

Multithreading Tran Giang Son, [Link]@[Link] 21 / 38

Review Multithreading Practice!

Python threading

• Removing GIL?
• Slower single-threaded performance
• 1 mutex per object reference. . .
• Potential deadlocks

• Less compatbility

Multithreading Tran Giang Son, [Link]@[Link] 22 / 38

Review Multithreading Practice!

How?

• 2 «How» questions:

Multithreading Tran Giang Son, [Link]@[Link] 23 / 38

Review Multithreading Practice!

How?

• 2 «How» questions:
• Q1: How does thread achieve concurrency?

Multithreading Tran Giang Son, [Link]@[Link] 23 / 38

Review Multithreading Practice!

How?

• 2 «How» questions:
• Q1: How does thread achieve concurrency?
• Q2: How to use thread?

Multithreading Tran Giang Son, [Link]@[Link] 23 / 38

Review Multithreading Practice!

How (Q1): Concurrency on Single Core

• Q1: How does thread achieve concurrency?

single core T1 T2 T3 T4 T5 T1 T2 T3 …

time

Multithreading Tran Giang Son, [Link]@[Link] 24 / 38

Review Multithreading Practice!

How (Q1): Concurrency on Multi Cores

• Q1: How does thread achieve concurrency?

core 0 T1 T5 T4 T3 T2 T1 T5 T4 …

core 1 T2 T1 T5 T4 T3 T2 T1 T5 …

core 2 T3 T2 T1 T5 T4 T3 T2 T1 …

core 3 T4 T3 T2 T1 T5 T4 T3 T2 …

time

Multithreading Tran Giang Son, [Link]@[Link] 25 / 38

Review Multithreading Practice!

How (Q2): Using thread

• Use the module

• Subclass Thread
• Create new instance
• Launch the new thread
• [optional] Wait for thread to finish

Multithreading Tran Giang Son, [Link]@[Link] 26 / 38

Review Multithreading Practice!

How (Q2): Using thread 1. Use the module

2. Subclass Thread
3. Create new instance
4. Launch the new thread
5. Wait for thread to finish

• The threading module

import threading

Multithreading Tran Giang Son, [Link]@[Link] 27 / 38

Review Multithreading Practice!

How (Q2): Using thread 1. Use the module

2. Subclass Thread
3. Create new instance
4. Launch the new thread
5. Wait for thread to finish

• Define a subclass of [Link]

• Override run() method to run in background
• [optional] Implement __init__() method for passing
parameters

Multithreading Tran Giang Son, [Link]@[Link] 28 / 38

Review Multithreading Practice!

How (Q2): Using thread 1. Use the module

2. Subclass Thread
3. Create new instance
4. Launch the new thread
5. Wait for thread to finish
class BackgroundThread([Link]):
def __init__(self, sleepTime):
[Link].__init__(self)
self.__sleepTime = sleepTime

def run(self):
[Link](self.__sleepTime)
print(f"Finished sleeping {self.__sleepTime}s")

Multithreading Tran Giang Son, [Link]@[Link] 29 / 38

Review Multithreading Practice!

How (Q2): Using thread 1. Use the module

2. Subclass Thread
3. Create new instance
4. Launch the new thread
5. Wait for thread to finish

• Create new instance of the thread class

backgroundThread = BackgroundThread(10)

Multithreading Tran Giang Son, [Link]@[Link] 30 / 38

Review Multithreading Practice!

How (Q2): Using thread 1. Use the module

2. Subclass Thread
3. Create new instance
4. Launch the new thread
5. Wait for thread to finish

• Launch the new thread [Link]()

• NOT .run()

[Link]() # note no args here

Multithreading Tran Giang Son, [Link]@[Link] 31 / 38

Review Multithreading Practice!

How (Q2): Using thread 1. Use the module

2. Subclass Thread
3. Create new instance
4. Launch the new thread
5. Wait for thread to finish

• [optional] Wait for thread to finish with .join()

[Link]()

Multithreading Tran Giang Son, [Link]@[Link] 32 / 38

Review Multithreading Practice!

How (Q2): Using thread

import threading
import time

class BackgroundThread([Link]):
def __init__(self, sleepTime):
[Link].__init__(self)
self.__sleepTime = sleepTime
def run(self):
[Link](self.__sleepTime)
print(f"Finished sleeping {self.__sleepTime}s")

backgroundThread = BackgroundThread(10)
[Link]() # note no args here
[Link]()
print("Finished main thread")
Multithreading Tran Giang Son, [Link]@[Link] 33 / 38
Review Multithreading Practice!

How: Extras

• Simple threading without subclassing:

def threadFunction(sleepTime):
[Link](sleepTime)
print(f"Finished sleeping {sleepTime}s")

t = [Link](target=threadFunction, args=(10,))
[Link]()

Multithreading Tran Giang Son, [Link]@[Link] 34 / 38

Review Multithreading Practice!

How: Extras

• Synchronization between threads [Link] (also

called mutex)
• .acquire()
• .release()
• Automatic .acquire() and .release() using with
statement
• Be careful with race conditions while using Lock

Multithreading Tran Giang Son, [Link]@[Link] 35 / 38

Review Multithreading Practice!

How: Extras

lock = [Link]()
[Link]()
# do something dangerous here
[Link]()

with lock:
# do something dangerous here
print("Dangerous function")

# lock is released automatically

Multithreading Tran Giang Son, [Link]@[Link] 36 / 38

Review Multithreading Practice!

Practice!

Multithreading Tran Giang Son, [Link]@[Link] 37 / 38

Review Multithreading Practice!

Practical work 8: multithreaded management system

• Copy your pw6 directory into pw8 directory

• Upgrade the persistence feature of your system to use
pickle in background thread, still with compression
• Push your work to corresponding forked Github repository

Multithreading Tran Giang Son, [Link]@[Link] 38 / 38

GUI Python GUI Toolkit Practice!

GUI Toolkit

Tran Giang Son, [Link]@[Link]

ICT Department, USTH

GUI Toolkit Tran Giang Son, [Link]@[Link] 1 / 28

GUI Python GUI Toolkit Practice!

GUI

GUI Toolkit Tran Giang Son, [Link]@[Link] 2 / 28

GUI Python GUI Toolkit Practice!

What

• GUI
• Graphical User Interface
• Interactive with graphical components
• Windows, scrollbars, buttons, textboxes,

• Sexy (?)

• CLI
• Command Line Interface
• Writing commands in terminal
• Wait for response from system
• Old school, boring (?)

GUI Toolkit Tran Giang Son, [Link]@[Link] 3 / 28

GUI Python GUI Toolkit Practice!

CLI vs GUI

Feature CLI GUI

Learning curve Steep Easy

Flexibility Very high Limited
Memory Low Higher
Speed Fast Slower
Interact Keyboard Keyboard, Mouse
Theming Limited Easy

GUI Toolkit Tran Giang Son, [Link]@[Link] 4 / 28

GUI Python GUI Toolkit Practice!

Why

• User friendly, intuitive for new comers

• Easy to learn, no need to remember comments
• Better multitasking

GUI Toolkit Tran Giang Son, [Link]@[Link] 5 / 28

GUI Python GUI Toolkit Practice!

Python GUI Toolkit

GUI Toolkit Tran Giang Son, [Link]@[Link] 6 / 28

GUI Python GUI Toolkit Practice!

Toolkits

Feature Tkinter PyQt Kivy wxPython

Included? Yes No No No
Cross platform Yes Yes Yes Yes
Backend Tcl/Tk Qt OpenGL wxWidgets

GUI Toolkit Tran Giang Son, [Link]@[Link] 7 / 28

GUI Python GUI Toolkit Practice!

Tkinter

• Simplicity
• TODO: image of student
• Flexibility
management system here
• Focusing on new comers

GUI Toolkit Tran Giang Son, [Link]@[Link] 8 / 28

GUI Python GUI Toolkit Practice!

Tkinter

• Window
• Widgets
• Layout
• Window event loop

GUI Toolkit Tran Giang Son, [Link]@[Link] 9 / 28

GUI Python GUI Toolkit Practice!

Tkinter: Window 1. Window

2. Widgets
• Types: Main window and sub window 3. Layout
4. Window event loop
• Important attributes: title, size
• Main window
import tkinter as tk
window = [Link]()
[Link]("Student Information System")
[Link]("800x600")
• Sub window
sub = [Link](window)
[Link]("Students")
[Link]("600x400")

GUI Toolkit Tran Giang Son, [Link]@[Link] 10 / 28

GUI Python GUI Toolkit Practice!

Tkinter: Widgets

• Everything is widget
• Frame
• Label
• Buttons
• Entry
• Check Button
• Radio Button
• List Box
• ComboBox
• Menu
• ...
• Important attributes
• Dimension: width = 400, height =
300
• Background color: bg = "green"

GUI Toolkit Tran Giang Son, [Link]@[Link] 11 / 28

GUI Python GUI Toolkit Practice!

Tkinter: Widgets 1. Window

2. Widgets
3. Layout
4. Window event loop
• Frame 5. Message box

• A container for other widgets

• [Link](window, width = 100, height = 100)

• Label
• Show texts
• [Link](window, text = "This is a Label")

GUI Toolkit Tran Giang Son, [Link]@[Link] 12 / 28

GUI Python GUI Toolkit Practice!

Tkinter: Widgets 1. Window

2. Widgets
3. Layout
• Button 4. Window event loop
5. Message box
• Clickable
• Handle click: command = onClickFunc

from tkinter import messagebox

def onClick():
[Link](message="Button 1 clicked")

[Link](window, text = "Button 1", command = onClick)

GUI Toolkit Tran Giang Son, [Link]@[Link] 13 / 28

GUI Python GUI Toolkit Practice!

Tkinter: Widgets 1. Window

2. Widgets
3. Layout
• Entry
4. Window event loop
• Input texts 5. Message box
• Can be used for password field (with show = "*")

entry = [Link](window)
[Link](-1, "Entry for text input")
• Checkbutton
• Checkboxes
• 2 states: check and uncheck
• [Link](window, text = "Checkbutton option
1")

GUI Toolkit Tran Giang Son, [Link]@[Link] 14 / 28

GUI Python GUI Toolkit Practice!

Tkinter: Widgets 1. Window

2. Widgets
3. Layout
4. Window event loop
• Radiobutton 5. Message box
• Single choice among options
• Text != Value

radioValue = [Link](value = "op1")

[Link](window, variable = radioValue,
text = "Radiobutton option 1", value = "op1")
[Link](window, variable = radioValue,
text = "Radiobutton option 2", value = "op2")

GUI Toolkit Tran Giang Son, [Link]@[Link] 15 / 28

GUI Python GUI Toolkit Practice!

Tkinter: Widgets 1. Window

2. Widgets
3. Layout
4. Window event loop
• Listbox 5. Message box
• A list of items
• Selectable item
• Get selected item: [Link]([Link])

icts = ["ICT", "I See Tea", "Icy Tea", "Ice City"]

listbox = [Link](window)
for i in icts:
[Link]([Link](i), i)

GUI Toolkit Tran Giang Son, [Link]@[Link] 16 / 28

GUI Python GUI Toolkit Practice!

Tkinter: Widgets 1. Window

2. Widgets
3. Layout
4. Window event loop
• Combobox 5. Message box
• A list of selectable items
• Initially collapsed, can be expanded
• Get selected item: [Link]()

from tkinter import ttk

icts = ["ICT", "I See Tea", "Icy Tea", "Ice City"]
[Link](window, values = icts)

GUI Toolkit Tran Giang Son, [Link]@[Link] 17 / 28

GUI Python GUI Toolkit Practice!

Tkinter: Layout 1. Window

2. Widgets
3. Layout
4. Window event loop
• Geometry manager 5. Message box
• Handle placements (positions) of widgets on windows
• Main container: [Link]
• Widget methods for geometry management
• .pack()
• .place()
• .grid()

GUI Toolkit Tran Giang Son, [Link]@[Link] 18 / 28

GUI Python GUI Toolkit Practice!

Tkinter: Layout 1. Window

2. Widgets
3. Layout
• .pack() 4. Window event loop
5. Message box
• Packing algorithm
• Similar to HTML div
• Default
• Vertically align
• Horizontally centered

• Alignment direction: side = [Link]

• Automatic expand: fill = tk.X, fill = tk.Y

GUI Toolkit Tran Giang Son, [Link]@[Link] 19 / 28

GUI Python GUI Toolkit Practice!

Tkinter: Layout 1. Window

# default window 2. Widgets
[Link](window, width = 100, ..., bg="red").pack() 3. Layout
[Link](window, width = 50, ..., bg="green").pack()
[Link](window, width = 25, ..., bg="blue").pack() 4. Window event loop
5. Message box
# secondary window with fill
[Link](sub, width = 100, ..., bg="red").pack(fill = tk.X)
[Link](sub, width = 50, ..., bg="green").pack(fill = tk.X)
[Link](sub, width = 25, ..., bg="blue").pack(fill = tk.X)

GUI Toolkit Tran Giang Son, [Link]@[Link] 20 / 28

GUI Python GUI Toolkit Practice!

Tkinter: Layout 1. Window

2. Widgets
3. Layout
• .place() 4. Window event loop
• Similar to HTML position: absolute 5. Message box
• Unit: pixels
• Absolute values
• Position: x = 10, y = 10
• Dimension: width = 400, height = 300

• Relative values [0...1]

• Position: relx = 0.1, rely = 0.1
• Dimension: relwidth = 0.5, relheight = 0.7

GUI Toolkit Tran Giang Son, [Link]@[Link] 21 / 28

GUI Python GUI Toolkit Practice!

Tkinter: Layout 1. Window

[Link](window, bg="red").place( 2. Widgets
x = 10, y = 10, width = 100, height = 100) 3. Layout
[Link](window, bg="green").place(
x = 20, y = 30, width = 50, height = 50) 4. Window event loop
[Link](window, bg="blue").place( 5. Message box
x = 150, y = 100, width = 25, height = 25)

Output:

GUI Toolkit Tran Giang Son, [Link]@[Link] 22 / 28

GUI Python GUI Toolkit Practice!

Tkinter: Layout 1. Window

2. Widgets
3. Layout
• .grid() 4. Window event loop
5. Message box
• Similar to HTML table
• Position: column = 0, row = 2
• Stretching: sticky = [Link]
• Padding: padx = 3, pady = 3
• Spanning
• columnspan = 3
• rowspan = 2

GUI Toolkit Tran Giang Son, [Link]@[Link] 23 / 28

GUI Python GUI Toolkit Practice!

Tkinter: Layout 1. Window

2. Widgets
3. Layout
4. Window event loop
[Link](window, text = "Username").grid(
5. Message box
column = 0, row = 0, sticky = [Link], padx = 3, pady = 3)
[Link](window, text = "Password").grid(
column = 0, row = 1, sticky = [Link], padx = 3, pady = 3)

[Link](window).grid(
column = 1, row = 0, sticky = [Link], padx = 3, pady = 3, columnspan = 4)
[Link](window).grid(
column = 1, row = 1, sticky = [Link], padx = 3, pady = 3, columnspan = 4)

[Link](window, text = "Login").grid(

column = 0, row = 2, columnspan = 2)
[Link](window, text = "Exit").grid(
column = 2, row = 2, columnspan = 2)

GUI Toolkit Tran Giang Son, [Link]@[Link] 24 / 28

GUI Python GUI Toolkit Practice!

Tkinter: Layout 1. Window

2. Widgets
3. Layout
4. Window event loop
5. Message box

GUI Toolkit Tran Giang Son, [Link]@[Link] 25 / 28

GUI Python GUI Toolkit Practice!

Tkinter: Window loop

• A blocking method
• Handles input, output events
• [Link]()

GUI Toolkit Tran Giang Son, [Link]@[Link] 26 / 28

GUI Python GUI Toolkit Practice!

Practice!

GUI Toolkit Tran Giang Son, [Link]@[Link] 27 / 28

GUI Python GUI Toolkit Practice!

Practical work 9: GUI’ed management system

• Copy your pw8 directory to pw9 directory

• Upgrade your user interface to GUI using Tkinter
• Push your work to corresponding forked Github repository

GUI Toolkit Tran Giang Son, [Link]@[Link] 28 / 28

Python For Data Science Cheat Sheet Excel Spreadsheets Pickled Files
>>> file = '[Link]' >>> import pickle
Importing Data >>> data = [Link](file) >>> with open('pickled_fruit.pkl', 'rb') as file:
pickled_data = [Link](file)
>>> df_sheet2 = [Link]('1960-1966',
Learn Python for data science Interactively at [Link] skiprows=[0],
names=['Country',
'AAM: War(2002)'])
>>> df_sheet1 = [Link](0, HDF5 Files
parse_cols=[0],
Importing Data in Python skiprows=[0], >>> import h5py
>>> filename = 'H-H1_LOSC_4_v1-815411200-4096.hdf5'
names=['Country'])
Most of the time, you’ll use either NumPy or pandas to import >>> data = [Link](filename, 'r')
your data: To access the sheet names, use the sheet_names attribute:
>>> import numpy as np >>> data.sheet_names
>>> import pandas as pd Matlab Files
Help SAS Files >>> import [Link]
>>> filename = '[Link]'
>>> from sas7bdat import SAS7BDAT >>> mat = [Link](filename)
>>> [Link]([Link])
>>> help(pd.read_csv) >>> with SAS7BDAT('urbanpop.sas7bdat') as file:
df_sas = file.to_data_frame()

Text Files Exploring Dictionaries

Stata Files Accessing Elements with Functions
Plain Text Files >>> data = pd.read_stata('[Link]') >>> print([Link]()) Print dictionary keys
>>> filename = 'huck_finn.txt' >>> for key in [Link](): Print dictionary keys
>>> file = open(filename, mode='r') Open the file for reading print(key)
>>> text = [Link]() Read a file’s contents Relational Databases meta
quality
>>> print([Link]) Check whether file is closed
>>> from sqlalchemy import create_engine strain
>>> [Link]() Close file
>>> print(text) >>> engine = create_engine('sqlite://[Link]') >>> pickled_data.values() Return dictionary values
>>> print([Link]()) Returns items in list format of (key, value)
Use the table_names() method to fetch a list of table names: tuple pairs
Using the context manager with
>>> with open('huck_finn.txt', 'r') as file:
>>> table_names = engine.table_names() Accessing Data Items with Keys
print([Link]()) Read a single line
print([Link]()) Querying Relational Databases >>> for key in data ['meta'].keys() Explore the HDF5 structure
print([Link]()) print(key)
>>> con = [Link]() Description
>>> rs = [Link]("SELECT * FROM Orders") DescriptionURL
Table Data: Flat Files >>> df = [Link]([Link]()) Detector
>>> [Link] = [Link]() Duration
GPSstart
Importing Flat Files with numpy >>> [Link]()
Observatory
Files with one data type Using the context manager with Type
UTCstart
>>> filename = ‘[Link]’ >>> with [Link]() as con:
>>> print(data['meta']['Description'].value) Retrieve the value for a key
>>> data = [Link](filename, rs = [Link]("SELECT OrderID FROM Orders")
delimiter=',', String used to separate values df = [Link]([Link](size=5))
[Link] = [Link]()
skiprows=2,
usecols=[0,2],
Skip the first 2 lines
Read the 1st and 3rd column
Navigating Your FileSystem
dtype=str) The type of the resulting array Querying relational databases with pandas
Magic Commands
Files with mixed data types >>> df = pd.read_sql_query("SELECT * FROM Orders", engine)
>>> filename = '[Link]' !ls List directory contents of files and directories
>>> data = [Link](filename, %cd .. Change current working directory
%pwd Return the current working directory path
delimiter=',',
names=True, Look for column header
Exploring Your Data
dtype=None)
NumPy Arrays os Library
>>> data_array = [Link](filename) >>> data_array.dtype Data type of array elements >>> import os
>>> data_array.shape Array dimensions >>> path = "/usr/tmp"
The default dtype of the [Link]() function is None. >>> wd = [Link]() Store the name of current directory in a string
>>> len(data_array) Length of array
>>> [Link](wd) Output contents of the directory in a list
Importing Flat Files with pandas >>> [Link](path) Change current working directory
pandas DataFrames >>> [Link]("[Link]", Rename a file
>>> filename = '[Link]' "[Link]")
>>> data = pd.read_csv(filename, >>> [Link]() Return first DataFrame rows
nrows=5, >>> [Link]("[Link]") Delete an existing file
Number of rows of file to read >>> [Link]() Return last DataFrame rows >>> [Link]("newdir") Create a new directory
header=None, Row number to use as col names >>> [Link] Describe index
sep='\t', Delimiter to use >>> [Link] Describe DataFrame columns
comment='#', Character to split comments >>> [Link]() Info on DataFrame
na_values=[""]) String to recognize as NA/NaN >>> data_array = [Link] Convert a DataFrame to an a NumPy array DataCamp
Learn R for Data Science Interactively
Python For Data Science Cheat Sheet Plot Anatomy & Workflow
Plot Anatomy Workflow
Matplotlib Axes/Subplot The basic steps to creating plots with matplotlib are:
Learn Python Interactively at [Link] 1 Prepare data 2 Create plot 3 Plot 4 Customize plot 5 Save plot 6 Show plot
>>> import [Link] as plt
>>> x = [1,2,3,4] Step 1
>>> y = [10,20,25,30]
>>> fig = [Link]() Step 2
Matplotlib Y-axis Figure >>> ax = fig.add_subplot(111) Step 3
[Link](x, y, color='lightblue', linewidth=3) Step 3, 4
Matplotlib is a Python 2D plotting library which produces >>>
>>> [Link]([2,4,6],
publication-quality figures in a variety of hardcopy formats [5,15,25],
color='darkgreen',
and interactive environments across marker='^')
platforms. >>> ax.set_xlim(1, 6.5)
X-axis
>>> [Link]('[Link]')

1 Prepare The Data Also see Lists & NumPy

>>> [Link]() Step 6

1D Data 4 Customize Plot

>>>
>>>
import numpy as np
x = [Link](0, 10, 100)
Colors, Color Bars & Color Maps Mathtext
>>> y = [Link](x) >>> [Link](x, x, x, x**2, x, x**3) >>> [Link](r'$sigma_i=15$', fontsize=20)
>>> z = [Link](x) >>> [Link](x, y, alpha = 0.4)
>>> [Link](x, y, c='k') Limits, Legends & Layouts
2D Data or Images >>> [Link](im, orientation='horizontal')
>>> im = [Link](img, Limits & Autoscaling
>>> data = 2 * [Link]((10, 10))
>>> data2 = 3 * [Link]((10, 10))
cmap='seismic') >>> [Link](x=0.0,y=0.1) Add padding to a plot
>>> Y, X = [Link][-3:3:100j, -3:3:100j] >>> [Link]('equal') Set the aspect ratio of the plot to 1
Markers >>> [Link](xlim=[0,10.5],ylim=[-1.5,1.5]) Set limits for x-and y-axis
>>> U = -1 - X**2 + Y
>>> V = 1 + X - Y**2 >>> fig, ax = [Link]() >>> ax.set_xlim(0,10.5) Set limits for x-axis
>>> from [Link] import get_sample_data >>> [Link](x,y,marker=".") Legends
>>> img = [Link](get_sample_data('axes_grid/bivariate_normal.npy')) >>> [Link](x,y,marker="o") >>> [Link](title='An Example Axes', Set a title and x-and y-axis labels
ylabel='Y-Axis',
Linestyles xlabel='X-Axis')
2 Create Plot >>> [Link](x,y,linewidth=4.0)
>>> [Link](loc='best')
Ticks
No overlapping plot elements

>>> import [Link] as plt >>> [Link](x,y,ls='solid') >>> [Link](ticks=range(1,5), Manually set x-ticks
>>> [Link](x,y,ls='--') ticklabels=[3,100,-12,"foo"])
Figure >>>
>>>
[Link](x,y,'--',x**2,y**2,'-.')
[Link](lines,color='r',linewidth=4.0)
>>> ax.tick_params(axis='y', Make y-ticks longer and go in and out
direction='inout',
>>> fig = [Link]() length=10)
>>> fig2 = [Link](figsize=[Link](2.0)) Text & Annotations
Subplot Spacing
Axes >>> [Link](1, >>> fig3.subplots_adjust(wspace=0.5, Adjust the spacing between subplots
-2.1, hspace=0.3,
All plotting is done with respect to an Axes. In most cases, a 'Example Graph', left=0.125,
subplot will fit your needs. A subplot is an axes on a grid system.
style='italic') right=0.9,
>>> [Link]("Sine", top=0.9,
>>> fig.add_axes() xy=(8, 0), bottom=0.1)
>>> ax1 = fig.add_subplot(221) # row-col-num xycoords='data', >>> fig.tight_layout() Fit subplot(s) in to the figure area
xytext=(10.5, 0),
>>> ax3 = fig.add_subplot(212) textcoords='data', Axis Spines
>>> fig3, axes = [Link](nrows=2,ncols=2) arrowprops=dict(arrowstyle="->", >>> [Link]['top'].set_visible(False) Make the top axis line for a plot invisible
>>> fig4, axes2 = [Link](ncols=3) connectionstyle="arc3"),) >>> [Link]['bottom'].set_position(('outward',10)) Move the bottom axis line outward

3 Plotting Routines 5 Save Plot

Save figures
1D Data Vector Fields >>> [Link]('[Link]')
>>> lines = [Link](x,y) Draw points with lines or markers connecting them >>> axes[0,1].arrow(0,0,0.5,0.5) Add an arrow to the axes Save transparent figures
>>> [Link](x,y) Draw unconnected points, scaled or colored >>> axes[1,1].quiver(y,z) Plot a 2D field of arrows >>> [Link]('[Link]', transparent=True)
>>> axes[0,0].bar([1,2,3],[3,4,5]) Plot vertical rectangles (constant width) >>> axes[0,1].streamplot(X,Y,U,V) Plot 2D vector fields
>>> axes[1,0].barh([0.5,1,2.5],[0,1,2])
6
Plot horiontal rectangles (constant height)
>>> axes[1,1].axhline(0.45) Draw a horizontal line across axes Data Distributions Show Plot
>>> axes[0,1].axvline(0.65) Draw a vertical line across axes >>> [Link](y) Plot a histogram
>>> [Link](x,y,color='blue') Draw filled polygons >>> [Link](y) Make a box and whisker plot >>> [Link]()
>>> ax.fill_between(x,y,color='yellow') Fill between y-values and 0 >>> [Link](z) Make a violin plot
2D Data or Images Close & Clear
>>> fig, ax = [Link]() >>> [Link]() Clear an axis
>>> axes2[0].pcolor(data2) Pseudocolor plot of 2D array Clear the entire figure
>>> im = [Link](img, Colormapped or RGB arrays >>> axes2[0].pcolormesh(data) Pseudocolor plot of 2D array
>>> [Link]()
cmap='gist_earth', >>> [Link]() Close a window
interpolation='nearest', >>> CS = [Link](Y,X,U) Plot contours
vmin=-2, >>> axes2[2].contourf(data1) Plot filled contours
vmax=2) >>> axes2[2]= [Link](CS) Label a contour plot DataCamp
Learn Python for Data Science Interactively
 LEARN DATA SCIENCE ONLINE
Start Learning For Free - [Link]

Data Science Cheat Sheet

Python - Intermediate

KEY BASICS, PRINTING AND GETTING HELP

This cheat sheet assumes you are familiar with the content of our Python Basics Cheat Sheet

s - A Python string variable l - A Python list variable

i - A Python integer variable d - A Python dictionary variable
f - A Python float variable

L I STS len(my_set) - Returns the number of objects in now - wks4 - Return a datetime object
[Link](3) - Returns the fourth item from l and my_set (or, the number of unique values from l) representing the time 4 weeks prior to now
deletes it from the list a in my_set - Returns True if the value a exists in newyear_2020 = [Link](year=2020,
[Link](x) - Removes the first item in l that is my_set month=12, day=31) - Assign a datetime
equal to x object representing December 25, 2020 to
[Link]() - Reverses the order of the items in l REGULAR EXPRESSIONS newyear_2020
l[1::2] - Returns every second item from l, import re - Import the Regular Expressions module newyear_2020.strftime("%A, %b %d, %Y")
commencing from the 1st item [Link]("abc",s) - Returns a match object if - Returns "Thursday, Dec 31, 2020"
l[-5:] - Returns the last 5 items from l specific axis the regex "abc" is found in s, otherwise None [Link]('Dec 31, 2020',"%b
[Link]("abc","xyz",s) - Returns a string where %d, %Y") - Return a datetime object
ST R I N G S all instances matching regex "abc" are replaced representing December 31, 2020
[Link]() - Returns a lowercase version of s by "xyz"
[Link]() - Returns s with the first letter of every RANDOM
word capitalized L I ST C O M P R E H E N S I O N import random - Import the random module
"23".zfill(4) - Returns "0023" by left-filling the A one-line expression of a for loop [Link]() - Returns a random float
string with 0’s to make it’s length 4. [i ** 2 for i in range(10)] - Returns a list of between 0.0 and 1.0
[Link]() - Returns a list by splitting the the squares of values from 0 to 9 [Link](0,10) - Returns a random
string on any newline characters. [[Link]() for s in l_strings] - Returns the integer between 0 and 10
Python strings share some common methods with lists list l_strings, with each item having had the [Link](l) - Returns a random item from
s[:5] - Returns the first 5 characters of s .lower() method applied the list l
"fri" + "end" - Returns "friend" [i for i in l_floats if i < 0.5] - Returns
"end" in s - Returns True if the substring "end" the items from l_floats that are less than 0.5 COUNTER
is found in s from collections import Counter - Import the
F U N C T I O N S F O R LO O P I N G Counter class
RANGE for i, value in enumerate(l): c = Counter(l) - Assign a Counter (dict-like)
Range objects are useful for creating sequences of print("The value of item {} is {}". object with the counts of each unique item from
integers for looping. format(i,value)) l, to c
range(5) - Returns a sequence from 0 to 4 - Iterate over the list l, printing the index location c.most_common(3) - Return the 3 most common
range(2000,2018) - Returns a sequence from 2000 of each item and its value items from l
to 2017 for one, two in zip(l_one,l_two):
range(0,11,2) - Returns a sequence from 0 to 10, print("one: {}, two: {}".format(one,two)) T RY/ E XC E P T
with each item incrementing by 2 - Iterate over two lists, l_one and l_two and print Catch and deal with Errors
range(0,-10,-1) - Returns a sequence from 0 to -9 each value l_ints = [1, 2, 3, "", 5] - Assign a list of
list(range(5)) - Returns a list from 0 to 4 while x < 10: integers with one missing value to l_ints
x += 1 l_floats = []
DICTIONARIES - Run the code in the body of the loop until the for i in l_ints:
max(d, key=[Link]) - Return the key that value of x is no longer less than 10 try:
corresponds to the largest value in d l_floats.append(float(i))
min(d, key=[Link]) - Return the key that DAT E T I M E except:
corresponds to the smallest value in d import datetime as dt - Import the datetime l_floats.append(i)
module - Convert each value of l_ints to a float, catching
S E TS now = [Link]() - Assign datetime and handling ValueError: could not convert
my_set = set(l) - Return a set object containing object representing the current time to now string to float: where values are missing.
the unique values from l wks4 = [Link](weeks=4)
- Assign a timedelta object representing a
timespan of 4 weeks to wks4

LEARN DATA SCIENCE ONLINE

Start Learning For Free - [Link]