INTERMEDIATE PYTHON

 Master control structures to customize script and algorithm flow.

 Concludes with a case study to apply learned skills.

 Data Visualization Importance

 Data visualization is crucial for data analysis.

 Helps explore datasets and extract insights.

 Essential for sharing insights with others.

 Example Plot by Hans Rosling

 Introduction to a bubble chart by Hans Rosling.

 Bubbles represent countries; size indicates population.

 Axes: GDP per capita (horizontal) and life expectancy (vertical).

 Plot Details

 Horizontal axis shows GDP per capita in US dollars.

 Source: GapMinder, Wealth and Health of Nations.

 Plot Insights

 Vertical axis shows life expectancy.

 Higher GDP per capita correlates with longer life expectancy.

 Variation in life expectancy among countries with similar income levels.

 Goal: By end of chapter, be able to create this plot.

 Introduction to Matplotlib

 Introduction to pyplot subpackage, imported as plt.

 Example: Line chart showing world population evolution.

 Use [Link] with year and population lists.

 Display plot with [Link].

 Plot Explanation

 :Plot shows years on horizontal axis and population on vertical axis.

 Plot Insights

 Line chart connects four data points.

 World population growth from 2.5 billion (1950) to 7 billion (2010).

 Raises questions about future population growth.

 Introduction to Scatter Plot

 Introduction to scatter plot.

  Modify previous code by using scatter function.

 Scatter Plot Details

 Scatter plot shows individual data points without connecting lines.

 Often a better choice for certain applications.

 Provides a more accurate representation of data based on individual points

HISTOGRAMS
Introduction to Histogram
o The histogram is a type of visualization that's very useful to explore your data.
o It helps to get an idea about the distribution of your variables.
o Imagine 12 values between 0 and 6, placed along a number line.
o To build a histogram, divide the line into equal chunks called bins.
Creating Bins
o Suppose you go for 3 bins, each with a width of 2.
o Count how many data points sit inside each bin.
o First bin: 4 data points.
o Second bin: 6 data points.
o Third bin: 2 data points.
Drawing the Histogram
o Draw a bar for each bin.
o The height of the bar corresponds to the number of data points in that bin.
o Resulting histogram shows the distribution of the 12 values.
o Most values are in the middle, with more values below 2 than above 4.
Using Matplotlib for Histograms
o Start by importing the pyplot package from matplotlib.
o Use the hist function to create the histogram.
o Documentation highlights the important arguments: x (list of values) and bins (number of
bins).
Example with Matplotlib
o Generate a histogram using the 12 values.
o Create a list with the 12 values.
o Call hist and pass the list as x, and specify bins to be 3.
o Call the show function to display the histogram.
 o Histograms provide a bigger picture of data distribution.
Population Pyramid Example
o Example: Population pyramid showing age distribution for males and females in the
European Union.
o Histograms are flipped 90 degrees, with horizontal bins.
o Largest bins for ages 40 to 44 (20 million males and 20 million females).
o These are the baby boomers, data from the year 2010.
o In 2050, distribution is flatter and baby boom generation is older.
o Histograms quickly show how demographics change over time, demonstrating their true
power.

Customization
o Introduction to the importance of customizing plots.
o Highlighting the challenge of making the correct plot to clearly convey the message.

Data visualization

o Discussion of various options for visualizations and customizations.

o Emphasis on choosing the right plot type and customization based on the data and the story.

Basic plot

o Building a simple line plot using existing script.

o Data includes projections until 2100 from the United Nations.
o Initial plot shows population trends but lacks clarity and focus.

Axis labels

o Importance of labeling axes.

o Use of xlabel and ylabel functions to add labels before calling show.
o Result: Annotated axes.
o Quick recap of labeling axes.
o Result: Annotated axes.

Title

o Adding a title to the plot using the title function.

o Example title: 'World Population Projections'.
o Result: Plot with a title.

Ticks

o First input: List of ticks (e.g., 0, 2, 4, etc.).

o Result: Shifted curve making the population clearer.
o Adjusting the y-axis to start from zero using yticks.

Dictionaries, Part 1
 Introduction

 Welcome and introduction to the new Python type: the dictionary.

 Emphasis on the usefulness of dictionaries in data handling.

LIST

 Scenario: Tracking populations for the World Bank.

 Initial approach: Using lists.
o Example: Afghanistan (30.55 million), Albania (2.77 million), Algeria (around 40
million).
 Creating two lists: one for country names and one for populations.

Problem with Lists

 Example task: Finding Albania's population.

o Steps:
1. Use index() method to find the position of Albania in the country list.
2. Use the index to find the corresponding population.
o Code demonstration:

countries = ["Afghanistan", "Albania", "Algeria"]

populations = [30.55, 2.77, 40]

index_albania = [Link]("Albania")

population_albania = populations[index_albania]

o Result: 2.77 (Albania's population in millions).

 Critique: Inconvenient and unintuitive method.

Introducing Dictionaries

 Concept: Directly connecting each country to its population using dictionaries.

 Creating a dictionary:
o Use curly brackets {}.
o Key-value pairs separated by colons :.

Dictionary Example

 Example conversion to dictionary:

world = {

"Afghanistan": 30.55,

"Albania": 2.77,

"Algeria": 40

 Accessing values:
o Syntax: dictionary[key].
 o Example: Finding Albania's population.

population_albania = world["Albania"]

o Result: 2.77 (Albania's population in millions).

Advantages of Dictionaries

 Intuitive: Keys directly connect to values.

 Efficient: Fast key lookup even for large dictionarie

Dictionaries
Dictionaries, Part 2
Recap
 Dictionary Basics:
o A dictionary ("world") is a set of key-value pairs.
o Accessing Values: Example, to access Albania's population: world["Albania"].
o Key Uniqueness: Keys must be unique; duplicate keys overwrite the previous entry.
 Key Requirements:
o Keys must be immutable (e.g., strings, booleans, integers, floats).
o Immutable objects cannot be changed after creation.
o Mutable objects (e.g., lists) cannot be used as keys.
Example: Principality of Sealand
 Context:
o Sealand is an unrecognized micronation in the North Sea with 27 inhabitants.
o Source: Wikipedia.
 Adding Data:
o Syntax: world["sealand"] = 27.
o Verification: sealand in world returns True if "sealand" is a key.
 Updating Values:
o Syntax: world["sealand"] = 28.
 Removing Data:
o Syntax: del world["sealand"].

o

List vs. Dictionary

Feature List Dictionary
Indexing Indexed by a range of numbers Indexed by unique keys
Element Selection Use square brackets to select elements Use square brackets to select elements
Updating
Use square brackets to update elements Use square brackets to update elements
Elements
Removing Use square brackets to remove
Use square brackets to remove elements
Elements elements
Data Structure Sequence of values Key-value pairs
When order matters, and subsets are When a lookup table with unique keys is
Use Case
needed needed

When to Use Which:

 Lists: Use when order matters and for easy subset selection.
 Dictionaries: Use for fast data lookup with unique keys.

Pandas, Part 1

Introduction

As a data scientist, you'll often work with large datasets. These datasets are often in a tabular format,
resembling a spreadsheet.

Examples of Tabular Datasets

Example 1: Chemical Plant Data

Each row is a measurement (observation). Variables include temperature, date, time, and location.
Example 2: BRICS Countries Data
Each row represents a country (observation). Variables include country name, capital, area in
millions of square kilometers, and population in millions.

Working with Datasets in Python

 Rectangular Data Structure: For tabular data, you might consider using a 2D NumPy array.
However, this isn't ideal for datasets with different data types.
 Data Types: In the BRICS example, area and population are floats, while country and capital
are strings.

Introducing Pandas

 Pandas: A high-level data manipulation tool built on NumPy, developed by Wes McKinney.
 DataFrame: In Pandas, tabular data is stored in an object called a DataFrame.

DataFrame Structure

 Rows: Represent observations.

 Columns: Represent variables.
 Row Labels: Each row has a unique label (e.g., BR for Brazil, RU for Russia).
 Column Labels: Columns have labels such as country, population, etc.
 Data Types: Columns can contain different data types.

Creating a DataFrame
 1. From a Dictionary:
o Use curly brackets to create key-value pairs.
o Keys are column labels, values are columns in list form.
o Example:

python

Copy code

import pandas as pd

data = {

'country': ['Brazil', 'Russia', 'India', 'China', 'South Africa'],

'capital': ['Brasilia', 'Moscow', 'New Delhi', 'Beijing', 'Pretoria'],

'area': [8.516, 17.10, 3.286, 9.597, 1.221],

'population': [200.4, 143.5, 1252, 1357, 52.98]

brics = [Link](data)

[Link] = ['BR', 'RU', 'IN', 'CH', 'SA']

o This creates a DataFrame similar to the BRICS table.

2. From a CSV File:
o Suppose the data is in a CSV file named [Link].
o Use Pandas read_csv function to import the data.
o Example:

pythony code

brics = pd.read_csv('[Link]', index_col=0)

o This reads the CSV file and sets the first column as the row indexes.

Customizing Data Import

 read_csv Function: Offers many arguments to customize data import.

 Documentation: Check the documentation for detailed options and customization.

By using Pandas, data scientists can efficiently handle and manipulate large, heterogeneous datasets
in Python.

Pandas ,part 2
1. Creating the DataFrame brics:
o Assume we have a DataFrame called brics with appropriate row and column labels.
2. Index and Select Data:
o Square Brackets: Using square brackets for basic indexing and selection.
o Advanced Methods: Introduction to loc and iloc for more powerful indexing.
 3. Column Access using Square Brackets:
o Selecting a single column returns a Pandas Series.
o Using double square brackets to keep the data in DataFrame format.
o Selecting multiple columns by providing a list of column labels inside another set of
square brackets.
4. Row Access using Square Brackets:
o Selecting rows by specifying a slice (e.g., 1:4).
5. Advanced Row Access using loc:
o loc is label-based indexing.
o Selecting a single row returns a Series, but double brackets return a DataFrame.
o Selecting multiple rows by providing a list of labels.
o Selecting both rows and columns by providing row labels and column labels separated
by a comma.
6. Advanced Row Access using iloc:
o iloc is position-based indexing.
o Selecting a row by its position index.
o Selecting multiple rows by providing a list of position indexes.
o Selecting both rows and columns by providing position indexes for both rows and
columns.

Let's write some code to illustrate these concepts using a sample DataFrame:

python

Copy code

import pandas as pd

# Sample DataFrame creation

brics = [Link]({

'country': ['Brazil', 'Russia', 'India', 'China', 'South Africa'],

'capital': ['Brasília', 'Moscow', 'New Delhi', 'Beijing', 'Pretoria'],

'area': [8.516, 17.10, 3.286, 9.597, 1.221],

'population': [200.4, 143.5, 1252, 1357, 52.98]

})

[Link] = ['BR', 'RU', 'IN', 'CH', 'SA']

# Column Access

print(brics['country']) # Returns a Series

print(type(brics['country']))
print(brics[['country']]) # Returns a DataFrame

print(type(brics[['country']]))

print(brics[['country', 'capital']]) # Selecting multiple columns

# Row Access

print(brics[1:4]) # Selecting rows by slicing

# Advanced Row Access using loc

print([Link]['RU']) # Returns a Series

print([Link][['RU']]) # Returns a DataFrame

print([Link][['RU', 'IN', 'CH']]) # Selecting multiple rows

print([Link][['RU', 'IN', 'CH'], ['country', 'capital']]) # Selecting rows and columns

# Advanced Row Access using iloc

print([Link][1]) # Returns a Series

print([Link][[1]]) # Returns a DataFrame

print([Link][[1, 2, 3]]) # Selecting multiple rows

print([Link][[1, 2, 3], [0, 1]]) # Selecting rows and columns

print([Link][:, [0, 1]]) # Keeping all rows, selecting specific columns

Comparsion Operators
NumPy Recap

Among those types was bool, short for boolean. Do you remember the bmi array from the intro
course? Here it is again. Using the greater than sign (>), we could find out which values in bmi were
above 23. Next, I used the resulting Boolean array to actually select those values. In this video, we'll
dive a little deeper into the world of comparison operators, like this greater than sign. Comparison
operators are operators that can tell how two Python values relate, and result in a boolean.

. Numeric Comparisons

In the simplest sense, you can use these operators on numbers. Say, for example, that you want to
check if 2 is smaller than 3. You type 2 < 3, and hit Enter. Because this is the case, you get True. You
can also check if two values are equal, with a double equals sign (==). From this call, we see that 2 ==
3 gives us False. Makes sense, because 2 is not equal to 3. You can also make a combination of equality
and smaller than. Have a look at this command that checks if 2 <= 3. It's True, but also 3 <= 3 is True.
Of course, you can also use comparison operators directly on variables that represent these integers,

All these operators also work for strings. Let's check if "carl" < "chris". According to the alphabet,
"carl" comes before "chris", so the result is True. Do you think that comparing a string and an
integer can work? Let's try to see if the integer 3 < "chris". We get an error. Typically, Python can't
tell how two objects with different types relate. Different numeric types, such as floats and integers,
are exceptions as this example shows: no error this time. In general, always make sure that you make
comparisons

Another exception arises when we move back to the example we started with, where we compared the
NumPy array, bmi, with an integer, 23. This works perfectly. NumPy figures out that you want to
compare every element in bmi with 23, and returns corresponding booleans. Behind the scenes,
NumPy builds a numpy array of the same size filled with the number 23, and then performs an
element-wise comparison. This is concise yet very efficient code, something data scientists love!

Comparator Summary

Have a look at this table that summarizes all comparison operators. You already know about some of
these. They're all pretty straightforward, except for the last one maybe. The exclamation mark
followed by an equals sign (!=) stands for inequality. It's basically the opposite of equality.

Operato Description Example (NumPy) Output

r
== Equal to [Link]([1, 2, 3]) == [Link]([1, 2, [ True, True, False]
4])
!= Not equal to [Link]([1, 2, 3]) != [Link]([1, 2, 4]) [False, False, True]
< Less than [Link]([1, 2, 3]) < [Link]([1, 3, 2]) [ True, True, False]
<= Less than or equal [Link]([1, 2, 3]) <= [Link]([1, 2, [ True, True, False]
2])
> Greater than [Link]([1, 2, 3]) > [Link]([1, 1, 2]) [False, True, True]
>= Greater or equal [Link]([1, 2, 3]) >= [Link]([1, 2, [ True, True, True]
2])

EXAMPLE

import numpy as np
# Define two arrays

a = [Link]([1, 2, 3])

b = [Link]([1, 2, 4])

# Equal to

print(a == b) # Output: [ True True False]

# Not equal to

print(a != b) # Output: [False False True]

# Less than

print(a < b) # Output: [False False True]

# Less than or equal to

print(a <= b) # Output: [ True True True]

# Greater than

print(a > b) # Output: [False False False]

# Greater than or equal to

print(a >= b) # Output: [ True True False]

Boolean Operators
You can use boolean operators for this. The three most common ones are and, or, and not.

and

The and operator works just as you would expect. It takes two booleans and returns True only if both
booleans themselves are True. This means that True and True evaluates to True, but False and True,
True and False, and False and False all evaluate to False. Instead of using booleans directly, we can
use the results of comparisons. Suppose we have a variable x equal to 12. To check if this variable is
greater than 5 but less than 15, we can use x > 5 and x < 15. As you already learned, the first part will
evaluate to True, and the second part will also evaluate to True. So the result of this expression, True
and True, is True. This makes sense, because 12 lies between 5 and 15.

or

The or operator works similarly, but the difference is that only one of the booleans it uses should be
True. This means that True or True equals True, but also False or True and True or False evaluate to
True. Only False or False results in False. Again, you can make combinations with variables. For
example, checking if a variable y which is equal to 5, is less than 7 or above 13. 5 < 7 is True, 5 > 13 is
False. The or operation thus returns True.
not

Finally, there's the not operator. It simply negates the boolean value you use it on. not True is False,
not False is True. The not operation is typically useful if you're combining different boolean
operations and then want to negate that result; you'll see some examples in the exercises.

NumPy

Now, for NumPy arrays, things are different. Retaking the bmi example from the intro course, we can
try to find out which BMIs are higher than 21 but lower than 22. The output of bmi > 21 is easily
found, and so is the one for bmi < 22. Let's now try to combine those with the and operator I just
introduced. Oops, an error. The truth value of an array with more than one element is ambiguous.
and clearly doesn't like an array of booleans to work on.

After some digging in the NumPy documentation, you can find the functions logical_and, logical_or,
and logical_not, the "array equivalents" of and, or, and not. To find out which BMIs are between 21
and 22, we thus need this call. Again, as we expect from NumPy, the and operation is performed
element-wise: True and True give True, like these ones, but False and True or True and False give
False, like for these elements. To actually select only these BMIs from the bmi array, we can use the
resulting array of booleans in square brackets. Again, NumPy wins when it comes to writing short yet
very expressive Python code. I can hear you asking, "Cool, but how does this work for Pandas
DataFrames, the de facto standard for dataset manipulation?" That's something you'll find out later
in this chapter.

NumPy Boolean Operators

Here’s a summary table for boolean operators in NumPy:

Function Description Example

logical_and Element-wise logical AND np.logical_and(bmi > 21, bmi < 22)
logical_or Element-wise logical OR np.logical_or(bmi < 21, bmi > 22)
logical_not Element-wise logical NOT np.logical_not(bmi > 21)

import numpy as np

# Creating a NumPy array

bmi = [Link]([20.1, 21.5, 22.3, 23.1, 24.0])

# Using logical_and to find BMIs between 21 and 22

result = np.logical_and(bmi > 21, bmi < 22)

print(result) # [False True False False False]

# Using logical_or to find BMIs less than 21 or greater than 22

result = np.logical_or(bmi < 21, bmi > 22)

print(result) # [ True False True True True]

# Using logical_not to negate a condition

result = np.logical_not(bmi > 22)

print(result) # [ True True False False False]

# Selecting elements based on condition

selected_bmi = bmi[np.logical_and(bmi > 21, bmi < 22)]

print(selected_bmi) # [21.5]

IF ,ELIF ELSE STATEMENTS

such as less than and greater than, and you also know how to combine the boolean results, using
boolean operators such as and and or. Things get really interesting when you can actually use these
concepts to change how your program behaves. Depending on the outcome of your comparisons, you
might want your Python code to behave differently. You can do this with conditional statements in
Python: if, else, and elif.

if

Let's start working in a script, [Link]. Suppose you have a variable z, equal to 4. If the
value is even, you want to print out: "z is even". This code does the trick. The modulo operator
% 2 will return 0 if z is even. If you run this, Python checks if the condition holds. It's true, so
the corresponding code is executed: "z is even" gets printed out. Let's compare this to the
general recipe for an if statement. It reads as follows: if condition, execute expression. Notice
the colon at the end, and the fact that you simply have to indent the Python code with four
spaces (or a tab) to tell Python what to do in the case the condition succeeds.

To exit the if statement, simply continue with some Python code without indentation, and Python will
know that it's not part of the if statement.

It's perfectly possible to have more lines inside the if statement, like this for example. The script now
prints out two lines if you run it. If the condition does not pass, the expression is not executed.

You can see this if we change z to be 5 and rerun the code: there's no output. Suppose now that you
want to print out "z is odd" in this case. How to do this?

else

Well, you can simply use an else statement, like this. If we run it with z equal to 5, the condition is not
true, so the expression for the else statement gets printed out. The general recipe looks like this: for
the else statement, you don't need to specify a condition. The corresponding expression gets run if the
condition of the if statement it belongs to does not hold.

elif

You can think of cases where even more customized behavior is necessary. Say you want different
printouts for numbers that are divisible by 2 and by 3. You can throw some elifs in there to get the job
done. Take this example. Can you tell what this script will print out if you run it? If z equals 3, the
first condition is False, so it goes over to the next condition. This condition does hold, so the
corresponding print statement is executed.

Suppose now that z equals 6. Both the if and elif conditions hold in this case. Will two printouts occur?
Nope. As soon as Python bumps into a condition that is true, it executes the corresponding code and
then leaves the control structure after that. This means the second condition, corresponding to the elif,
is never reached, so there's no corresponding printout.

Example Script: [Link]

Here is an example script to demonstrate the use of if, else, and elif statements in Python:

# [Link]

z=4

# Check if z is even

if z % 2 == 0:

print("z is even")

print("This is another line inside the if statement")

# Adding an else statement

else:

print("z is odd")

# Using elif for more complex conditions

z=3

if z % 2 == 0:

print("z is divisible by 2")

elif z % 3 == 0:

print("z is divisible by 3")

# Another elif example

z=6

if z % 2 == 0:

print("z is divisible by 2")

elif z % 3 == 0:

print("z is divisible by 3")

Explanation

 Basic if Statement: Checks if z is even and prints a message if true.

 Exiting if Statement: The script continues with unindented code after the if block.
 Multiple Lines in if Statement: Shows how to include multiple lines of code within an if block.
 Adding else Statement: Adds an alternative action if the if condition is not met.
 Using elif for More Conditions: Demonstrates how to handle multiple conditions with elif.
 Order of Execution: Shows that once a true condition is found, the rest of the elif statements
are skipped.

Filtering Pandas DataFrames

In previous videos, I gave some examples of how the NumPy array can be useful to do comparison
operations and boolean operations on an element-wise basis. Let's now use this knowledge on the
Pandas DataFrame.

brics

For starters, let's import the BRICS dataset again from the CSV file; here it is.

Goal

Suppose you now want to keep the countries, so the observations in this case, for which the area is
greater than 8 million square kilometers. There are three steps to this. First of all, we want to get the
area column from brics. Next, we perform the comparison on this column and store its result. Finally,
we should use this result to do the appropriate selection on the DataFrame.

Step 1: Get Column

So the first step, getting the area column from brics. There are many different ways to do this. What's
important here is that we ideally get a Pandas Series, not a Pandas DataFrame. Let's do this with
square brackets, like this: brics['area']. This loc alternative, and this iloc version, would also work
perfectly fine.

Step 2: Compare

Next, we actually perform the comparison. To see which rows have an area greater than 8, we simply
append > 8 to the code from before, like this: brics['area'] > 8. Now we get a Series containing
booleans. If you compare it to the actual area values, you can see that the areas with a value over 8
correspond to True, and the ones with a value under 8 correspond to False now. Let me store this
Boolean Series as is_huge.

. Step 3: Subset DF

The final step is using this boolean Series to subset the Pandas DataFrame. This is something I haven't
shown you yet. To do this, you put is_huge inside square brackets. The result is exactly what we want:
only the countries with an area greater than 8, namely Brazil, Russia, and China.

Summary

So let's summarize this: I selected the area column, performed a comparison on this column, and
stored it as is_huge so that I can use it to index the brics DataFrame. These different commands do
the trick. However, we can also write this in a one-liner: simply put the code that defines is_huge
directly in the square brackets. Great!

Boolean Operators

Now we haven't used boolean operators yet. Remember that we used this logical_and function from
the NumPy package to do an element-wise boolean operation on NumPy arrays? Because Pandas is
built on NumPy, you can also use that function here. Suppose you only want to keep the observations
that have an area between 8 and 10 million square kilometers. After importing NumPy as np, we can
use the logical_and() function to create a Boolean Series. The only thing left to do is placing this code
inside square brackets to subset brics appropriately. This time, only Brazil and China are included.
Russia has an area of 17 million square kilometers, which doesn't meet the conditions. I hope these
examples have shown you how easy it is to filter DataFrames to get interesting results.

Example Code

python

import pandas as pd

import numpy as np

# Load the BRICS dataset

brics =pd.read_csv('[Link]')

# Step 1: Get the 'area' column

area = brics['area']

# Step 2: Compare the area

is_huge = area > 8

# Step 3: Subset DataFrame using the boolean Series

huge_countries = brics[is_huge]

print(huge_countries)

# Using boolean operators

# Filter countries with area between 8 and 10 million square kilometers

is_between_8_and_10 = np.logical_and(brics['area'] > 8, brics['area'] < 10)

subset_countries = brics[is_between_8_and_10]

print(subset_countries)
Explanation

 Getting the Column: Access the 'area' column from the DataFrame using square brackets.
 Performing Comparison: Create a boolean Series by comparing the 'area' values.
 Subsetting the DataFrame: Use the boolean Series to filter the DataFrame and get only the
rows that match the condition.
 Boolean Operators: Apply logical_and from NumPy to filter rows based on multiple
conditions.

While Loops
In the previous chapter, you learned about if-elif-else statements, which control the flow of your script
by executing code based on conditions. Unlike if statements, which execute their code block only once
if the condition is True, while loops execute their code block repeatedly as long as their condition
remains True.

Structure of while Loops

The syntax of a while loop is quite similar to an if statement. Here’s a basic outline of its structure:

Pythoncode:

while condition:

# Code to execute repeatedly

 condition: This is a boolean expression that is evaluated before each iteration. If it evaluates to
True, the code inside the loop is executed. If it evaluates to False, the loop ends, and the
program continues with the next statement following the loop.

Example of while Loop

Let’s consider an example where you want to simulate an algorithm that reduces an error value until
it falls below a certain threshold:

Python code:

# Initialize the error

error = 50

# While loop to repeatedly perform an action

while error > 1:

error = error / 4 # Update error value

print(error) # Print the updated error

How It Works
 1. Initialization: You start with an error value of 50.
2. Condition Check: The while loop checks if error is greater than 1.
o If True, the code inside the loop executes.
o If False, the loop ends.
3. Inside the Loop:
o The error value is divided by 4.
o The new error value is printed.
4. Re-evaluation: The loop condition is checked again. If error is still greater than 1, the loop
continues. If not, the loop ends.

Example Execution

 First Iteration:
o error = 50
o error is divided by 4: 50 / 4 = 12.5
o 12.5 is printed.
 Second Iteration:
o error = 12.5
o 12.5 / 4 = 3.125
o 3.125 is printed.
 Third Iteration:
o error = 3.125
o 3.125 / 4 = 0.78125
o 0.78125 is printed.

Once error is less than or equal to 1, the loop exits.

Potential Pitfalls

1. Infinite Loop: If you forget to update the error within the loop, the condition may always be
True, causing an infinite loop. For example:

Pythoncode:

while True:

print("This will run forever")

To stop an infinite loop, press Ctrl + C in most environments.

2. Condition Evaluation: Make sure your loop condition eventually becomes False to avoid an
infinite loop. Always check that the loop contains some logic to change the condition.

Practical Use Cases

 Numerical Methods: Iteratively improving calculations until a result is within a desired range.
 Monitoring Systems: Continuously checking conditions until they are met.
 Games or Simulations: Repeatedly updating game state or simulation until a condition is
satisfied.

The for Loop

In Python, the for loop is a powerful tool for iterating over sequences like lists, tuples, and strings.
Unlike the while loop, which continues as long as a condition is true, the for loop iterates over a
sequence and executes a block of code for each item in that sequence.

Syntax of for Loop

Here’s the basic structure of a for loop:

Python code:

for variable in sequence:

# Code to execute for each item in the sequence

 variable: This is a temporary variable that holds the value of the current item in the sequence
during each iteration.
 sequence: This can be any iterable object (like a list, tuple, string, etc.) that you want to loop
over.

Example: Iterating Over a List

Suppose you have a list called fam containing the heights of family members:

Python code:

fam = [1.73, 1.68, 1.80, 1.75]

To print each height in the list separately, you can use a for loop like this:

Python code:

for height in fam:

print(height)

How It Works

1. Initialization: The for loop starts by taking the first item from the fam list and assigns it to the
variable height.
2. Iteration: It then executes the code inside the loop (in this case, print(height)), printing the
current height.
3. Next Item: The loop moves to the next item in the fam list and repeats the process until all
items have been processed.

Example with enumerate()

If you also want to display the index of each item along with its value, you can use the enumerate()
function. This function returns both the index and the value of each item in the sequence.

Pythoncode:

for index, height in enumerate(fam):

print("Index:", index, "Height:", height)

How It Works

 enumerate(fam): This generates pairs of (index, height) where index is the position of height in
the list fam.
 Iteration: For each pair, index and height are assigned values, and the print statement displays
both.

Example: Iterating Over a String

You can also use a for loop to iterate over characters in a string. Here’s how you might capitalize and
print each character in the string "family":

Pythoncode:

word = "family"

for c in word:

print([Link]())

Summary

 Basic Loop: for variable in sequence: iterates over items in the sequence.
 With enumerate(): for index, item in enumerate(sequence): allows access to both the index and
the value.
 Strings: for c in "string": iterates over each character in the string.

Using for loops makes it easy to handle and process collections of data without having to manually
manage indices or iteration counters.

Looping Over Different Data Structures

In Python, looping over different data structures involves different approaches depending on the type
of structure you're working with. Here’s a breakdown of how to handle loops with dictionaries and
NumPy arrays.

Looping Over Dictionaries

Dictionaries store data as key-value pairs, and you often want to iterate over these pairs to access both
the key and its associated value.

Example: Dictionary Loop

world = {

"Afghanistan": 38928341,

"Brazil": 212559417,

"China": 1439323776,
 "Denmark": 5792203

To print each key-value pair:

for key, value in [Link]():

print(f"Country: {key}, Population: {value}")

 [Link](): This method returns a view object that displays a list of a dictionary's key-value
tuple pairs.

Key Points for Dictionaries:

 Order: Dictionaries in Python (from version 3.7+) maintain insertion order. However, if you
are using an older version, the order may not be preserved.
 Variable Names: The names key and value are arbitrary; you can use any valid variable
names.

Looping Over NumPy Arrays

NumPy arrays can be one-dimensional (1D) or multi-dimensional (2D or more). The method for
iterating over these arrays varies slightly.

1D NumPy Array

Given a 1D array bm

import numpy as np

bmi = [Link]([22.5, 27.3, 19.8, 25.0])

To print each element:

for value in bmi:

print(value)

2D NumPy Array

For a 2D array meas created from np_height and np_weight:

Python code:

np_height = [Link]([1.73, 1.68, 1.80, 1.75])

np_weight = [Link]([65.0, 58.0, 72.0, 70.0])

meas = np.column_stack((np_height, np_weight))

To print each element:

Python code:
import numpy as np

for row in meas:

for value in row:

print(value)

Using nditer() for Multi-dimensional Arrays

For more efficient and convenient iteration over a multi-dimensional NumPy array, you can use
[Link]():

Python code:

import numpy as np

meas = np.column_stack((np_height, np_weight))

for value in [Link](meas):

print(value)

Recap

 Dictionaries: Use the items() method to loop through key-value pairs.

 NumPy Arrays: Use a basic for loop for 1D arrays. For 2D arrays or higher dimensions, you
can use nested loops or [Link]().

These methods help you handle different data structures effectively, allowing you to extract and
manipulate the data as needed.

Looping Over Pandas DataFrames ,part 2

When working with Pandas DataFrames, you might want to loop through rows or columns to process
data. Here’s a guide on how to do this effectively.

Basic For Loop with DataFrames

Initial Attempt

A basic for loop over a DataFrame does not give you the row data directly. Instead, it gives you the
column names. For example:

import pandas as pd

# Importing the DataFrame

brics = pd.read_csv('[Link]')

# Basic for loop

for item in brics:

print(item)

This will print column names, not row data.

Iterating Over Rows with iterrows()

To iterate over rows, you should use the iterrows() method. This method returns an iterator that
yields index and row data for each row in the DataFrame.

Example Using iterrows()

import pandas as pd

# Importing the DataFrame

brics = pd.read_csv('[Link]')

# Iterating over rows

for lab, row in [Link]():

print(f"Label: {lab}")

print(f"Row data:\n{row}\n")

 lab is the index label of the row.

 row is a Pandas Series containing row data.

Selective Print

If you want to print specific columns, like the capital, you can access the column within the loop:

for lab, row in [Link]():

print(f"Label: {lab}, Capital: {row['capital']}")

Adding a New Column

To add a new column based on existing data, you can use a for loop, but this method can be inefficient
for large DataFrames. Here's how you might do it:

Example: Adding name_length Column

brics['name_length'] = None # Initialize the new column

for lab, row in [Link]():

[Link][lab, 'name_length'] = len(row['country'])

print(brics)

More Efficient Approach: Using apply()

Instead of using a for loop, you can use the apply() function to perform element-wise operations on
DataFrame columns. This method is more efficient and concise.

Example Using apply()

import pandas as pd

# Importing the DataFrame

brics = pd.read_csv('[Link]')

# Adding a new column using apply

brics['name_length'] = brics['country'].apply(len)

print(brics)

Key Points

 iterrows(): Useful for iterating over rows when you need both the index and row data.
However, it's less efficient for large DataFrames.
 apply(): A more efficient way to apply a function to a DataFrame column. It performs
operations in a vectorized manner, which is generally faster and more readable.

Summary

 Use iterrows() for row-wise iteration when necessary, but be mindful of its performance
implications for large datasets.
 Prefer apply() for applying functions to DataFrame columns as it is more efficient and concise.

These techniques will help you manipulate and transform data in Pandas DataFrames more
effectively.

Simulating Random Events

In this section, we'll learn how to use random numbers to simulate scenarios with chance or
probability. We will explore how to generate random numbers, set seeds for reproducibility, and use
these techniques to simulate events like rolling dice or tossing coins.

Random Number Generation

1. Random Generators

To simulate random events, you need random number generators. In Python, you can use the numpy
library for this purpose. Specifically, the random package within numpy is helpful.

Basic Example:

import numpy as np

# Generate a random number between 0 and 1

random_number = [Link]()
print(random_number)

2. Seeding for Reproducibility

Computers generate pseudo-random numbers, which means the numbers are generated using a
deterministic algorithm starting from a seed. Setting a seed ensures that the sequence of random
numbers can be reproduced.

Example with Seed:

import numpy as np

# Set the seed

[Link](123)

# Generate two random numbers

num1 = [Link]()

num2 = [Link]()

print(num1, num2)

If you set the seed to the same value (123 in this case), you'll get the same sequence of numbers each
time you run the script. This is useful for debugging and sharing results.

3. Coin Toss Simulation

To simulate a coin toss, you can use the randint() function from [Link], which generates
random integers.

Example of Coin Toss:

import numpy as np

# Set the seed for reproducibility

[Link](123)

# Generate a random integer, 0 or 1

coin = [Link](0, 2)

print("Heads" if coin == 0 else "Tails")

This code will print either "Heads" or "Tails" based on the random integer generated. The seed
ensures the result is the same each time you run the script.

Simulating Complex Scenarios

For more complex simulations, such as the Empire State Building scenario where you simulate
moving up and down based on dice rolls, follow these steps:

1. Define the Rules:

 o Move down one step for rolls of 1 or 2.
o Move up one step for rolls of 3, 4, or 5.
o On a roll of 6, roll again to determine how many steps to move up.
o There is a 0.1% chance of falling and starting from step 0 again.
2. Simulate the Process: Use a while loop to simulate the movement and dice rolls until you reach
the desired step or fall. You can run this simulation many times to estimate the probability of
reaching the target step.

Example Simulation Code:

import numpy as np

def simulate_walk():

[Link](123)

steps = 0

while steps < 60:

roll = [Link](1, 7)

if roll in [1, 2]:

steps = max(0, steps - 1) # Move down

elif roll in [3, 4, 5]:

steps += 1 # Move up

elif roll == 6:

extra_roll = [Link](1, 7)

steps += extra_roll # Move up by extra_roll steps

# Simulate falling down

if [Link]() < 0.001: steps = 0 # Fall down to step 0

return steps >= 60

# Run the simulation multiple times

simulations = 10000

success_count = sum(simulate_walk() for _ in range(simulations))

probability = success_count / simulations

print(f"Probability of reaching 60 steps: {probability:.4f}")

This code runs the simulation 10,000 times to estimate the probability of reaching 60 steps. The use of
[Link]() ensures reproducibility, while [Link]() simulates the chance of falling.
Summary

 Random Number Generation: Use [Link] for generating random numbers. Set a seed
for reproducibility.
 Coin Toss: Use randint() to simulate a coin toss and conditionals to interpret results.
 Complex Simulations: Use loops and conditional statements to simulate more complex
scenarios and estimate probabilities by running multiple simulations.

This approach will help you model and understand processes involving randomness and probability
effectively.

Random Walk
In this section, we’ll explore the concept of a random walk and see how to simulate it using Python. A
random walk involves taking a series of random steps, and it can be used to model various phenomena
such as the movement of molecules in a gas or the fluctuation of financial markets.

Understanding Random Walks

1. Concept of a Random Walk

A random walk is a sequence of steps where each step is determined randomly. For example, if you
use a dice to determine your next step, and you repeat this process multiple times, you get a sequence
of random steps, or a random walk.

oExample Use Cases:

 The movement of particles in a liquid.
 The evolution of stock prices over time.
2. Simulating Random Walks

To simulate a random walk, you'll need to build a list that tracks the progression of your steps. Let’s
start with a simple example of a random walk using coin tosses.

Simulating a Random Walk: Coin Toss Example

1. Basic Coin Toss Simulation

First, let’s simulate a series of coin tosses and track the outcomes.

Code Example:

import numpy as np

# Set seed for reproducibility

[Link](123)

# Initialize the outcomes list

outcomes = []

# Simulate 10 coin tosses

for _ in range(10):
 coin = [Link](0, 2) # Random integer 0 or 1

if coin == 0:

[Link]("heads")

else:

[Link]("tails")

print(outcomes)

This script will print a list of 10 outcomes, which are either "heads" or "tails". This is a random
sequence but not a random walk yet.

2. Converting to a Random Walk

To convert this into a random walk, you need to track the cumulative number of "tails" over time.
Here’s how you can modify the previous script:

Code Example:

import numpy as np

# Set seed for reproducibility

[Link](123)

# Initialize the list with the starting value

tails = [0]

# Simulate 10 coin tosses

for _ in range(10):

coin = [Link](0, 2) # Random integer 0 or 1

if coin == 1:

# Increment the count of tails

[Link](tails[-1] + 1)

else:

# Maintain the previous count if heads

[Link](tails[-1])

print(tails)
In this script, the list tails starts with [0], representing the initial state (no tails). Each time a 1 (tails) is
generated, the count of tails is incremented by 1. If a 0 (heads) is generated, the count remains the
same. The resulting list shows the cumulative number of tails over the sequence of coin tosses.

Analysis

1. Comparison with Non-Sequential List

Compare the outputs of the random sequence list and the random walk list:

The first list contains independent outcomes of heads or tails.

o
The random walk list shows how the count of tails evolves over time, illustrating the
o
cumulative effect of each random step.
2. Visualizing the Walk

For better understanding, you can plot the random walk using libraries like matplotlib:

import numpy as np

import [Link] as plt

# Set seed for reproducibility

[Link](123

# Initialize the list with the starting value

tails = [0]

# Simulate 10 coin tosses

for _ in range(10):

coin = [Link](0, 2) # Random integer 0 or 1

if coin == 1:

# Increment the count of tails

[Link](tails[-1] + 1)

else:

# Maintain the previous count if heads

[Link](tails[-1])

# Plot the random walk

[Link](tails, marker='o')
[Link]('Random Walk Simulation')

[Link]('Toss Number')

[Link]('Cumulative Tails Count')

[Link]()

This plot visualizes the random walk, showing how the number of tails changes over each toss.

Summary

 Random Walk: A sequence of steps determined randomly, useful for modeling various
stochastic processes.
 Simulation: Use lists and loops to simulate random walks. Track cumulative outcomes to see
the evolution over time.
 Visualization: Plot the results to better understand and interpret the random walk.

By simulating and analyzing random walks, you can model and explore complex systems involving
randomness and uncertainty.

Understanding and Visualizing the Distribution of Random

Walks
In this final segment of the course, we'll dive into how to analyze and visualize the distribution of
random walks. By simulating a random walk multiple times, you can understand the distribution of
final outcomes and calculate probabilities.

1. Concept of Distribution

 Distribution: The distribution of a random walk is the collection of all possible final outcomes
from many simulations. It shows how frequently each outcome occurs.
 Application: By simulating a process like tossing a die or flipping a coin multiple times, you
can observe the distribution of outcomes and derive probabilities.

2. Random Walk Simulation

To simulate the random walk, we use the following approach:

1. Simulate Random Walks

o Simulate a random walk by running a series of random steps multiple times.
o Each run will produce a final outcome, which contributes to the distribution.
2. Example: Coin Toss Simulation

In this example, we simulate a coin toss game multiple times and track the number of tails obtained.

Code Example:

import numpy as np

import [Link] as plt

# Set seed for reproducibility

[Link](123)

# Initialize an empty list to store final counts

final_tails = []

# Number of simulations

num_simulations = 1000

# Run the simulation

for _ in range(num_simulations):

tails_count = 0

for _ in range(10):

coin = [Link](0, 2) # Random integer 0 or 1

tails_count += coin

final_tails.append(tails_count)

# Plot the histogram of the results

[Link](final_tails, bins=10, edgecolor='black')

[Link]('Distribution of Tails After 10 Tosses')

[Link]('Number of Tails')

[Link]('Frequency')

[Link]()

3. Visualizing the Distribution

1. Histogram of Results
o Histogram: A histogram is used to visualize the distribution of outcomes.
o Bins: The histogram is divided into bins to show the frequency of outcomes in each
range.
2. Increasing the Number of Simulations
o 100 Simulations: Initial histogram might not be smooth.
o 1,000 Simulations: Provides a clearer distribution.
o 10,000 Simulations: Distribution starts to resemble the theoretical distribution more
closely.

Code Example for Different Simulations:

num_simulations = 10000 # Change this value to 100, 1000, or 10000

# Initialize an empty list to store final counts

final_tails = []

# Run the simulation

for _ in range(num_simulations):

tails_count = 0

for _ in range(10):

coin = [Link](0, 2) # Random integer 0 or 1

tails_count += coin

final_tails.append(tails_count)

# Plot the histogram of the results

[Link](final_tails, bins=10, edgecolor='black')

[Link](f'Distribution of Tails After 10 Tosses ({num_simulations} Simulations)')

[Link]('Number of Tails')

[Link]('Frequency')

[Link]()

4. Interpreting the Results

 Bell-Shaped Curve: As the number of simulations increases, the histogram approaches a bell-
shaped curve, which is the theoretical distribution of the number of tails.
 Theoretical Distribution: This is what you would expect to see if you calculated the distribution
analytically.

Summary

 Simulation: Simulate the random walk multiple times to gather data on possible outcomes.
 Distribution: Visualize the distribution using histograms to understand the frequency of
different outcomes.
 Increasing Simulations: More simulations lead to a more accurate representation of the
theoretical distribution.

By analyzing and visualizing the distribution of random walks, you gain insights into the behavior of
stochastic processes and can make informed predictions about probabilities.

Distribution of Random Walks

In this final section, you will learn how to analyze the distribution of random walks by simulating the
process multiple times and visualizing the results.

1. Concept of Distribution
  Distribution: In a random walk, the final position or outcome after a series of steps can be
represented as a distribution. This distribution shows how often each possible final outcome
occurs across many simulations.

2. Simulating Random Walks

To answer the question of the likelihood of reaching a specific number of steps, such as 60 steps high,
you need to simulate the random walk many times. Each simulation will give you a different final
position, and by compiling these results, you get a distribution of final outcomes.

3. Example: Coin Tosses

Here's how you can simulate the process of counting tails in 10 coin tosses to build a distribution:

1. Simulation Code

import numpy as np

import [Link] as plt

# Set seed for reproducibility

[Link](123)

# Initialize an empty list to store final counts

final_tails = []

# Number of simulations

num_simulations = 1000 # You can change this to 100, 1000, or 10000

# Run the simulation

for _ in range(num_simulations):

tails_count = 0

for _ in range(10):

coin = [Link](0, 2) # Random integer 0 or 1

tails_count += coin

final_tails.append(tails_count)

# Plot the histogram of the results

[Link](final_tails, bins=10, edgecolor='black')

[Link](f'Distribution of Tails After 10 Tosses ({num_simulations} Simulations)')

[Link]('Number of Tails')
[Link]('Frequency')

[Link]()

2. Histogram Interpretation
o 100 Simulations: The histogram might look jagged and less smooth.
o 1,000 Simulations: The histogram becomes smoother and better represents the
distribution.
o 10,000 Simulations: The histogram approaches a bell-shaped curve, resembling the
theoretical distribution.

4. Visualizing the Distribution

 Histogram: Use a histogram to visualize the distribution of final outcomes. This will show the
frequency of different outcomes and give insight into the overall distribution pattern.
 Increasing Simulations: As you increase the number of simulations, the histogram will become
smoother and more accurate. With 10,000 simulations, the distribution will closely match the
theoretical distribution.

5. Conclusion

 Distribution of Outcomes: By running a large number of simulations, you create a distribution

that approximates the theoretical distribution.
 Practical Use: This approach helps in understanding the behavior of random processes and
estimating probabilities.

By visualizing and analyzing the distribution of random walks, you gain valuable insights into how
often specific outcomes occur and can make informed decisions based on these probabilities.