Week 2 - NUMPY
Vector, matrix, multi-array (tensor) computations
Base computation for ML libraries: scikit-learn
Tensor computations → pytorch, tensorflow
Using list comprehension to create a dictionary to count frequency:
s = "hello ho"
frequence = {char : [Link](char) for char in sorted(s) if char != ' '}
print(frequence) # {'e': 1, 'h': 2, 'l': 2, 'o': 2}
Why multidimensional Array:
Images : 2D array --> Compute parallelly --> on the GPU
Sound, time series data: 1D array
--> Sequential data: Time Series Data
Text : 1D array
→ All kind of data can be converted to numerical array
Numpy syntax
Multi-dimensional array : ndarray
c = []
for i in range(len(a)):
[Link](a[i]*b[i]) # C = a * b
LIST vs Numpy
List can store dynamic types
List = [True, "2", 3.0, 4]
NumPy arrays: all contain the same type
[Link]([1, 2, 3, 4], dtype='float32')
Shape: shape
Number of dimensions: ndim
Data size: size
Type: The elements are all of the same type, referred to as the array dtype.
out = [Link](2,3) # create a random ndimension-array
#[[0.16750036 0.20326896 0.34482044]
# [0.40357482 0.73957514 0.28669615]]
[Link] # print the shape (2,3)
[Link] # the dimension --> 2 (row)
[Link] # total number of elements 6
[Link] # dtype(['float64'])
# to type casting data type
[Link](int)
[Link] #dtype('int64')
Dimensions are called axes.
a = [[0., 0., 0.], #a[2][3]
[1., 1., 1.]]
Your array has 2 axes.
**
The first axis has a length of 2 and
the second axis has a length of 3.
**
AXIS 0 is vertical
AXIS 1 is horizontal
This is for 2D array
EXPLANATION
import numpy as np
# Create the array
out = [Link]([
[[1, 2, 3, 4], # Layer 1
[5, 6, 7, 8],
[9, 10, 11, 12]],
[[10, 20, 30, 40], # Layer 2
[50, 60, 70, 80],
[90, 100, 110, 120]]
])
print("Original shape:", [Link]) # (2, 3, 4)
# Sum along axis 0
result = [Link](axis=0)
print("Result shape:", [Link]) # (3, 4)
print("Result:\n", result)
out[0] (Layer 1) has shape (3, 4).
out[1] (Layer 2) has shape (3, 4).
When you sum along axis=0 , the two layers get added element-wise:
[[ 11 22 33 44]
[ 55 66 77 88]
[ 99 110 121 132]] --> this is (3,4)
Function NameNaN-safe Version
[Link] [Link]
May have It will place "nan" if there is any error but not stop the program --> it ignore that
some error bugs and add the sum of other numbers
Slicing and Indexing
using [Link]
feature[[Link](feature[:,2] != 0)] : this one will check where the value at index will be
different from zero
then return the new array to assign for the new array
import torch
import numpy
a = [Link]([[1,2,3], [4,5,6], [5,6,7]])
[Link](a!=3)
--> print # [0,0,1,1,1,2,2,2], [0,1,0,1,2,0,1,2]
--> the first array is x index and second array is y index
--> so u can see the value 3 at index (0,2) is missing
a[[Link](a!=3)] = 1
--> this one different ==> this will print the 2D array but other value != 3 will
become 1
--> [1,1,3,
1,1,1,
1,1,1]
[Link](a[:,2] != 6) ---> so it will filter out all the row and col 2 --> and
check if which one does not have 6
->> print out # [0,2] whhich mean row 0 and row 2 does not have 6
a[[Link](a[:,2] != 6) --> print out the matrix with first and last document
[Link](a[1, :] != 6) # go to row 2 [4,5,6] --> iterate all col --> print out
[0,1] which is 4 and 5
# 2d index -> id index for column
a[:, [Link](a[1,:] != 6)[0]] # filter array
The condition means to extract: [0, 1] from (array([0, 1]),)
### `a[:, [0, 1]]`
This means: select all rows (`:`), but only columns `0` and `1`.
# [[1,2],[4,5], [5,6]]
arr = [Link]([1, 5, 2, 8, 3, 7])
# Get indices of elements greater than 4
indices = [Link](arr > 4)
print(indices) # Output: (array([1, 3, 5]),)
# Create a new array: if element > 4, keep it, otherwise replace with 0
new_arr = [Link](arr > 4, arr, 0)
print(new_arr) # Output: [0 5 0 8 0 7]
Concatenation
Splitting
Notice that N split-points, leads to N + 1 subarrays. The related
functions [Link] and [Link] are similar:
Element-wise operator
The + is a wrapper for add function
--> x + 2 = [Link](x,2)
Absolute value
The corresponding NumPy ufunc is [Link] , which is also available under the alias [Link] :
Specifying output
For binary ufuncs, there are some interesting aggregates that can be computed directly from the
object.
reduce on the add ufunc returns the sum of all elements in the
array | similarly --> multiply give the product
Aggregation
See in the other file
Broadcasting
Rules of Broadcasting
Broadcasting in NumPy follows a strict set of rules to determine the interaction between the two
arrays:
Rule 1: If the two arrays differ in their number of dimensions, the shape of the one with fewer
dimensions is padded with ones on its leading (left) side.
Rule 2: If the shape of the two arrays does not match in any dimension, the array with shape
equal to 1 in that dimension is stretched to match the other shape.
Rule 3: If in any dimension the sizes disagree and neither is equal to 1, an error is raised.
To make these rules clear, let's consider a few examples in detail.
By rule 1 the dimension will be padded-->
By rule 2 --> it is stretched
Second example
Rule 1 ->
Rule 2 ->
Third example
Rule 1
Rule 2
However --> it does not fit base on RULE 3 --> cannot be broadcast
So to fix this, WE DONT rely on DEFAULT padding
--> We gonna pad it OURSELF by using [Link]
--> this will pad the a array with another dimension from the RIGHT
Some Boolean
Fancy Indexing
Combine Index with slicing
mask --> True False True False --> print every other value in the X
Sorting
Partial Sort : Partitioning
Sometimes we're not interested in sorting the entire array, but simply want to find the k smallest
values in the array. NumPy provides this in the [Link] function. [Link] takes an
array and a number K; the result is a new array with the smallest K values to the left of the partition,
and the remaining values to the right, in arbitrary order:
. ROW
COL
Pandas: Row & Column Selection
Selecting subsets of your data is a fundamental operation in Pandas.
Selecting a single column: df['column_name'] or df.column_name (returns a Series)
Selecting multiple columns: df[['col1', 'col2']] (returns a DataFrame)
Selecting rows by slicing: df[start_index:end_index]
Example:
Python
import pandas as pd
data = {'name': ['Alice', 'Bob', 'Charlie', 'David'],
'age': [25, 30, 35, 40],
'city': ['New York', 'Los Angeles', 'Chicago', 'Houston']}
df = [Link](data)
# Select the 'age' column
print(df['age'])
#0 25
#1 30
#2 35
#3 40
#Name: age, dtype: int64
# Select 'name' and 'city' columns
print(df[['name', 'city']])
# Select the first two rows again B(2) is exclusive
print(df[0:2])
name age city
0 Alice 25 New York
1 Bob 30 Los Angeles
Indexing: loc[] vs iloc[]
Pandas provides two powerful methods for precise data selection:
loc[] (Label-based indexing): Selects data based on the labels of rows and columns.
Slicing with loc is inclusive of the end label.
iloc[] (Integer-location based indexing): Selects data based on the integer position of rows
and columns.
- Slicing with iloc is exclusive of the end position.
Syntax:
[Link][row_labels, column_labels]
[Link][row_positions, column_positions]
Example:
Python
# Set 'name' as the index for label-based indexing
df_indexed = df.set_index('name')
# --- Using loc[] ---
# Select row with label 'Bob'
print(df_indexed.loc['Bob'])
# Select rows from 'Alice' to 'Charlie' and columns 'age' and 'city'
print(df_indexed.loc['Alice':'Charlie', ['age', 'city']])
# --- Using iloc[] ---
# Select the first row (position 0)
print([Link][0])
name Alice
age 25
city New York
Name: 0, dtype: object
# Select the first three rows and the first two columns --> not have City
print([Link][0:3, 0:2])
name age
0 Alice 25
1 Bob 30
2 Charlie 35
Concatenate
The [Link]() function is used to combine two or more DataFrames either by rows or by
columns.
[Link]([df1, df2, ...], axis=0) : Stacks DataFrames vertically (appends rows).
[Link]([df1, df2, ...], axis=1) : Stacks DataFrames horizontally (appends columns).
Example:
Python
df1 = [Link]({'A': ['A0', 'A1'], 'B': ['B0', 'B1']})
df2 = [Link]({'A': ['A2', 'A3'], 'B': ['B2', 'B3']})
df3 = [Link]({'C': ['C0', 'C1'], 'D': ['D0', 'D1']})
# Concatenate by rows (axis=0)
row_concat = [Link]([df1, df2])
print(row_concat)
A B
0 A0 B0
1 A1 B1
0 A2 B2
1 A3 B3
# Concatenate by columns (axis=1)
col_concat = [Link]([df1, df3], axis=1)
print(col_concat)
A B C D
0 A0 B0 C0 D0
1 A1 B1 C1 D1
# But if we concate some info that does not share same index UNIT -->
row_concat = [Link]([df1, df3])
print(row_concat)
df3 have C and D while df1 have A and B
A B C D
0 A0 B0 NaN NaN
1 A1 B1 NaN NaN
0 NaN NaN C0 D0
1 NaN NaN C1 D1