Pandas functions
Series and Dataframe Objects in Pandas
In Python's pandas library, the main difference between a DataFrame and a
Series object is their structure and intended use:
1. Series:
A Series is essentially a one-dimensional labeled array capable of holding
any data type (integers, strings, floats, etc.).
It has an index that labels each element in the series.
It can be thought of as a single column of data.
Example:Output:
python
Copy code
import pandas as pd
s = [Link]([1, 2, 3, 4], index=['a', 'b', 'c', 'd'])
print(s)
css
Copy code
a 1
b 2
c 3
d 4
dtype: int64
Pandas functions 1
2. DataFrame:
A DataFrame is a two-dimensional labeled data structure with columns of
potentially different types (like a table with rows and columns).
Each column in a DataFrame is a Series, and the DataFrame is a collection
of Series aligned along a common index.
It can be thought of as a table (or spreadsheet) where rows and columns
can hold different types of data.
Example:Output:
python
Copy code
data = {'Column1': [1, 2, 3, 4], 'Column2': ['A', 'B',
'C', 'D']}
df = [Link](data)
print(df)
css
Copy code
Column1 Column2
0 1 A
1 2 B
2 3 C
3 4 D
In summary:
A Series is a single-dimensional array, while a DataFrame is a multi-
dimensional table.
A DataFrame is essentially a collection of Series aligned in a tabular format.
Correlation and Covariance
Pandas functions 2
Correlation and Covariance are both measures used in statistics and data
analysis to describe the relationship between two variables. However, they differ
in their interpretation, scale, and how they measure this relationship:
1. Covariance:
Definition: Covariance measures the direction of the linear relationship
between two variables. It tells us whether two variables tend to increase or
decrease together.
F ormula : Cov(X, Y ) = n1i = 1 ∑ n(Xi− Xˉ)(Yi− Y ˉ)
Interpretation:
If the covariance is positive, both variables tend to increase or decrease
together.
If the covariance is negative, when one variable increases, the other tends
to decrease.
If the covariance is zero, there is no linear relationship between the
variables.
Scale: The magnitude of covariance is not standardized, meaning it can take
any value and is sensitive to the units of the variables. This makes it difficult to
compare across different datasets.
2. Correlation:
Definition: Correlation measures both the strength and the direction of the
linear relationship between two variables, but it is normalized and unit-free,
making it easier to interpret and compare across different datasets.
Formula (for Pearson correlation):
Corr(X, Y ) = σXσY Cov(X, Y )
Cov(X, Y )
Corr(X, Y ) = Cov(X, Y )σXσY Corr(X, Y ) =
σX σY
Where σX\sigma_XσX and σY\sigma_YσY are the standard deviations of X and Y.
Pandas functions 3
Interpretation:
The correlation coefficient (r) ranges from 1 to 1:
rr
r=1r = 1r=1: Perfect positive correlation (variables increase together).
r=−1r = -1r=−1: Perfect negative correlation (one variable increases
while the other decreases).
r=0r = 0r=0: No linear correlation.
Scale: Correlation is standardized, meaning the values range between -1 and
1, making it easier to compare relationships across different variables or
datasets.
Key Differences:
Feature Covariance Correlation
Measures the direction of the Measures the strength and
Meaning
relationship between variables direction of the relationship
Unbounded (can be any positive or
Range Always between -1 and 1
negative number)
Easier to interpret due to
Interpretation Difficult to interpret due to scale
standardization
Depends on the units of the
Units Unit-free
variables
Easier to compare across
Comparison Difficult to compare across datasets
datasets
In summary:
Covariance tells you if two variables move together (positively or negatively)
but not how strongly.
Correlation tells you both the direction and strength of the linear relationship in
a more interpretable and standardized form.
.loc and .iloc
Pandas functions 4
In pandas, .loc and .iloc are used to select rows and columns from a DataFrame,
but they differ in how they index the data:
1. .loc : Label-based indexing
Definition: .loc is primarily used for selecting data by label or index name. It
includes both the start and end labels when selecting a range.
Behavior:
It allows for label-based indexing, which means you can select rows and
columns based on their explicit labels (index names or column names).
It supports slicing and selecting specific rows and columns by their labels.
Example:
python
Copy code
import pandas as pd
data = {'A': [1, 2, 3], 'B': [4, 5, 6], 'C': [7, 8, 9]}
df = [Link](data, index=['row1', 'row2', 'row3'])
# Using .loc to select by labels
df_loc = [Link]['row1', 'A'] # Selects the value in 'row
1' and column 'A'
print(df_loc)
Output:
Copy code
1
Selecting rows by label:
python
Copy code
Pandas functions 5
[Link]['row2'] # Selects all columns of row2
Selecting multiple rows and columns:
python
Copy code
[Link]['row1':'row2', ['A', 'B']] # Slices rows from
'row1' to 'row2' and columns 'A' and 'B'
2. .iloc : Position-based indexing
Definition: .iloc is used for selecting data by integer position. It excludes the
end index when selecting a range.
Behavior:
It allows for position-based indexing, which means you select rows and
columns based on their integer index positions, regardless of the actual
labels.
Like Python slicing, .iloc excludes the ending position when slicing
ranges.
Example:
python
Copy code
# Using .iloc to select by positions
df_iloc = [Link][0, 0] # Selects the value at the 0th ro
w and 0th column (first row, first column)
print(df_iloc)
Output:
Pandas functions 6
Copy code
1
Selecting rows by position:
python
Copy code
[Link][1] # Selects all columns of the second row (in
dex 1)
Selecting multiple rows and columns:
python
Copy code
[Link][0:2, 0:2] # Slices rows from position 0 to 1 a
nd columns from position 0 to 1
Key Differences:
Feature .loc (Label-based) .iloc (Position-based)
Position-based (integer
Indexing method Label-based (index/column names)
positions)
Start/end Includes both start and end in
Excludes the end in ranges
inclusion ranges
Type of input Row and column labels Integer index positions
When you know the label/index of When you want to select by
Use case
rows/columns integer positions
Summary:
Use .loc when you want to select rows and columns by labels.
Use .iloc when you want to select rows and columns by position.
Pandas functions 7