Exploratory Data Analysis in Python
Exploratory Data Analysis in Python
1. Loading Libraries
import pandas as pd
import numpy as np
import [Link] as plt
import seaborn as sns
from scipy import stats
from [Link] import MinMaxScaler, StandardScaler
Exploratory Data Analysis in Python
2. Loading the Dataset
# Example: Loading a CSV file
df = pd.read_csv('your_dataset.csv')
Exploratory Data Analysis in Python
3. Data Overview
# Display the first few rows of the dataset
print([Link]())
# Display summary statistics
print([Link]())
# Display information about the dataset
print([Link]())
Exploratory Data Analysis in Python
4. Cleaning Data
# Handling missing values
print([Link]().sum())
[Link]([Link](), inplace=True)
# Handling duplicates
print([Link]().sum())
df.drop_duplicates(inplace=True)
Exploratory Data Analysis in Python
5. Preprocessing Data
# Encoding categorical variables
df = pd.get_dummies(df, columns=['categorical_column'])
# Feature Engineering
df['new_feature'] = df['existing_feature1'] * df['existing_feature2']
Exploratory Data Analysis in Python
6. Outlier Detection and Treatment
# Using Z-score to identify outliers
z_scores = [Link](df['column_name'])
abs_z_scores = [Link](z_scores)
filtered_entries = (abs_z_scores < 3)
df = df[filtered_entries]
Exploratory Data Analysis in Python
7. Scaling and Normalization
# Min-Max Scaling
scaler = MinMaxScaler()
df[['column1', 'column2']] = scaler.fit_transform(df[['column1', 'column2']])
# Alternatively, for Standardization
# scaler = StandardScaler()
# df[['column1', 'column2']] = scaler.fit_transform(df[['column1', 'column2']])
Exploratory Data Analysis in Python
8. Data Visualization (Examples)
# Histogram
[Link](figsize=(10, 6))
[Link](df['column_name'], kde=True)
[Link]('Histogram of column_name')
[Link]()
# Boxplot
[Link](figsize=(10, 6))
[Link](x=df['column_name'])
[Link]('Boxplot of column_name')
[Link]()
# Scatter plot
[Link](figsize=(10, 6))
[Link](x='column1', y='column2', data=df)
[Link]('Scatter plot between column1 and column2')
[Link]()
# Heatmap for correlation
[Link](figsize=(12, 8))
[Link]([Link](), annot=True, cmap='coolwarm')
[Link]('Correlation Heatmap')
[Link]()
Exploratory Data Analysis in Python
Exploratory Data Analysis in Python
9. Summarizing Findings
print("Key Findings:")
print("1. Description of key patterns or anomalies.")
print("2. Potential relationships between features.")
print("3. Insights on missing values and outliers.")