Exploratory Data Analysis
(EDA) in Python
Understanding, Cleaning, and Visualizing Data
1
What is EDA?
Definition In simple words:
Exploratory Data Analysis (EDA) is the process of It’s the first step before any machine learning or
examining and summarizing data to discover decision-making — you “interview” your data to
patterns, detect anomalies, test hypotheses, and know what it’s telling you.
check assumptions.
2
Why Learn EDA?
Reason Why It Matters
Understand data Identify data structure, missing values, outliers
Improve quality Clean data ensures accurate results
Gain insights Visualize relationships and distributions
Prepare for ML Helps feature selection and preprocessing
Communicate Tell clear data stories to stakeholders
3
Steps in EDA Workflow
Load the dataset
Inspect the structure (shape, datatypes, nulls)
Clean the data (handle missing, duplicates, incorrect values)
Summarize statistics
Visualize data patterns
Detect outliers
Interpret and document insights
4
Python Libraries for EDA
Core Libraries Common Imports
•
pandas → Data handling & cleaning
•
numpy → Numerical operations
•
matplotlib → Visualization
•
seaborn → Statistical plots
•
scipy → Advanced statistics
5
Data Cleaning Techniques
Task Function / Method
Detect missing values [Link]().sum()
Fill missing values [Link]([Link]())
Drop missing rows [Link]()
Remove duplicates df.drop_duplicates()
Convert data types df['col'] = df['col'].astype(float)
6
7
Example – Cleaning Titanic Dataset
📊 Result:
A cleaned dataset ready for analysis. We've handled the
most common data quality issues: missing values and
duplicates.
8
Summary Statistics
Purpose: Summarize central tendency, spread, and behavior.
Numeric Summary Categorical Counts
Aggregations
9
Correlation Analysis
Goal:
Understand relationships between numeric variables.
📈 Insight:
Higher fare often correlates with higher class and survival
rate.
10
Visualization Techniques
Plot Type Purpose Code Example
Histogram Distribution [Link](df['age'])
Scatter Plot Relationships [Link](x='age', y='fare', data=df)
Box Plot Outlier detection [Link](x='class', y='fare', data=df)
11
Outlier Detection
Using IQR Method Using Z-score
12
Example – EDA on Titanic Dataset
Observations:
•
1st class passengers paid higher fares.
•
Age and fare show weak correlation.
13
Where Can You Use EDA?
Field Application
Healthcare Analyze patient data for disease trends
Finance Detect stock or transaction anomalies
E-commerce Study customer purchase behavior
Cybersecurity Identify abnormal network activity
Education Analyze student performance data
Research Summarize and validate experimental data
14
Quick Recap
Understand data
Clean and handle missing values
Summarize statistics
Visualize patterns
Detect and handle outliers
Communicate insights
15
Practice Exercise
Task: Perform EDA on the tips dataset
Load and inspect the dataset
Handle missing or duplicate values
Generate summary statistics
Plot:
Histogram for total_bill
Scatter plot between total_bill and tip
Box plot by day
Write 3 insights from your analysis
16