Data Science with Python
NumPy, Pandas, Matplotlib & Beyond — 10-Page Guide
# Topic Page
1 Python Data Stack 2
2 NumPy Essentials 3
3 Pandas for Data Wrangling 4
4 Data Cleaning 5
5 Exploratory Analysis 6
6 Matplotlib & Seaborn 7
7 Statistical Foundations 8
8 SQL for Data Scientists 9
9 Project Workflow 10
CHAPTER 1
The Python Data Science Stack
Python dominates data science due to its readable syntax and rich ecosystem. The core stack: NumPy
(arrays), Pandas (tabular data), Matplotlib/Seaborn (visualisation), Scikit-learn (ML), and Jupyter
(interactive notebooks).
• Jupyter Lab: interactive notebook environment for exploration and storytelling
• Conda / venv: environment management to isolate project dependencies
• pip install numpy pandas matplotlib seaborn scikit-learn jupyterlab
• Google Colab: free cloud Jupyter with GPU — great for ML experiments
CHAPTER 2
NumPy Essentials
NumPy provides N-dimensional array objects with vectorised operations orders of magnitude faster than
pure Python loops. All ML libraries are built on NumPy arrays.
import numpy as np
arr = [Link]([1, 2, 3, 4, 5])
matrix = [Link]((3, 3))
[Link](42)
X = [Link](1000, 10) # 1000 samples, 10 features
print([Link](axis=0)) # column means
print([Link](axis=0)) # column std devs
dot = X.T @ X # matrix multiplication
CHAPTER 3
Pandas for Data Wrangling
Pandas DataFrame is the workhorse of data science. It provides SQL-like operations on tabular data with
intuitive syntax.
import pandas as pd
df = pd.read_csv('[Link]')
print([Link], [Link])
df['revenue'] = df['price'] * df['qty']
monthly = [Link]('month')['revenue'].sum()
top5 = [Link](5, 'revenue')
merged = [Link](customers, on='cust_id', how='left')
CHAPTER 4
Data Cleaning
Real-world data is messy. Data cleaning typically consumes 60-80% of a data scientist's time. Identify and
handle missing values, duplicates, outliers, and type mismatches.
• [Link]().sum() — count missing per column
• [Link]([Link]()) — impute with median for numerical cols
• [Link](subset=['target']) — drop rows missing the target variable
• [Link]().sum() / df.drop_duplicates() — find and remove duplicates
• IQR method: outliers outside Q1-1.5*IQR or Q3+1.5*IQR
CHAPTER 5
Exploratory Data Analysis
EDA reveals distributions, correlations, and anomalies before modelling. Ask: What is the shape of each
feature? Are there correlations? Are there data leakage risks?
# Summary stats
[Link]()
# Correlation matrix
import seaborn as sns
corr = [Link]()
[Link](corr, annot=True, fmt='.2f')
# Value counts for categorical
df['category'].value_counts(normalize=True)
# Check class imbalance
df['target'].value_counts()
CHAPTER 6
Matplotlib & Seaborn
Visualisation communicates findings clearly. Matplotlib is the foundation; Seaborn builds high-level
statistical plots on top.
• Line plot: trends over time ([Link])
• Bar chart: categorical comparisons ([Link])
• Histogram: distribution of continuous variable ([Link])
• Scatter plot: relationship between two variables ([Link])
• Box plot: distribution + outliers by category ([Link])
• Pair plot: pairwise scatter matrix ([Link])
CHAPTER 7
Statistical Foundations
Statistics underpins data science. Descriptive stats summarise data; inferential stats draw conclusions
about populations from samples.
• Central Tendency: mean (sensitive to outliers), median (robust), mode
• Spread: variance, standard deviation, IQR
• Hypothesis Testing: p-value < 0.05 rejects null hypothesis at 95% confidence
• A/B Testing: t-test for means; chi-squared for proportions
• Correlation vs Causation: correlation does not imply causation — design experiments
CHAPTER 8
SQL for Data Scientists
SQL remains essential. Most data lives in relational databases and data warehouses (BigQuery, Redshift,
Snowflake). Master SELECT, JOINs, GROUP BY, window functions, and CTEs.
-- Top customers by revenue last 90 days
WITH recent AS (
SELECT customer_id, SUM(amount) AS total
FROM orders
WHERE order_date >= CURRENT_DATE - 90
GROUP BY customer_id
)
SELECT [Link], [Link],
RANK() OVER (ORDER BY [Link] DESC) AS rnk
FROM recent r JOIN customers c USING(customer_id)
ORDER BY rnk LIMIT 10;
CHAPTER 9
End-to-End Project Workflow
A data science project follows a structured lifecycle. Skipping steps leads to models that look good in
notebooks but fail in production.
• 1. Business Understanding: define success metric and stakeholder needs
• 2. Data Collection: databases, APIs, web scraping, data partnerships
• 3. EDA & Cleaning: understand distributions, fix quality issues
• 4. Feature Engineering: create informative inputs from raw data
• 5. Modelling: baseline → iterate → tune hyperparameters
• 6. Evaluation: cross-validation, hold-out test set, business metric
• 7. Deployment: REST API (FastAPI), monitoring, scheduled retraining