Python Data Analysis

The Complete Cheatsheet for Analysts & Engineers

This cheatsheet covers the complete Python data analysis workflow — from environment setup
and raw data loading to cleaning, transformation, aggregation, visualization, and performance
optimization. Each section includes ready-to-use code snippets with inline comments. Whether
you are a beginner building your first pipeline or an experienced analyst looking for a quick
reference, this guide has you covered.

Table of Contents
# Section Topic
1 Environment Setup Installation, virtual environments, Jupyter
2 NumPy Fundamentals Arrays, operations, broadcasting
3 Pandas — Data Loading CSV, Excel, SQL, JSON, APIs
4 Pandas — Inspection & Cleaning dtypes, nulls, duplicates, transforms
5 Pandas — Filtering & Selection loc, iloc, query, boolean indexing
6 Pandas — Aggregation groupby, pivot_table, resample
7 String & DateTime Operations str accessor, date parsing, time series
8 Merging & Reshaping merge, concat, melt, stack/unstack
9 Visualization Matplotlib, Seaborn, Plotly Express
10 Performance & Best Practices Memory, vectorization, profiling
11 Quick Reference Card Most-used one-liners at a glance
1. Environment Setup
A reproducible environment is the foundation of any data project. Python's ecosystem has
several tools for managing dependencies and isolating project environments. The most widely
used approach for data work is a combination of pip + virtualenv or conda.

1.1 Installing Core Libraries

# Create and activate a virtual environment
python -m venv .venv
source .venv/bin/activate # Linux / macOS
.venv\Scripts\activate # Windows
# Install the core data stack
pip install pandas numpy matplotlib seaborn plotly
pip install openpyxl xlrd # Excel support
pip install sqlalchemy psycopg2 # SQL databases
pip install requests # HTTP / API calls
pip install jupyter jupyterlab # Notebooks
pip install scikit-learn # Machine learning

1.2 Jupyter Notebook Tips

Jupyter is the standard environment for exploratory data analysis. Use these shortcuts to work
faster:

Shortcut Action
Shift + Enter Run current cell and move to next
Ctrl + Enter Run current cell, stay
A/B Insert cell above / below
DD Delete current cell
M/Y Switch cell to Markdown / Code
Tab Autocomplete
Shift + Tab Show function docstring
2. NumPy Fundamentals
NumPy is the backbone of numerical computing in Python. Pandas is built on top of it.
Understanding NumPy arrays helps you write faster, more memory-efficient code.

2.1 Creating Arrays

import numpy as np
a = [Link]([1, 2, 3, 4, 5]) # 1D array
b = [Link]([[1, 2], [3, 4]]) # 2D array
[Link]((3, 4)) # 3x4 array of zeros
[Link]((2, 3)) # 2x3 array of ones
[Link](0, 10, 2) # [0, 2, 4, 6, 8]
[Link](0, 1, 5) # 5 evenly spaced values
[Link](42)
[Link](3, 3) # uniform [0,1]
[Link](3, 3) # standard normal

2.2 Array Operations & Broadcasting

a = [Link]([1, 2, 3])
b = [Link]([4, 5, 6])
a + b # element-wise: [5, 7, 9]
a * b # element-wise: [4, 10, 18]
a ** 2 # [1, 4, 9]
[Link](a, b) # dot product: 32
[Link](), [Link]() # statistics
[Link](3, 1) # change shape
a[a > 1] # boolean mask: [2, 3]
[Link](a > 1, a, 0) # conditional: [0, 2, 3]

2.3 Useful NumPy Functions

[Link](a) # sort ascending
[Link](a) # indices that would sort a
[Link](a) # unique values
[Link]([a, b]) # join arrays
[Link]([a, b]) # stack vertically
[Link]([a, b]) # stack horizontally
[Link](a, 1, 4) # clamp values between 1 and 4
[Link](a), [Link](a) # element-wise log / exp
[Link](a).any() # check for NaN
np.nan_to_num(a) # replace NaN with 0
3. Pandas — Data Loading
Pandas can read data from almost any source. Below are the most common formats with their
key parameters. Always check the shape and dtypes after loading.

3.1 CSV and Text Files

import pandas as pd
df = pd.read_csv('[Link]') # basic
df = pd.read_csv('[Link]', sep=';') # semicolon delimiter
df = pd.read_csv('[Link]', encoding='utf-8')
df = pd.read_csv('[Link]', nrows=1000) # first 1000 rows
df = pd.read_csv('[Link]', skiprows=2) # skip header rows
df = pd.read_csv('[Link]', usecols=['a','b']) # select columns
df = pd.read_csv('[Link]', parse_dates=['date'])
df = pd.read_csv('[Link]', index_col='id') # set index
df = pd.read_csv('[Link]', dtype={'age': int}) # enforce dtype

3.2 Excel Files

df = pd.read_excel('[Link]') # first sheet
df = pd.read_excel('[Link]', sheet_name='Q1') # by name
df = pd.read_excel('[Link]', sheet_name=0) # by index
sheets = pd.read_excel('[Link]', sheet_name=None) # all sheets → dict
# Write back to Excel
df.to_excel('[Link]', index=False)
with [Link]('[Link]') as writer:
df1.to_excel(writer, sheet_name='Sheet1')
df2.to_excel(writer, sheet_name='Sheet2')

3.3 SQL Databases

from sqlalchemy import create_engine
engine = create_engine('postgresql://user:pass@host:5432/db')
df = pd.read_sql('SELECT * FROM sales LIMIT 100', engine)
df = pd.read_sql_table('customers', engine)
df = pd.read_sql_query('SELECT id, name FROM users WHERE active=1', engine)
# Write DataFrame to SQL
df.to_sql('table_name', engine, if_exists='replace', index=False)
df.to_sql('table_name', engine, if_exists='append', index=False)

3.4 JSON and APIs

df = pd.read_json('[Link]')
df = pd.read_json('[Link] # URL directly
# From REST API using requests
import requests
resp = [Link]('[Link] headers={'Authorization':
'Bearer TOKEN'})
data = [Link]()
df = [Link](data['results']) # adjust key as needed
4. Pandas — Inspection & Cleaning
4.1 Inspecting Your DataFrame
[Link] # (rows, columns)
[Link] # column data types
[Link]() # dtypes + non-null counts + memory
[Link]() # stats: mean, std, min, max, quartiles
[Link](include='all') # include non-numeric columns
[Link](10) # first 10 rows
[Link](5) # last 5 rows
[Link](5) # 5 random rows
[Link]() # list column names
[Link] # inspect index
df.memory_usage(deep=True) # memory per column in bytes

4.2 Handling Missing Values

[Link]().sum() # NaN count per column
[Link]().mean() * 100 # % missing per column
[Link]() # drop rows with any NaN
[Link](subset=['col1', 'col2']) # only check specific cols
[Link](thresh=5) # keep rows with >= 5 non-NaN
[Link](0) # fill all NaN with 0
[Link]({'col1': 0, 'col2': 'N/A'}) # per-column fill values
df['col'].fillna(df['col'].mean()) # fill with mean
df['col'].fillna(method='ffill') # forward fill
df['col'].fillna(method='bfill') # backward fill
df['col'].interpolate() # linear interpolation

4.3 Removing Duplicates & Fixing Types

[Link]().sum() # count duplicate rows
df.drop_duplicates() # remove all duplicate rows
df.drop_duplicates(subset=['email']) # deduplicate on one column
df.drop_duplicates(keep='last') # keep last occurrence
df['age'] = df['age'].astype(int)
df['price'] = df['price'].astype(float)
df['active'] = df['active'].astype(bool)
df['date'] = pd.to_datetime(df['date'])
df['category'] = df['category'].astype('category') # memory efficient

4.4 Renaming, Reordering, and Dropping

[Link](columns={'old_name': 'new_name'}) # rename columns
[Link] = [[Link]().replace(' ', '_') for c in [Link]] # normalize
[Link](columns=['col1', 'col2']) # drop columns
[Link](index=[0, 1, 2]) # drop rows by index
df[['col3', 'col1', 'col2']] # reorder columns
df.reset_index(drop=True) # reset row index
df.set_index('id') # set column as index
5. Pandas — Filtering & Selection
5.1 Boolean Indexing
df[df['age'] > 30]
df[df['city'] == 'Jakarta']
df[(df['age'] > 25) & (df['city'] == 'Jakarta')] # AND
df[(df['age'] < 20) | (df['age'] > 60)] # OR
df[~df['active']] # NOT
df[df['name'].[Link]('Ali')]
df[df['category'].isin(['A', 'B', 'C'])]
df[df['score'].between(60, 90)]
df[df['value'].notna()] # exclude NaN

5.2 loc and iloc

# loc: label-based selection
[Link][0] # row with label 0
[Link][0:5, ['name', 'age']] # rows 0-5, two columns
[Link][df['age'] > 30, 'name'] # filtered rows, one column
# iloc: integer position-based selection
[Link][0] # first row
[Link][-1] # last row
[Link][0:5, 0:3] # first 5 rows, first 3 columns
[Link][[0, 2, 4], :] # rows 0, 2, 4 — all columns

5.3 Query Syntax

[Link]('age > 30')
[Link]('age > 30 and city == "Jakarta"')
[Link]('[Link](60, 90)', engine='python')
# Using external variables with @
min_age = 25
[Link]('age > @min_age')
6. Pandas — Aggregation
6.1 GroupBy
# Single aggregation
[Link]('category')['sales'].sum()
[Link]('category')['sales'].mean()
[Link]('category')['sales'].agg(['sum', 'mean', 'count'])
# Multiple columns
[Link](['region', 'category'])['sales'].sum()
# Multiple aggregations on multiple columns
[Link]('region').agg(
total_sales=('sales', 'sum'),
avg_qty=('qty', 'mean'),
num_orders=('order_id', 'count')
)
# Custom aggregation function
[Link]('category')['price'].agg(lambda x: [Link]() - [Link]())

6.2 Pivot Tables

pd.pivot_table(
df,
values='sales',
index='region',
columns='category',
aggfunc='sum',
fill_value=0
)
# Cross-tabulation (counts)
[Link](df['region'], df['category'])
[Link](df['region'], df['category'], normalize='index') # row %

6.3 Time Series Resampling

df = df.set_index('date') # date column must be datetime
[Link]('D').sum() # daily
[Link]('W').mean() # weekly
[Link]('M').agg({'sales': 'sum', 'visits': 'mean'})
[Link]('Q').sum() # quarterly
[Link]('Y').sum() # yearly
7. String & DateTime Operations
7.1 String Operations (str accessor)
df['name'].[Link]()
df['name'].[Link]()
df['name'].[Link]() # remove whitespace
df['name'].[Link]('old', 'new')
df['name'].[Link]('pattern') # boolean mask
df['name'].[Link]('A')
df['name'].[Link]('z')
df['name'].[Link]() # string length
df['name'].[Link](' ', expand=True) # split into columns
df['name'].[Link](r'(\d+)') # regex extract
df['email'].[Link]('@').str[1] # get domain part

7.2 DateTime Operations (dt accessor)

df['date'] = pd.to_datetime(df['date'])
df['date'].[Link]
df['date'].[Link]
df['date'].[Link]
df['date'].[Link]
df['date'].[Link] # 0=Monday, 6=Sunday
df['date'].dt.is_weekend # custom — use dayofweek >= 5
df['date'].[Link]('%Y-%m') # format to string
# Date arithmetic
df['date'] + [Link](days=7)
(df['end_date'] - df['start_date']).[Link] # difference in days
# Filtering by date
df[df['date'] >= '2024-01-01']
df[df['date'].[Link] == 3] # March only
8. Merging & Reshaping
8.1 Merging DataFrames
# Equivalent to SQL JOIN
[Link](df1, df2, on='id') # INNER JOIN
[Link](df1, df2, on='id', how='left') # LEFT JOIN
[Link](df1, df2, on='id', how='right') # RIGHT JOIN
[Link](df1, df2, on='id', how='outer') # FULL OUTER JOIN
[Link](df1, df2, left_on='user_id', right_on='id') # different key names
# Concatenating
[Link]([df1, df2]) # stack rows
[Link]([df1, df2], axis=1) # stack columns
[Link]([df1, df2], ignore_index=True) # reset index

8.2 Reshaping with melt and pivot

# Wide to long (unpivot)
[Link](df, id_vars=['id', 'name'], value_vars=['Q1', 'Q2', 'Q3', 'Q4'],
var_name='quarter', value_name='sales')
# Long to wide (pivot)
[Link](index='date', columns='product', values='sales')
# Stack / Unstack (for MultiIndex)
[Link]() # columns → rows
[Link]() # rows → columns
9. Visualization
Python has three major visualization libraries, each suited to different use cases. Matplotlib is
the base layer with the most control; Seaborn adds statistical aesthetics on top; Plotly Express
creates interactive charts with minimal code.

9.1 Matplotlib
import [Link] as plt
fig, axes = [Link](1, 2, figsize=(12, 5))
# Bar chart
axes[0].bar(df['category'], df['sales'], color='steelblue')
axes[0].set_title('Sales by Category')
axes[0].set_xlabel('Category')
axes[0].set_ylabel('Sales')
# Line chart
axes[1].plot(df['date'], df['value'], marker='o', linewidth=2)
axes[1].set_title('Value Over Time')
plt.tight_layout()
[Link]('[Link]', dpi=150, bbox_inches='tight')
[Link]()

9.2 Seaborn
import seaborn as sns
sns.set_theme(style='whitegrid')
[Link](df['value'], bins=30, kde=True) # histogram + density
[Link](x='category', y='sales', data=df) # box plot
[Link](x='age', y='income', hue='city', data=df)
[Link]([Link](), annot=True, cmap='coolwarm') # correlation matrix
[Link](df[['col1','col2','col3']]) # pairwise plots
[Link](x='category', data=df,
order=df['category'].value_counts().index)

9.3 Plotly Express (Interactive)

import [Link] as px
fig = [Link](df, x='category', y='sales', color='region', title='Sales by
Category')
fig = [Link](df, x='date', y='value', color='product')
fig = [Link](df, x='age', y='income', size='score', hover_data=['name'])
fig = [Link](df, values='sales', names='category')
fig = [Link](df, locations='country', color='gdp', projection='natural
earth')
fig.update_layout(template='plotly_white', font_size=13)
[Link]()
fig.write_html('[Link]') # export as interactive HTML
10. Performance & Best Practices
10.1 Reducing Memory Usage
# Check memory usage
df.memory_usage(deep=True).sum() / 1024**2 # in MB
# Downcast numeric types
df['int_col'] = pd.to_numeric(df['int_col'], downcast='integer')
df['float_col'] = pd.to_numeric(df['float_col'], downcast='float')
# Use category for low-cardinality strings
df['status'] = df['status'].astype('category')
# Read only needed columns
df = pd.read_csv('[Link]', usecols=['id', 'date', 'sales'])
# Read in chunks for large files
chunks = pd.read_csv('[Link]', chunksize=50000)
df = [Link]([chunk[chunk['sales'] > 0] for chunk in chunks])

10.2 Vectorization vs Loops

Avoid Python for-loops over DataFrame rows. Use vectorized operations, apply(), or NumPy
where possible. The performance difference can be 10x–1000x.

# SLOW — never do this

for i, row in [Link]():
[Link][i, 'tax'] = row['price'] * 0.11
# FAST — vectorized
df['tax'] = df['price'] * 0.11
# For complex logic, use [Link] or [Link]
df['tier'] = [Link](df['score'] >= 90, 'Gold', 'Silver')
conditions = [df['score'] >= 90, df['score'] >= 70, df['score'] >= 50]
choices = ['Gold', 'Silver', 'Bronze']
df['tier'] = [Link](conditions, choices, default='None')

10.3 Profiling & Timing

# Time a single line (Jupyter)
%timeit df['col'].apply(lambda x: x * 2)
# Profile a block
%%time
result = [Link]('category').agg({'sales': 'sum'})
# Line profiler (install line_profiler)
# %load_ext line_profiler
# %lprun -f my_function my_function(df)
11. Quick Reference Card
The most-used one-liners — bookmark this page.

Task One-liner

Load CSV pd.read_csv('[Link]')

Shape [Link]

Column list [Link]()

dtypes [Link]

Missing counts [Link]().sum()

Describe [Link]()

Sample rows [Link](5)

Filter rows df[df['col'] > val]

Select cols df[['a', 'b']]

Drop column [Link](columns=['c'])

Rename col [Link](columns={'a': 'b'})

Fill NaN [Link](0)

Drop NaN rows [Link]()

Remove dupes df.drop_duplicates()

Sort df.sort_values('col', ascending=False)

GroupBy sum [Link]('col')['val'].sum()

Value counts df['col'].value_counts()

String contains df[df['col'].[Link]('x')]

Parse date pd.to_datetime(df['date'])

Merge [Link](df1, df2, on='id')

Concat rows [Link]([df1, df2])

Pivot table pd.pivot_table(df, values='v', index='i', columns='c', aggfunc='sum')

Apply function df['col'].apply(lambda x: x*2)

Correlation matrix [Link]()

Export CSV df.to_csv('[Link]', index=False)

Export Excel df.to_excel('[Link]', index=False)

Python Data Analysis Cheatsheet — v2.0 — For educational use