Data Analysis with Python – Pandas, NumPy & Visualization
Computer Science / Data Science | Study Notes | 2025
1. Introduction
Python has become the dominant language for data analysis, combining a clean syntax with powerful
libraries. The core data analysis stack consists of NumPy (numerical computing), Pandas (tabular data),
Matplotlib and Seaborn (visualization), and SciPy (scientific computing). These tools are used extensively
in quantitative finance, academic research, and industry data science.
2. NumPy Fundamentals
NumPy (Numerical Python) provides the ndarray — a fast, memory-efficient multi-dimensional array —
and vectorized mathematical operations. NumPy operations execute in compiled C code, making them
orders of magnitude faster than pure Python loops.
Key NumPy Operations
• Array creation: [Link]([1,2,3]), [Link]((3,4)), [Link]((2,2)), [Link](0,10,0.5),
[Link](0,1,100), [Link](1000).
• Array arithmetic: operations apply element-wise: a + b, a * b, a ** 2. No loops needed.
• Broadcasting: NumPy automatically expands arrays of different shapes for element-wise
operations (e.g., adding a scalar to a 2D array, or a row vector to a matrix).
• Linear algebra: [Link](A, B), [Link](A), [Link](A), [Link](A, b).
• Aggregation: [Link](), [Link](), [Link](), [Link](), [Link](), [Link](). Add axis=
parameter to aggregate along rows or columns.
3. Pandas for Data Analysis
Pandas introduces two primary data structures: Series (1D labeled array) and DataFrame (2D labeled
table). DataFrames are the workhorse of data analysis — analogous to a spreadsheet or SQL table in
Python.
Loading and Inspecting Data
• pd.read_csv('[Link]'), pd.read_excel('[Link]'), pd.read_json('[Link]').
• [Link](10), [Link](5): view first/last rows. [Link]: (rows, cols). [Link]: column data types.
• [Link](): summary statistics for numeric columns. [Link](): index, dtypes, and memory
usage.
• [Link]().sum(): count missing values per column.
Data Cleaning
• [Link](): drop rows with missing values. [Link](value): fill missing values. [Link]([Link]()):
fill with column mean.
• df.drop_duplicates(): remove duplicate rows. [Link](columns={'old': 'new'}): rename columns.
• df['col'].astype(float): change column data type. pd.to_datetime(df['date']): parse date strings.
• df['col'].[Link]().[Link](): string cleaning — remove whitespace, lowercase.
Selection and Filtering
• df['col']: select a column (returns Series). df[['col1','col2']]: select multiple columns (returns
DataFrame).
• [Link][row_label, col_label]: label-based indexing. [Link][row_int, col_int]: integer-based indexing.
• Boolean filtering: df[df['sales'] > 1000]. Multiple conditions: df[(df['sales'] > 1000) & (df['region'] ==
'South')].
Grouping and Aggregation
• [Link]('category')['sales'].sum(): total sales per category.
• [Link](['region','year']).agg({'sales': 'sum', 'profit': 'mean'}): multiple aggregations.
• df.pivot_table(values='sales', index='region', columns='year', aggfunc='sum'): Excel-style pivot
table.
4. Visualization with Matplotlib and Seaborn
Matplotlib
Matplotlib is the foundational visualization library. Every plot consists of a Figure (the canvas) and one or
more Axes (the actual plot). Common plot types:
• Line chart: [Link](x, y). Add labels: [Link](), [Link](), [Link](), [Link]().
• Bar chart: [Link](x, height). Horizontal: [Link]().
• Scatter plot: [Link](x, y, c=color, s=size, alpha=0.5).
• Histogram: [Link](data, bins=30, edgecolor='black').
• Subplots: fig, axes = [Link](2, 3, figsize=(12,8)). Access each subplot via axes[row][col].
Seaborn
Seaborn builds on Matplotlib with a higher-level API and better default aesthetics. Ideal for statistical
visualization:
• [Link](df['col'], kde=True): distribution plot with kernel density estimate.
• [Link](x='category', y='value', data=df): box plots for comparing distributions.
• [Link]([Link](), annot=True, cmap='coolwarm'): correlation heatmap — essential for
feature analysis.
• [Link](df): scatter matrix for all numeric column pairs. Quick EDA tool.
5. Applied Finance Example
Combining these tools for financial analysis:
1. Load historical price data: df = pd.read_csv('[Link]', parse_dates=['date'], index_col='date').
2. Calculate daily returns: df['returns'] = df['close'].pct_change().
3. Annualized Sharpe Ratio: sharpe = (df['returns'].mean() / df['returns'].std()) * [Link](252).
4. Rolling volatility: df['vol_30d'] = df['returns'].rolling(30).std() * [Link](252).
5. Correlation matrix of multiple assets: returns_df.corr() then visualize with [Link]().
Personal study notes — for educational use only.