Data Analytics & Financial Applications: Notes
1. Data Sourcing
Definition: Data sourcing is the process of collecting financial data from
reliable sources for analysis and decision-making.
Importance: Accurate data is the foundation for financial analytics. Poor
data quality can result in wrong predictions and decisions.
Sources of Financial Data: 1. Stock Data APIs: Yahoo Finance, Alpha
Vantage, Quandl 2. Financial Reports: Company annual/quarterly reports 3.
News & Social Media: Twitter, Bloomberg, Reuters 4. Government Data: RBI
reports, SEBI filings 5. Market Databases: NSE/BSE datasets
Example: Fetching Data Using Yahoo Finance API
import yfinance as yf
data = [Link]('[Link]', start='2022-01-01', end='2024-12-31')
print([Link]())
2. Data Cleaning
Definition: Data cleaning is the process of removing errors, filling missing
values, and correcting formats in raw data.
Importance: Dirty data can distort analysis and result in poor decision-
making.
Steps: 1. Handle Missing Values:
data = [Link](method='ffill')
2. Remove Duplicates:
data = data.drop_duplicates()
3. Fix Formatting:
data['Date'] = pd.to_datetime(data['Date'])
data = data.sort_values(by='Date')
4. Remove Outliers: Remove extreme values that distort analysis.
3. Data Processing
Definition: Data processing transforms cleaned data into a format suitable
for analysis.
Key Steps:
3.1 Normalization/Scaling
from [Link] import MinMaxScaler
scaler = MinMaxScaler(feature_range=(0, 1))
data_scaled = scaler.fit_transform(data[['Close']])
3.2 Feature Engineering
data['MA5'] = data['Close'].rolling(window=5).mean()
data['MA20'] = data['Close'].rolling(window=20).mean()
3.3 Aggregation
monthly_avg = data['Close'].resample('M').mean()
3.4 Time-Series Preparation
X_train, y_train = [], []
for i in range(60, len(data_scaled)):
X_train.append(data_scaled[i-60:i, 0])
y_train.append(data_scaled[i, 0])
4. Data Visualization
Purpose: To convert processed data into visual formats for easier
understanding and decision-making.
Types: | Chart Type | Purpose | |————|———| | Line Chart | Shows trends
over time | | Candlestick Chart | Shows price movement | | Histogram |
Shows distribution of returns | | Scatter Plot | Shows relationships between
variables |
Examples:
Line Chart:
import [Link] as plt
[Link](figsize=(14,6))
[Link](data['Date'], data['Close'], color='blue')
[Link]('Stock Closing Price Over Time')
[Link]('Date')
[Link]('Close Price')
[Link]()
Candlestick Chart:
import plotly.graph_objects as go
fig = [Link](data=[[Link](
x=data['Date'],
open=data['Open'], high=data['High'],
low=data['Low'], close=data['Close']
)])
fig.update_layout(title='Candlestick Chart of Stock Prices')
[Link]()
5. Real-World Applications
Stock Trend Analysis: Predicting price trends.
Risk Assessment: Identifying risky assets.
Portfolio Optimization: Selecting best mix of investments.
Regulatory Reporting: Presenting clean data for compliance.
End of Notes