QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

QUANT
PRO
Advanced Quant Finance Course
WEEK 5 — Days 31 to 39

9 3 1 LIVE
DAYS NEW LIBS PROJECT SYSTEM

Statistics · Implied Volatility · GARCH · Live Data Pipeline · Transaction Costs · Walk-Forward Testing ·
Project 3

You completed Weeks 1-4. Week 5 answers that question.

You have 10 quant models, a backtested Every model gets statistically validated.
strategy, and a GitHub portfolio. Now you are Every backtest gets realistic costs. Every
asking: "But does any of this actually work signal gets live data. Every result gets proven
with real money?" on unseen data.

Quant Pro | Advanced Quant Finance Course | Week 5 Page 1

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

WEEK 5 AT A GLANCE
Week 5 is the bridge between a student project and a professional tool. You built 10 models in Weeks 1-4.
This week you make them work in the real world — with statistics to prove they are real, live data that
refreshes automatically, realistic costs applied to every trade, and validation on data the strategy never
saw.

Day Topic Key Skill Project Output

31 Why Strategies Fail — ADX filter, whipsaw, regime Regime-filtered vs raw

Regimes detection backtest

32 The Math Bridge Normal dist, log returns, Visual explainers with real
regression intuition examples

33 Statistical Validation p-values, Sharpe t-stat, Full stats report on your

bootstrap CI strategy

34 Implied Vol + Vol Surface IV extraction, vol smile, IVR Vol smile chart for NIFTY
options

35 GARCH — Volatility That GARCH(1,1), persistence, GARCH vol vs rolling std

Thinks dynamic VaR comparison

36 Live Data Pipeline yfinance, schedule, error Auto-download + signal +

handling, logs alert daily

37 Transaction Cost Modelling STT, slippage, bid-ask, net PnL Gross vs net vs stress PnL
chart

38 Walk-Forward Testing WFE, OOS validation, Robustness report on your

permutation test strategy

39 Project 3 — Live Dashboard Combine all Week 5 skills into GitHub: live dashboard +
one system README

What Week 4 Showed You What Week 5 Adds

Backtests with impressive-looking returns Statistical proof returns are real, not luck

ML model with "59% accuracy" Proper time-series CV so 59% actually means 59%

Bought/sold at exact close price STT + slippage + bid-ask — real net PnL

pd.read_csv("[Link]") — file missing yfinance pipeline — live data, no CSV ever

Best SMA from grid search on same data Walk-forward validation on unseen data only

Rolling std dev as volatility proxy GARCH — dynamic vol that updates every day

Quant Pro | Advanced Quant Finance Course | Week 5 Page 2

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

DAY

Why Strategies Really Fail

31 Market regimes, whipsaw, and the ADX filter that fixes both

The Problem Nobody Told You About in Week 3

Your Week 3 SMA crossover strategy looked great on the backtest chart. Smooth equity curve, positive
Sharpe ratio, decent win rate. But there is a silent assumption buried in every backtest you have ever run:
the market behaved the same way throughout the entire test period. It did not. Markets cycle through
completely different regimes, and a strategy built for one regime fails badly in another.

> REAL-WORLD ANALOGY

Think of a cricket batsman who averages 60 runs on flat pitches in Mumbai but averages 12 on turning pitches
in Ahmedabad. His overall average looks decent. But if you send him to Ahmedabad without understanding
this — you lose the match. A trading strategy that crushes a trending market and destroys itself in a sideways
market has exactly the same problem. You need to know which pitch you are on before you play.

The Three Market Regimes

Every market is in one of three states at any given time. Most strategies are built for one specific regime
and fail in the others. Understanding which regime you are in is one of the most valuable skills in all of
quant finance.

THREE MARKET REGIMES

TRENDING (Bull or Bear) SIDEWAYS (Choppy) VOLATILE (Crisis)

■■■■■■■■■■■■■■■■■■■■■ ■■■■■■■■■■■■■■■■■ ■■■■■■■■■■■■■■■■■
Price moves clearly in Price oscillates up Wild swings both
one direction for weeks and down in a tight ways, no clear
or months at a time range for months direction at all

TCS Jan-Mar 2023: +18% NIFTY Aug-Oct 2023: COVID Mar 2020:
Clear uptrend arrow-up Flat between 19000-19500 -38% then +40%

Best strategy: Best strategy: Best strategy:

SMA crossover, Mean reversion, Reduce position
momentum signals RSI buy/sell size, sell vol

SMA crossover in trending: YES SMA crossover in sideways: NO (whipsaw!)

What is Whipsaw?
Whipsaw is what kills trend-following strategies in sideways markets. The price crosses your moving
average, triggers a buy signal, then immediately reverses back — triggering a sell signal. This happens five

Quant Pro | Advanced Quant Finance Course | Week 5 Page 3

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

times in two weeks, each time costing you transaction costs and a small loss. The strategy bleeds to death
while the market goes nowhere.

WHIPSAW IN A SIDEWAYS MARKET — THE DEATH OF TREND-FOLLOWING

Market: 100 -> 102 -> 99 -> 103 -> 98 -> 101 -> 97 -> 102
SMA(20) stays near 100 throughout — no clear direction

Day 1: Price crosses above SMA -> BUY at 102 Entry: 102
Day 3: Price crosses below SMA -> SELL at 99 Loss: -3
Day 5: Price crosses above SMA -> BUY at 103 Entry: 103
Day 7: Price crosses below SMA -> SELL at 98 Loss: -5
Day 9: Price crosses above SMA -> BUY at 101 Entry: 101
Day 11:Price crosses below SMA -> SELL at 97 Loss: -4

Total: 6 trades, -12 points, plus all transaction costs

The market went exactly nowhere. The strategy lost money.
This is whipsaw. This is why the ADX filter exists.

The Fix — ADX Filter (Average Directional Index)

ADX measures the strength of a trend — not the direction, just how strong any trend is. ADX above 25
means a real trend is in place. ADX below 20 means the market is going sideways — choppy, no direction.
The fix is simple: only take SMA crossover signals when ADX is above 25. In sideways markets, step aside
and do nothing.

ADX QUICK REFERENCE

ADX 0-20 -> No trend. Market is sideways. STAY OUT completely.

ADX 20-25 -> Developing trend. Proceed with caution.
ADX > 25 -> Real trend confirmed. SMA crossover signals are valid.
ADX > 40 -> Very strong trend. Often near exhaustion — be careful.

IMPORTANT: ADX does NOT tell you direction (up or down).

The +DI and -DI lines tell you direction.
ADX only measures STRENGTH. A high ADX in a downtrend is still high ADX.

Python Code — Day 31: ADX Filter

import pandas as pd
import numpy as np
import [Link] as plt

df = pd.read_csv("[Link]")
df = df[df["SYMBOL"] == "TCS"].sort_values("DATE1").reset_index(drop=True)
df["DATE1"] = pd.to_datetime(df["DATE1"])

# Calculate ADX from scratch

def calculate_adx(df, period=14):
high, low, close = df["HIGH"], df["LOW"], df["CLOSE"]
prev_close = [Link](1)

Quant Pro | Advanced Quant Finance Course | Week 5 Page 4

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

tr = [Link]([high-low, (high-prev_close).abs(),
(low-prev_close).abs()], axis=1).max(axis=1)
up_move = high - [Link](1)
down_move = [Link](1) - low
plus_dm = [Link]((up_move > down_move) & (up_move > 0), up_move, 0)
minus_dm = [Link]((down_move > up_move) & (down_move > 0), down_move, 0)
atr = [Link](tr).ewm(alpha=1/period, adjust=False).mean()
plus_di = 100*[Link](plus_dm).ewm(alpha=1/period,adjust=False).mean()/atr
minus_di= 100*[Link](minus_dm).ewm(alpha=1/period,adjust=False).mean()/atr
dx = 100 * (plus_di - minus_di).abs() / (plus_di + minus_di)
adx = [Link](alpha=1/period, adjust=False).mean()
return adx

df["ADX"] = calculate_adx(df)
df["SMA_20"] = df["CLOSE"].rolling(20).mean()
df["SMA_50"] = df["CLOSE"].rolling(50).mean()

# Raw signal vs ADX-filtered signal

df["Raw_Sig"] = (df["SMA_20"] > df["SMA_50"]).astype(int)
df["Filt_Sig"] = [Link](
(df["SMA_20"] > df["SMA_50"]) & (df["ADX"] > 25), 1, 0)

# Simulate both
def simulate(df, sig_col, cap=100_000):
df = [Link]()
df["Pos"] = df[sig_col].shift(1).diff()
c, sh = cap, 0
eq = []
for _, row in [Link]():
if row["Pos"]==1 and sh==0:
en=row["CLOSE"]; sh=int(c/en); c-=sh*en
elif row["Pos"]==-1 and sh>0:
c+=sh*row["CLOSE"]; sh=0
[Link](c + sh*row["CLOSE"])
n = (df["Pos"].abs()>0).sum()//2
ret = (eq[-1]/cap - 1)*100
return [Link](eq), ret, n

eq_raw, r_raw, n_raw = simulate(df, "Raw_Sig")

eq_filt, r_filt, n_filt = simulate(df, "Filt_Sig")

print(f"Raw : {r_raw:+.1f}% | {n_raw} trades (whipsaw included)")

print(f"Filtered: {r_filt:+.1f}% | {n_filt} trades (only in trends)")
print("Fewer trades + less whipsaw = better risk-adjusted return")

Day 31 Key Takeaways

• Every market cycles through three regimes: trending, sideways, volatile. No single strategy wins in all
three.
• Whipsaw bleeds trend-following strategies dry in sideways markets — repeated small losses from false
signals.

Quant Pro | Advanced Quant Finance Course | Week 5 Page 5

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

• ADX above 25 confirms a real trend exists. Below 20 means the market is going nowhere — do not
trade.
• Adding an ADX filter typically cuts trade count by 30-50% but removes most losing low-quality signals.
• From today forward: always ask what regime your strategy was tested in before trusting any backtest
result.

Quant Pro | Advanced Quant Finance Course | Week 5 Page 6

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

DAY

The Math Bridge

32 Normal distribution, log returns, regression — the building blocks of every quant model

Why This Day Exists

Week 4 used advanced formulas — Black-Scholes, Monte Carlo GBM, factor regression — without fully
explaining the mathematical building blocks underneath. This day fills that gap. No calculus required. Pure
intuition with real stock market examples. By the end you will understand why the formulas in Week 4 look
the way they do.

1. The Normal Distribution — The Bell Curve

A distribution is simply the answer to: if you collected every value of something and drew a bar chart of how
often each value occurs — what shape would it make? The Normal Distribution makes a bell shape. Most
values cluster near the centre. Values far from the centre become increasingly rare.

THE BELL CURVE OF STOCK RETURNS

Frequency
| xxxxxxxxxx
| xxxxxxxxxxxxxxxx
| xxxxxxxxxxxxxxxxxxxxxxxx
| xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
| xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|______________________________________________
-3% -2% -1% 0% +1% +2% +3%
^
Most common: small moves near zero

Reading this for TCS (daily vol = 1.2%, mean = +0.05%):

68% of days: move between -1.15% and +1.25% (1 sigma)
95% of days: move between -2.35% and +2.45% (2 sigma)
99.7% of days: move between -3.55% and +3.65% (3 sigma)

68-95-99.7 RULE: memorise this. You will use it every week.

VaR(95%) = mean - 1.645 x sigma comes directly from this rule.

* KEY INSIGHT
The 68-95-99.7 rule is the single most useful fact in statistics for traders. It directly gives you what "normal"
daily moves look like. A -4% day on TCS is a 3+ sigma event — happens roughly once every 2-3 years under
normal conditions. When you see it happen more often, that tells you something is wrong with the normal
assumption — fat tails are present.

Quant Pro | Advanced Quant Finance Course | Week 5 Page 7

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

2. Real Returns Are NOT Perfectly Normal

Here is the uncomfortable truth that caused the 2008 financial crisis, the LTCM collapse, and countless
blown-up strategies. Real stock returns have two critical differences from a perfect bell curve:

NORMAL vs REAL STOCK RETURNS — THE TWO KEY DIFFERENCES

Normal bell curve: Real stock returns:

xxxxxx xxxxxx
xxxxxxxxxx xxxxxxxxxx
xxxxxxxxxxxxxx vs xxxxxxxxxxxxxx
xxxxxxxxxxxxxxxx x xxxxxxxxxxxxxxxx x
Tails fall off FAT TAILS: crashes happen
smoothly more often than predicted

DIFFERENCE 1: FAT TAILS (Excess Kurtosis greater than 0)

Extreme moves (crashes, spikes) occur MORE frequently than
the bell curve predicts. COVID crash was a 20-sigma event
under normal assumptions — theoretically impossible. It happened.

DIFFERENCE 2: NEGATIVE SKEW

The left tail (big losses) is LONGER than the right tail.
Crashes are bigger and faster than rallies.
"Stairs up, elevator down" — this is the skew you are seeing.

Why it matters: parametric VaR and Black-Scholes both assume

perfect normality. Both underestimate tail risk as a result.

3. Simple Returns vs Log Returns

You learned simple returns in Week 1. Quant models usually use log returns. Here is exactly why — and
when to use each.

SIMPLE vs LOG RETURNS

Simple: r = (Price_today - Price_yesterday) / Price_yesterday

Log: r = ln(Price_today / Price_yesterday)

Example: TCS yesterday Rs 3500, today Rs 3570

Simple return = (3570-3500)/3500 = 2.00%
Log return = ln(3570/3500) = 1.98% (very close for small moves)

WHY LOG RETURNS ARE USED IN QUANT MODELS:

1. They are ADDITIVE over time

Simple: multiply returns (1.02 x 0.97 x 1.03 x ...) messy
Log: just add them (+2% + -3% + +3% = +2%) clean

2. Stock prices cannot go below zero

log(price) is always well-defined. This is why GBM

Quant Pro | Advanced Quant Finance Course | Week 5 Page 8

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

(Monte Carlo from Day 24) models log prices, not raw prices.

3. Log returns are approximately normally distributed

Making them compatible with most statistical models.

RULE: Use simple returns for reporting and strategy comparison.

Use log returns inside mathematical models and formulas.

4. Linear Regression in 90 Seconds

Regression appears in beta calculation, factor models, and pairs trading hedge ratios. The core idea is
simple: find the best straight line through a cloud of data points.

REGRESSION — FITTING THE BEST LINE

NIFTY daily return (x) vs TCS daily return (y):

TCS% +4 | *
+3| * *
+2| * * * <- line of best fit
+1| * *
0| * *
-1| * *
-2| *
■■■■■■■■■■■■■■■■■■■■■■
-2 -1 0 +1 +2 NIFTY%

Slope of the line = BETA

Beta = 1.2 means: NIFTY rises 1% -> TCS typically rises 1.2%
Intercept = ALPHA (return not explained by market movement)

Python one-liner:
from [Link] import linregress
slope, intercept, r, p, se = linregress(nifty_ret, tcs_ret)
# slope = beta, intercept = alpha, r**2 = R-squared

Day 32 Key Takeaways

• The bell curve is the baseline model for returns. 68-95-99.7 rule: memorise it and use it every week.
• Real returns have fat tails (crashes more common) and negative skew (losses bigger than gains).
Always check for these.
• Log returns are additive over time and work inside mathematical models. Simple returns are for human
communication.
• Linear regression finds the best-fit line through data — giving you beta, alpha, and R-squared directly.
• Every advanced model in Weeks 6-10 builds on these four concepts. Understanding them here pays
dividends throughout.

Quant Pro | Advanced Quant Finance Course | Week 5 Page 9

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

DAY

Statistical Validation

33 Is your strategy real or did you just get lucky? The numbers that tell the truth

The Hardest Question in Quant Finance

Every backtest produces a number. Return +18%. Sharpe 1.4. Win rate 62%. These look convincing. But
they answer the wrong question. The right question is: could I have gotten this exact result by random
chance, even if my strategy has zero genuine edge? Statistical validation is the only honest answer.

> REAL-WORLD ANALOGY

You flip a coin 10 times and get 7 heads. Is the coin biased? The probability of getting 7+ heads from a fair
coin is about 17% — so there is a 17% chance this was pure luck. Not convincing. Now flip 1000 times and get
700 heads. That probability is essentially zero. That is statistically significant. The same logic applies to every
backtest result. Ten winning trades means almost nothing. One hundred winning trades starts to mean
something real.

The p-value — Your Lie Detector

A p-value answers: "If my strategy had zero real edge, what is the probability I would see a result this good
just by random chance?" The lower the p-value, the less likely the result is just luck.

p-value INTERPRETATION

p > 0.10 -> Weak evidence. Very plausibly just luck.

p = 0.05 -> The standard threshold. Borderline.
p < 0.05 -> Significant. Less than 5% chance of luck.
p < 0.01 -> Strong. Less than 1% chance of luck.

Applied to backtest returns:

"65% win rate over 8 trades" -> p = 0.29 NOT significant
"58% win rate over 80 trades" -> p < 0.05 SIGNIFICANT
"54% win rate over 300 trades"-> p < 0.01 VERY SIGNIFICANT

Same win rate. Completely different meaning depending on sample size.

The Sharpe t-statistic

Sharpe t-statistic = Sharpe Ratio x sqrt(Number of Trades)
Need: t-statistic > 1.96 for 95% significance

Sharpe = 1.5, Trades = 6 -> t = 1.5 x 2.45 = 3.67 (looks ok...)

...but 6 trades is statistically meaningless regardless. Reject.

Sharpe = 0.9, Trades = 80 -> t = 0.9 x 8.94 = 8.05 significant

Quant Pro | Advanced Quant Finance Course | Week 5 Page 10

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

Sharpe = 1.1, Trades = 30 -> t = 1.1 x 5.48 = 6.02 significant

Rule: Need at least 30 trades for any validity. 100+ is ideal.

Python Code — Day 33: Complete Statistical Report

import pandas as pd
import numpy as np
from scipy import stats

trades_df = pd.read_csv("[Link]")
returns = trades_df["Return_Pct"].values / 100

n = len(returns)
mean_r = [Link](returns)
std_r = [Link](returns)
skew = [Link](returns)
kurt = [Link](returns)

print("=" * 42)
print(" STRATEGY STATISTICAL REPORT")
print("=" * 42)
print(f" Trades : {n}")
print(f" Mean Return : {mean_r*100:.2f}%")
print(f" Std Deviation : {std_r*100:.2f}%")
print(f" Skewness : {skew:.2f} (neg=bad tail risk)")
print(f" Kurtosis : {kurt:.2f} (pos=fat tails)")

# Is mean return significantly different from zero?

t_stat, p_value = stats.ttest_1samp(returns, 0)
print(f" t-statistic : {t_stat:.2f}")
print(f" p-value : {p_value:.4f}")
print(f" Significant? : {'YES' if p_value < 0.05 else 'NO'}")

# Sharpe t-statistic
rf = 0.065 / 252
sharpe = (mean_r - rf) / std_r * [Link](252)
sr_t = sharpe * [Link](n)
print(f" Sharpe Ratio : {sharpe:.2f}")
print(f" Sharpe t-stat : {sr_t:.2f} (need >1.96)")

# 95% Confidence Interval

se = std_r / [Link](n)
ci_lo = (mean_r - 1.96*se)*100
ci_hi = (mean_r + 1.96*se)*100
print(f" 95% CI : [{ci_lo:.2f}%, {ci_hi:.2f}%]")
if ci_lo < 0:
print(" WARNING: CI includes 0 — may have no real edge")

# Bootstrap Sharpe confidence interval

boot = [[Link]([Link](returns,n,replace=True)) /
[Link]([Link](returns,n,replace=True)) * [Link](252)
for _ in range(1000)]
print(f" Bootstrap Sharpe 95% CI: [{[Link](boot,2.5):.2f},",

Quant Pro | Advanced Quant Finance Course | Week 5 Page 11

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

f"{[Link](boot,97.5):.2f}]")

# Normality test
_, p_norm = [Link](returns)
print(f" Shapiro-Wilk p : {p_norm:.4f}")
print(" Normal?" , "YES" if p_norm > 0.05 else "NO — use historical VaR")

# GOLDEN RULE
From this day forward, every backtest result you present must include: number of trades, p-value, Sharpe
t-statistic, and bootstrap confidence interval. Without these four numbers, no result — including your own —
can be trusted.

Day 33 Key Takeaways

• p-value < 0.05 means the result is statistically significant — less than 5% chance it is random luck.
• Always report trade count alongside every metric. A Sharpe of 2.0 from 5 trades is completely
meaningless.
• Bootstrap CI shows the true uncertainty in your Sharpe Ratio — a wide range from -0.3 to 2.1 should
scare you.
• Negative skewness and high kurtosis warn that real losses will be larger than your normal-distribution
model predicts.
• If the 95% CI for mean return includes zero, your strategy may have no genuine edge — be very
cautious.

Quant Pro | Advanced Quant Finance Course | Week 5 Page 12

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

DAY

Implied Volatility and the Vol Surface

34 What the options market is telling you about the future — if you know how to listen

Two Kinds of Volatility: Backward vs Forward

In Week 2 you calculated Historical Volatility — the rolling standard deviation of past returns. It looks
backward. Implied Volatility (IV) is completely different. It looks forward. It is the volatility that option buyers
and sellers are agreeing on today, embedded in option prices, reflecting their expectation of future risk.

> REAL-WORLD ANALOGY

Two weather forecasters. The first checks last month's temperature records: "average was 34 degrees — that
is historical vol." The second studies satellite imagery and monsoon patterns: "next week will hit 42 degrees —
unusual storm coming." That second forecast is implied volatility. The market's best forward-looking estimate
of uncertainty, right now, in today's option prices. When an RBI meeting is next week, IV spikes — even before
anything has happened.

How IV is Extracted — Reverse Black-Scholes

On Day 22 you used Black-Scholes forward: give it sigma, get a price. IV runs it backward: give it the
market price, find the sigma that makes Black-Scholes produce that exact price. This is done numerically
using a root-finding algorithm called Brent's method.

FORWARD Black-Scholes:
Input: S=22000, K=22500, T=30d, r=6.5%, sigma=18%
Output: Call price = Rs 312

REVERSE (Implied Volatility):

Input: S=22000, K=22500, T=30d, r=6.5%, Market price=Rs 380
Question: What sigma makes Black-Scholes give Rs 380?
Answer: sigma = 22.4% <-- This is the Implied Volatility

Why Rs 380 instead of Rs 312?

RBI policy meeting in 5 days. Options market is pricing in
MORE uncertainty than history alone suggests. IV captures this.

The Volatility Smile — Why Black-Scholes Is Incomplete

If Black-Scholes were perfect, all options on the same stock with the same expiry would show the same IV
regardless of strike. In reality this never happens. Different strikes show different IV — a pattern called the
volatility smile or skew.

NIFTY OPTIONS — VOLATILITY SKEW (typical equity pattern)

Quant Pro | Advanced Quant Finance Course | Week 5 Page 13

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

IV%
28 - * *
26 - * *
24 - * *
22 - * *
20 - * *
18 - * *
16 - * * * *
■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
Far OTM ATM OTM Far OTM
ITM Puts Spot Calls Calls

Left side (puts) is STEEPER than right side (calls)

= the equity volatility SKEW (not a symmetric smile)

Why? Institutional investors buy OTM puts as crash insurance.

This permanent demand keeps OTM put IV elevated.
Result: Black-Scholes systematically underprices downside risk.

* KEY INSIGHT
IV Rank (IVR) = (Current IV - 52-week Low) / (52-week High - 52-week Low) x 100. IVR above 70 means
options are expensive vs history — conditions favour selling premium. IVR below 30 means options are cheap
— conditions favour buying options or adding hedges. This single number tells you which side of the options
market has the statistical edge right now.

Python Code — Day 34: Extracting Implied Volatility

import numpy as np
from [Link] import norm
from [Link] import brentq
import [Link] as plt

def bs_price(S, K, T, r, sigma, opt="call"):

if T <= 0 or sigma <= 0:
return max(0, S-K) if opt=="call" else max(0, K-S)
d1 = ([Link](S/K) + (r + 0.5*sigma**2)*T) / (sigma*[Link](T))
d2 = d1 - sigma*[Link](T)
if opt=="call": return S*[Link](d1) - K*[Link](-r*T)*[Link](d2)
return K*[Link](-r*T)*[Link](-d2) - S*[Link](-d1)

# brentq finds the sigma where bs_price(sigma) = market_price

def implied_vol(mkt_px, S, K, T, r, opt="call"):
try:
return brentq(lambda s: bs_price(S,K,T,r,s,opt)-mkt_px,
0.001, 5.0, xtol=1e-6)
except ValueError:
return [Link]

S, r, T = 22000, 0.065, 30/365

Quant Pro | Advanced Quant Finance Course | Week 5 Page 14

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

strikes = [Link](20900, 23200, 200)

# Simulate market prices with realistic equity skew

# (In practice: fetch from NSE option chain via nsepython)
def mkt_px(S, K, T, r):
m = K/S
iv = 0.18 + 0.18*(1-m)**2 if m < 1 else 0.16 + 0.06*(m-1)**2
return bs_price(S, K, T, r, iv, "call")

ivs = [implied_vol(max(mkt_px(S,K,T,r),0.01),S,K,T,r)*100 for K in strikes]

[Link](figsize=(10, 5))
[Link](strikes, ivs, "o-", color="royalblue", lw=2, ms=5)
[Link](S, color="tomato", linestyle="--", lw=1.5, label=f"ATM = {S}")
[Link]("NIFTY Volatility Smile — 30-Day Options", fontweight="bold")
[Link]("Strike Price")
[Link]("Implied Volatility (%)")
[Link](); [Link](True, alpha=0.3)
[Link]("vol_smile.png", dpi=150, bbox_inches="tight")
[Link]()

Day 34 Key Takeaways

• Historical vol looks backward at past moves. Implied vol looks forward — the market's expectation of
future uncertainty.
• IV is extracted by reverse Black-Scholes: find the sigma that produces the observed market option price.
• The vol skew in equity markets is permanent — OTM puts always carry higher IV than equivalent OTM
calls.
• IV Rank (IVR) tells you whether options are cheap or expensive vs the past year. Use it to pick sides.
• IV spikes before known events like earnings, RBI meetings, and budget day — this event premium is
systematic.

Quant Pro | Advanced Quant Finance Course | Week 5 Page 15

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

DAY

GARCH — Volatility That Thinks

35 Model volatility clustering the way professional risk desks actually do it

The Problem with Rolling Standard Deviation

In Week 2 you measured volatility using rolling(20).std(). It works. But it treats every day in the 20-day
window equally — the crash 19 days ago counts the same as yesterday. This means it reacts too slowly to
new shocks and is too slow to forget old ones.

> REAL-WORLD ANALOGY

Judging how dangerous a river is by averaging the water level over the last 20 days. If there was a flood 19
days ago but it has been calm since, your average still shows high danger — even though the flood is over.
Conversely, if a storm just hit yesterday but the previous 19 days were calm, your average barely reflects the
new danger. Rolling std dev makes exactly this mistake with stock volatility.

Volatility Clustering — The Fact GARCH Captures

Look at any stock over several years. You will see one pattern immediately: turbulent periods follow
turbulent periods. Calm periods follow calm periods. This is called volatility clustering — one of the most
reliable facts in all of finance.

VOLATILITY CLUSTERING IN DAILY RETURNS

+4% | | ||
+3% | | || | |||
+2% | | || | || ||||||| |
+1% |||||||||||||||| | |||||||||||||||
0% |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
-1% |||||||||||||||| ||||||||||||||||||
-2% | | || |||||||| |
-3% | |||
<-- CALM -------> <-- TURBULENT --> <-- CALM -->

Rolling std dev: treats the calm and turbulent the same
if both fall within the 20-day window.
GARCH: knows you are IN a volatile period right now.
GARCH reacts faster to new shocks. Calms down faster too.

GARCH(1,1) — The Formula in Plain English

Quant Pro | Advanced Quant Finance Course | Week 5 Page 16

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

GARCH says tomorrow's variance is a combination of three things: a constant baseline, how large
yesterday's actual shock was, and how large yesterday's estimated variance was. If yesterday was wild,
today will probably also be wild.

GARCH(1,1) FORMULA:
var(today) = omega + alpha x shock^2(yesterday) + beta x var(yesterday)
^ ^ ^
Baseline How much recent How much yesterday
(constant) SHOCK matters today VARIANCE persists

Typical values for a large Indian stock like TCS:

omega = 0.000002 (tiny constant baseline)
alpha = 0.09 (9% weight on recent shock — reactive)
beta = 0.90 (90% weight on yesterday variance — sticky)
alpha + beta = 0.99 (very persistent — shocks last many days)

MUST HAVE: alpha + beta < 1.0 for the model to be stable.

Long-run vol = sqrt(omega / (1 - alpha - beta)) x sqrt(252)

This is the annualised vol GARCH expects in normal calm conditions.

* KEY INSIGHT
Alpha controls how quickly GARCH reacts to a new shock. Beta controls how long the shock persists. For
Indian large-cap stocks, beta is typically 0.88-0.93, meaning a volatility spike on Monday still has 88-93% of its
force on Tuesday. This is why markets stay elevated for days after a crash — the nervousness does not
disappear overnight.

Python Code — Day 35: Fitting GARCH(1,1)

# Install: pip install arch --break-system-packages
import pandas as pd, numpy as np
import [Link] as plt
from arch import arch_model

df = pd.read_csv("[Link]")
df = df[df["SYMBOL"]=="TCS"].sort_values("DATE1").copy()
df["DATE1"] = pd.to_datetime(df["DATE1"])

# arch needs returns in PERCENT form — multiply by 100

df["Return"] = df["CLOSE"].pct_change() * 100
df = [Link]()

# Fit GARCH(1,1)
# dist="t" uses t-distribution for fat tails — always use this
model = arch_model(df["Return"], vol="Garch", p=1, q=1,
mean="Zero", dist="t")
result = [Link](disp="off")

omega = [Link]["omega"]
alpha = [Link]["alpha[1]"]

Quant Pro | Advanced Quant Finance Course | Week 5 Page 17

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

beta = [Link]["beta[1]"]
lr_vol = [Link](omega/(1-alpha-beta)) * [Link](252)

print("=== GARCH(1,1) PARAMETERS ===")

print(f"omega (baseline) : {omega:.6f}")
print(f"alpha (shock wt) : {alpha:.4f}")
print(f"beta (persistence): {beta:.4f}")
print(f"alpha + beta : {alpha+beta:.4f} (must be < 1)")
print(f"Long-run Ann. Vol : {lr_vol:.1f}%")

# Forecast next 5 days

fc = [Link](horizon=5)
fvols = [Link]([Link][-1])
print("\nForecasted Daily Vol (next 5 days):")
for i, v in enumerate(fvols, 1):
print(f" Day {i}: {v:.2f}% (ann: {v*[Link](252):.1f}%)")

# Compare GARCH vs rolling std

df["Rolling"] = df["Return"].rolling(20).std()
df["GARCH"] = result.conditional_volatility

fig, axes = [Link](2,1,figsize=(13,8),sharex=True)

c = ["seagreen" if r>=0 else "tomato" for r in df["Return"]]
axes[0].bar(df["DATE1"], df["Return"], color=c, alpha=0.6)
axes[0].set_title("TCS Daily Returns", fontweight="bold")
axes[0].grid(True, alpha=0.2)

axes[1].plot(df["DATE1"],df["GARCH"],color="royalblue",lw=2,
label="GARCH(1,1)")
axes[1].plot(df["DATE1"],df["Rolling"],color="tomato",lw=1.5,
linestyle="--",label="20-day Rolling Std")
axes[1].set_title("GARCH vs Rolling Std Dev", fontweight="bold")
axes[1].set_ylabel("Daily Vol (%)")
axes[1].legend(); axes[1].grid(True, alpha=0.2)
plt.tight_layout()
[Link]("garch_comparison.png", dpi=150, bbox_inches="tight")
[Link]()

Day 35 Key Takeaways

• Volatility clusters: turbulent follows turbulent. GARCH models this directly. Rolling std dev does not.
• GARCH(1,1) needs only three parameters: omega (baseline), alpha (shock weight), beta (persistence).
• alpha + beta must be less than 1. Near 0.99 means shocks take many days to fully fade away.
• Always use dist="t" in arch_model — real returns have fat tails that the normal distribution misses
completely.
• The GARCH next-day vol forecast is your best risk estimate — feed it directly into position sizing
calculations.

Quant Pro | Advanced Quant Finance Course | Week 5 Page 18

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

DAY

Live Data Pipeline — Build It Once, Run It Forever

36 The system that downloads, computes, alerts, and logs — automatically every market day

Why Automation Changes Everything

Every project in Weeks 1-4 started the same way: open Jupyter, manually download a CSV, run the
notebook. Fine for learning. Completely impractical for a real system. A professional quant pipeline runs
automatically at 4 PM every market day — download fresh data, compute signals, check alerts, save results
— all while you are doing something else entirely.

> REAL-WORLD ANALOGY

A security system at a bank vault. A bad system requires a guard to manually check the door each time
someone asks. A good system checks the door every 60 seconds, logs every event, sends an alert if anything
unusual happens, and files a daily report — automatically, every day, without anyone telling it to. Your quant
pipeline should work exactly the same way. Set it up once. Let it run indefinitely.

Pipeline Architecture — 7 Steps

DAILY QUANT PIPELINE — RUNS AT 4:05 PM EVERY MARKET DAY

Step 1: DOWNLOAD yfinance fetches fresh OHLCV for all stocks

|
Step 2: VALIDATE check for missing data, outliers, stale prices
|
Step 3: COMPUTE calculate SMA, RSI, ADX, GARCH vol, signals
|
Step 4: ALERT CHECK did any golden cross or RSI extreme fire today?
|
Step 5: STORE save signals to CSV/JSON for tomorrow
|
Step 6: REPORT print clean dashboard table to terminal
|
Step 7: LOG write timestamp and status to log file

If any step fails: log the error, skip that stock, continue.
Never let one failed stock crash the whole pipeline.

Python Code — Day 36: Complete Automated Pipeline

# Install: pip install yfinance schedule --break-system-packages
import pandas as pd, numpy as np
import yfinance as yf

Quant Pro | Advanced Quant Finance Course | Week 5 Page 19

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

import json, os, logging

from datetime import date, datetime
import schedule, time

SYMBOLS = ["[Link]","[Link]","[Link]","[Link]","[Link]"]
NAMES = [[Link](".NS","") for s in SYMBOLS]
DATA_DIR = "live_data/"
[Link](DATA_DIR, exist_ok=True)

# Logging writes to file AND shows in terminal

[Link](level=[Link],
format="%(asctime)s %(levelname)s %(message)s",
handlers=[[Link]("[Link]"),
[Link]()])
log = [Link]()

def download_data():
[Link]("Downloading...")
for sym, name in zip(SYMBOLS, NAMES):
try:
df = [Link](sym).history(period="2y")
[Link] = pd.to_datetime([Link]).tz_localize(None)
df.to_csv(f"{DATA_DIR}{name}.csv")
[Link](f" {name}: {len(df)} rows")
except Exception as e:
[Link](f" {name}: FAILED - {e}")

def rsi_calc(s, p=14):

d=[Link](); g=[Link](lower=0).ewm(com=p-1,min_periods=p).mean()
l=(-d).clip(lower=0).ewm(com=p-1,min_periods=p).mean()
return 100 - 100/(1+g/l)

def compute_signals():
signals = {}
for name in NAMES:
try:
df = pd.read_csv(f"{DATA_DIR}{name}.csv",
index_col=0, parse_dates=True).sort_index()
if len(df) < 60: continue
c = df["Close"]
df["SMA_20"] = [Link](20).mean()
df["SMA_50"] = [Link](50).mean()
df["RSI"] = rsi_calc(c)
t, p = [Link][-1], [Link][-2]
signals[name] = {
"price" : round(float(t["Close"]),2),
"rsi" : round(float(t["RSI"]),1),
"golden" : p["SMA_20"]<=p["SMA_50"] and t["SMA_20"]>t["SMA_50"],
"death" : p["SMA_20"]>=p["SMA_50"] and t["SMA_20"]<t["SMA_50"],
"ob" : float(t["RSI"]) > 70,
"os" : float(t["RSI"]) < 30,

Quant Pro | Advanced Quant Finance Course | Week 5 Page 20

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

}
except Exception as e:
[Link](f" {name}: compute failed - {e}")
return signals

def report_and_store(signals):
alerts = []
for n, s in [Link]():
if s["golden"]: [Link](f"GOLDEN CROSS: {n} @ {s['price']}")
if s["death"]: [Link](f"DEATH CROSS : {n} @ {s['price']}")
if s["ob"]: [Link](f"OVERBOUGHT : {n} RSI={s['rsi']}")
if s["os"]: [Link](f"OVERSOLD : {n} RSI={s['rsi']}")
print(f"\n{'='*50}")
print(f" QUANT DASHBOARD {[Link]()}")
print(f"{'='*50}")
for n, s in [Link]():
tag = "GOLDEN" if s["golden"] else "DEATH" if s["death"] else "HOLD"
print(f" {n:<12} Rs{s['price']:>8.0f} RSI={s['rsi']:>5.1f} {tag}")
if alerts:
print(f"\n {len(alerts)} ALERT(S):")
for a in alerts: print(f" -> {a}")
[Link]({"date":str([Link]()),"alerts":alerts,"signals":signals},
open("daily_signals.json","w"), indent=2)
[Link](f"Pipeline done. {len(alerts)} alerts.")

def daily_job():
download_data()
report_and_store(compute_signals())

if __name__ == "__main__":
daily_job() # Run immediately on start
[Link]().[Link]("16:05").do(daily_job)
[Link]("Scheduler active. Daily at 16:05.")
while True:
schedule.run_pending()
[Link](60)

Day 36 Key Takeaways

• yfinance gives 2 years of free OHLCV for NSE stocks — use the ".NS" suffix ([Link], [Link] etc).
• Wrap every stock in try/except — one failed ticker must never crash the entire pipeline.
• Use the logging module instead of print() — every event gets a timestamp and writes to a file
automatically.
• [Link]().[Link]("16:05").do(job) runs any function at a fixed time — no Linux cron knowledge
needed.
• Save signals to JSON daily — you build an automatic historical log of every alert your system has ever
fired.

Quant Pro | Advanced Quant Finance Course | Week 5 Page 21

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

DAY

Transaction Cost Modelling

37 The gap between backtest PnL and real money — and exactly how to close it

The Silent Killer of Backtests

Your Week 3 backtest bought and sold at the exact closing price, with zero cost, instantly. In real trading,
none of that is true. There is a government tax on every sell. A gap between the buy and sell price in the
order book. A price movement the moment your order hits the market. These seem small individually but
compound across many trades into something that can completely wipe out a profitable strategy.

COMPLETE COST BREAKDOWN — ONE ROUND-TRIP TRADE (NSE Delivery)

Trade: Buy TCS Rs 3,500 -> Sell Rs 3,600 Gross gain: Rs 100/share
Capital: Rs 98,000 | Shares: 28

BUY SIDE:
Bid-ask spread (buy at ask not mid) 0.05% = Rs 49
Slippage (price moves as order fills) 0.05% = Rs 49

SELL SIDE:
STT Securities Transaction Tax 0.10% = Rs 101 <- LARGEST COST
NSE Exchange + SEBI charges 0.0035%= Rs 3
Bid-ask spread on sell 0.05% = Rs 49
Slippage on sell 0.05% = Rs 49
GST on charges = Rs 1

TOTAL ROUND-TRIP COST: = Rs 252

Gross profit: Rs 2,800
NET profit after costs: Rs 2,548 (-9%)

For a strategy making 0.20% per trade: GROSS=Rs196 COST=Rs252

Result: LOSING after costs despite positive backtest returns.

! IMPORTANT WARNING
STT alone — 0.1% on the sell side — costs Rs 100 on every Rs 1 lakh sell order. For a strategy with 20 round
trips per year on Rs 1 lakh capital, STT alone costs Rs 2,000 per year. Many strategies that look profitable in
backtests are losing strategies after STT is applied. This is not a minor detail. It changes the conclusion
completely.

Python Code — Day 37: Realistic Cost Modelling

import pandas as pd, numpy as np
import [Link] as plt

COSTS = {

Quant Pro | Advanced Quant Finance Course | Week 5 Page 22

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

"stt_pct" : 0.001, # 0.1% on SELL side only

"exchange_pct" : 0.0000345, # NSE + SEBI
"gst_rate" : 0.18,
"slippage_pct" : 0.0005, # per side
"spread_pct" : 0.0005, # per side
}

def one_way_cost(c):
exch = c["exchange_pct"]
return (exch + exch*c["gst_rate"] + c["stt_pct"]/2
+ c["slippage_pct"] + c["spread_pct"])

oc = one_way_cost(COSTS)
print(f"One-way: {oc*100:.3f}% Round-trip: {oc*200:.3f}%")
print(f"On Rs 1 lakh: Rs {oc*2*100_000:.0f} per round trip")

df = pd.read_csv("[Link]")
df = df[df["SYMBOL"]=="TCS"].sort_values("DATE1").reset_index(drop=True)
df["DATE1"] = pd.to_datetime(df["DATE1"])
df["SMA_20"] = df["CLOSE"].rolling(20).mean()
df["SMA_50"] = df["CLOSE"].rolling(50).mean()
df["Signal"] = (df["SMA_20"] > df["SMA_50"]).astype(int)
df["Position"] = df["Signal"].shift(1).diff()

def simulate(df, buy_c=0.0, sell_c=0.0, cap=100_000):

sh, en, c = 0, 0, cap
eq, trades = [], 0
for _, row in [Link]():
if row["Position"]==1 and sh==0:
px=row["CLOSE"]*(1+buy_c); sh=int(c/px); c-=sh*px; trades+=1
elif row["Position"]==-1 and sh>0:
px=row["CLOSE"]*(1-sell_c); c+=sh*px; sh=0; trades+=1
[Link](c + sh*row["CLOSE"])
ret=(eq[-1]/cap-1)*100
npt=ret/(trades//2) if trades>0 else 0
return [Link](eq), ret, trades//2, npt

eq_g, rg, ng, npg = simulate(df, 0, 0)

eq_r, rr, nr, npr = simulate(df, oc, oc)
eq_s, rs, ns, nps = simulate(df, 0.003, 0.003)

print("\n=== PERFORMANCE COMPARISON ===")

for lbl, ret, n, npt in [
("Gross (zero costs)", rg, ng, npg),
("Net (realistic) ", rr, nr, npr),
("Stress (high slip)", rs, ns, nps),
]:
v = "VIABLE" if npt > oc*2*100 else "NOT VIABLE"
print(f"{lbl}: {ret:+.1f}% trades={n} net/trade={npt:+.2f}% {v}")

fig, ax = [Link](figsize=(13,5))
[Link](df["DATE1"],eq_g,color="seagreen",lw=2,label=f"Gross {rg:+.1f}%")
[Link](df["DATE1"],eq_r,color="royalblue",lw=2,label=f"Net {rr:+.1f}%")

Quant Pro | Advanced Quant Finance Course | Week 5 Page 23

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

[Link](df["DATE1"],eq_s,color="tomato",lw=1.5,linestyle="--",
label=f"Stress {rs:+.1f}%")
[Link](100_000,color="gray",linestyle=":",lw=1)
ax.set_title("Strategy: Gross vs Net vs Stress PnL",fontweight="bold")
[Link](); [Link](True,alpha=0.2)
[Link]("cost_analysis.png",dpi=150,bbox_inches="tight")
[Link]()

Day 37 Key Takeaways

• STT (0.1% on sell) is the largest single cost for NSE delivery traders — Rs 100 per lakh on every sell.
• Net profit per trade must comfortably exceed round-trip cost. If not, the strategy cannot survive in
production.
• Always run three scenarios: zero cost, realistic, and high slippage stress. Present all three.
• High-frequency strategies are destroyed by costs. Low-frequency monthly strategies survive much
better.
• Never present a backtest result without also showing the net-of-costs version. Gross PnL is marketing.
Net PnL is truth.

Quant Pro | Advanced Quant Finance Course | Week 5 Page 24

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

DAY

Walk-Forward Testing — The Truth Test

38 Prove your strategy works on data it has never seen before

Why Your Backtest Is Lying to You

On Day 19 you ran a grid search and found that SMA(15,40) produced the best backtest returns. But best
on what data? The exact data you used to find the parameters. This is like studying last year's exam
answers and claiming you understand the subject because you scored 95% on that same exam. The only
honest test is one where the data was never touched during development.

> REAL-WORLD ANALOGY

A cricket team selects its eleven based on their statistics against Team B specifically — then plays Team B in
the final. Of course the statistics look great. They were built on that exact opponent. Now imagine selecting the
eleven based on performance against completely different teams, then playing Team B for the first time. The
second approach gives you a genuine measure of selection quality. Walk-forward testing does exactly this for
trading strategies.

WALK-FORWARD TESTING — HOW IT WORKS

YOUR FULL DATA:

+-----------------------+-------------+
| TRAIN: Find params | TEST: Verify|
| 2019 -------- 2022 | 2023 - 2024 |
+-----------------------+-------------+
Simple split: ONE test period. Could be lucky or unlucky.

WALK-FORWARD: MULTIPLE rolling test periods

Window 1: Train 2019-2020 | Test Q1 2021
Window 2: Train 2019-2021 | Test Q2 2021
Window 3: Train 2019-2021 | Test Q3 2021
Window 4: Train 2019-2022 | Test Q1 2022
Final result = ALL test periods combined.

Walk-Forward Efficiency (WFE) = OOS Return / IS Return

WFE > 0.5 -> strategy generalises well
WFE < 0.2 -> strategy is memorising history (overfit)

Python Code — Day 38: Walk-Forward Engine

import pandas as pd, numpy as np, itertools

df = pd.read_csv("[Link]")
df = df[df["SYMBOL"]=="TCS"].sort_values("DATE1").reset_index(drop=True)
df["DATE1"] = pd.to_datetime(df["DATE1"])

Quant Pro | Advanced Quant Finance Course | Week 5 Page 25

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

def backtest(df, fast, slow, cap=100_000):

d=[Link]()
d["F"]=d["CLOSE"].rolling(fast).mean()
d["S"]=d["CLOSE"].rolling(slow).mean()
d["P"]=(d["F"]>d["S"]).astype(int).shift(1).diff()
c,sh=cap,0
for _,r in [Link]():
if r["P"]==1 and sh==0: sh=int(c/r["CLOSE"]); c-=sh*r["CLOSE"]
elif r["P"]==-1 and sh>0: c+=sh*r["CLOSE"]; sh=0
return ((c+sh*d["CLOSE"].iloc[-1])/cap-1)*100

def walk_forward(df, frange, srange, train=504, test=63):

results=[]
for start in range(0, len(df)-train-test, test):
tr=[Link][start:start+train]
te=[Link][start+train:start+train+test]
best, bf, bs = -999, 20, 50
for f, s in [Link](frange, srange):
if f>=s: continue
r=backtest(tr,f,s)
if r>best: best,bf,bs=r,f,s
oos=backtest(te,bf,bs)
d=df["DATE1"].iloc[start+train].date()
[Link]({"window":str(d),"fast":bf,"slow":bs,
"IS":round(best,2),"OOS":round(oos,2)})
print(f" {d} ({bf}/{bs}) IS={best:+.1f}% OOS={oos:+.1f}%")
return [Link](results)

print("Running Walk-Forward...")
wf=walk_forward(df,[10,15,20,25],[30,40,50,60])
print(wf.to_string(index=False))
wfe=wf["OOS"].mean()/wf["IS"].mean()
wr=(wf["OOS"]>0).mean()*100
print(f"\nMean IS : {wf['IS'].mean():+.2f}%")
print(f"Mean OOS : {wf['OOS'].mean():+.2f}%")
print(f"OOS WinRate: {wr:.0f}%")
print(f"WFE: {wfe:.2f}")
print("Excellent" if wfe>0.5 else "Acceptable" if wfe>0.2 else "WARNING: overfit")

# GOLDEN RULE
The test set is a one-time-use resource. The moment you look at test results and use them to improve your
strategy — even once — it is no longer a test set. It has become training data. Treat your OOS period like a
sealed envelope: open it only when you are completely done building.

Day 38 Key Takeaways

• In-sample results always look better than out-of-sample. The question is: by how much?

Quant Pro | Advanced Quant Finance Course | Week 5 Page 26

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

• Walk-Forward Efficiency (WFE) above 0.5 means the strategy generalises well to new data.
• OOS win rate across multiple windows is the most honest measure of whether your strategy is robust.
• A strategy that works consistently across all walk-forward windows is real. One that works in 2 of 6 is
not.
• The test set is one-time use only. Looking at it to improve the strategy converts it to training data.

Quant Pro | Advanced Quant Finance Course | Week 5 Page 27

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

DAY

Project 3 — Live Quant Dashboard

39 Every Week 5 skill combined into one system that runs automatically every market day

Project 3 Overview
Today you build the system that brings everything from Week 5 together. ADX regime filter from Day 31.
Statistical validation from Day 33. Live data pipeline from Day 36. Realistic costs from Day 37. GARCH
volatility from Day 35. Walk-forward validated signals. All running automatically every day at 4 PM without
you touching anything.

Section 1 Live data download via yfinance — 5 NIFTY stocks, 2-year history, daily refresh

Section 2 ADX regime filter — only generate signals when trend strength confirms

Section 3 SMA crossover + RSI alerts — signal generation with regime awareness

Section 4 GARCH(1,1) next-day volatility forecast per stock

Section 5 Transaction cost model — net PnL after STT, slippage, spread

Section 6 Portfolio VaR (historical method) on equal-weighted positions

Section 7 Statistical check — is current 3-month period Sharpe significant?

Section 8 4-panel matplotlib dashboard saved as PNG daily

Section 9 Alerts log written to JSON — permanent record of every signal

Section 10 GitHub upload — week5/ folder, full README, dashboard PNG committed

Complete Project 3 Code

# PROJECT 3 — LIVE QUANT DASHBOARD — Quant Pro Week 5
import pandas as pd, numpy as np, yfinance as yf
import [Link] as plt
import [Link] as gridspec
from arch import arch_model
from scipy import stats
from datetime import date
import json, warnings
[Link]("ignore")

SYMBOLS = ["[Link]","[Link]","[Link]","[Link]","[Link]"]
NAMES = [[Link](".NS","") for s in SYMBOLS]
COST_RT = 0.0023 # round-trip cost fraction

# 1. Download fresh data

raw = {}

Quant Pro | Advanced Quant Finance Course | Week 5 Page 28

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

for sym, name in zip(SYMBOLS, NAMES):

df = [Link](sym).history(period="2y")
[Link] = pd.to_datetime([Link]).tz_localize(None)
raw[name] = df.sort_index()
print(f" {name}: {len(df)} rows")

# Helpers
def rsi(s,p=14):
d=[Link](); g=[Link](lower=0).ewm(com=p-1,min_periods=p).mean()
return 100-100/(1+g/(-d).clip(lower=0).ewm(com=p-1,min_periods=p).mean())

def adx_val(df,p=14):
h,l,c=df["High"],df["Low"],df["Close"]
tr=[Link]([h-l,([Link]()).abs(),([Link]()).abs()],axis=1).max(1)
up=[Link](); dn=[Link]()-l
pm=[Link]((up>dn)&(up>0),up,0); mm=[Link]((dn>up)&(dn>0),dn,0)
atr=[Link](tr).ewm(alpha=1/p,adjust=False).mean()
pd_=100*[Link](pm).ewm(alpha=1/p,adjust=False).mean()/atr
md_=100*[Link](mm).ewm(alpha=1/p,adjust=False).mean()/atr
return (100*(pd_-md_).abs()/(pd_+md_)).ewm(alpha=1/p,adjust=False).mean()

# 2. Signals + GARCH per stock

results = {}
for name, df in [Link]():
c=df["Close"]; ret=c.pct_change().dropna()*100
df["SMA_20"]=[Link](20).mean(); df["SMA_50"]=[Link](50).mean()
df["RSI"]=rsi(c); df["ADX"]=adx_val(df).values
gm=arch_model([Link](),vol="Garch",p=1,q=1,mean="Zero",dist="t")
gr=[Link](disp="off")
fv=float([Link]([Link](horizon=1).[Link][-1,0]))
t=[Link][-1]; p=[Link][-2]
g=(p["SMA_20"]<=p["SMA_50"]) and (t["SMA_20"]>t["SMA_50"])
d=(p["SMA_20"]>=p["SMA_50"]) and (t["SMA_20"]<t["SMA_50"])
trend=float(t["ADX"])>25
sig="GOLDEN" if g and trend else "DEATH" if d and trend else "HOLD"
results[name]={"df":df,"close":float(t["Close"]),"rsi":float(t["RSI"]),
"gv":gr.conditional_volatility,"fv":fv,"signal":sig}

# 3. Portfolio VaR
pivot=([Link]({n:raw[n]["Close"].pct_change() for n in NAMES})
.dropna())
w=[Link](len(NAMES))/len(NAMES)
pr=(pivot@w).dropna()
var95=[Link](pr,5)
cvar95=pr[pr<=var95].mean()

# 4. Statistical check
rec=[Link](63).values
_,pval=stats.ttest_1samp(rec,0)

# 5. Dashboard
fig=[Link](figsize=(16,12))

Quant Pro | Advanced Quant Finance Course | Week 5 Page 29

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

gs=[Link](2,2,hspace=0.4,wspace=0.35)

ax1=fig.add_subplot(gs[0,0])
for n,r in [Link]():
[Link](r["df"].index[-120:],r["gv"][-120:],lw=1.5,label=n)
ax1.set_title("GARCH Vol — Last 120 Days",fontweight="bold")
[Link](fontsize=7); [Link](True,alpha=0.2)

ax2=fig.add_subplot(gs[0,1]); [Link]("off")
td=[[n,f"{r['close']:.0f}",f"{r['rsi']:.1f}",f"{r['fv']:.2f}%",r["signal"]]
for n,r in [Link]()]
[Link](cellText=td,colLabels=["Stock","Price","RSI","Fcast Vol","Signal"],
loc="center",cellLoc="center")
ax2.set_title("Signal Dashboard",fontweight="bold",pad=20)

ax3=fig.add_subplot(gs[1,0])
[Link](pr*100,bins=60,color="steelblue",alpha=0.7,edgecolor="white")
[Link](var95*100,color="red",lw=2,label=f"VaR(95%): {var95*100:.2f}%")
[Link](cvar95*100,color="darkred",lw=2,linestyle="--",
label=f"CVaR: {cvar95*100:.2f}%")
ax3.set_title("Portfolio Return Distribution",fontweight="bold")
[Link](fontsize=8); [Link](True,alpha=0.2)

ax4=fig.add_subplot(gs[1,1])
tcs=results["TCS"]["df"].iloc[-252:]
[Link]([Link],tcs["Close"],color="lightgray",lw=1,label="Close")
[Link]([Link],tcs["SMA_20"],color="royalblue",lw=2,label="SMA 20")
[Link]([Link],tcs["SMA_50"],color="tomato",lw=2,label="SMA 50")
ax4.set_title("TCS — 1 Year",fontweight="bold")
[Link](fontsize=8); [Link](True,alpha=0.2)

[Link](f"Quant Dashboard {[Link]()} | p={pval:.3f} VaR={var95*100:.2f}%",

fontsize=13,fontweight="bold")
[Link]("week5_dashboard.png",dpi=150,bbox_inches="tight")
[Link]()
print("Saved: week5_dashboard.png")

Week 5 Completion Checklist

• I can explain p-value, Sharpe t-statistic, and bootstrap CI to a non-statistician without using jargon.
• I have extracted implied volatility from an option price and can explain why OTM puts always show
higher IV.
• I have fitted GARCH(1,1), read alpha and beta correctly, and used the forecast for next-day risk
estimation.
• My pipeline auto-downloads data daily via yfinance — no more manual CSV downloads ever again.
• Every backtest I run now shows gross PnL, net-of-costs PnL, and high-slippage stress scenario.
• I have run walk-forward testing and can report OOS win rate and Walk-Forward Efficiency (WFE).
• Project 3 dashboard is uploaded to GitHub with a complete README showing all metrics honestly.

Quant Pro | Advanced Quant Finance Course | Week 5 Page 30

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

# GOLDEN RULE
You have now crossed a critical line. Weeks 1-4 were theory and demonstration. Week 5 is production.
Everything here runs on live data, with real costs, with statistical proof, validated on unseen data. That is the
difference between a student project and a professional tool.

Quant Pro | Advanced Quant Finance Course | Week 5 Page 31

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

WEEK 5 — MASTER CHEATSHEET

Statistics — [Link]
Function What It Does Example / Key Note

stats.ttest_1samp(r,0) Test if mean return != 0 t, p =

stats.ttest_1samp(returns, 0)

[Link](arr) Skewness — neg=left tail risk [Link](returns)

[Link](arr) Excess kurtosis — fat tails? [Link](returns)

[Link](arr) Normality test — p>0.05=normal stat, p = [Link](returns)

sharpe x sqrt(n) Sharpe t-stat — need >1.96 sr_t = sharpe *

[Link](n_trades)

[Link](arr,5) 5th percentile = VaR(95%) [Link](port_ret, 5)

GARCH — arch library

Function What It Does Key Note

arch_model(ret,vol="Garch") Define GARCH(1,1) Use dist="t" always for fat

tails

[Link](disp="off") Fit model silently Returns result object

result.conditional_volatility Fitted vol series In % units matching input

[Link](horizon=N) Forecast N days ahead .[Link][-1] to access

[Link]["alpha[1]"] Shock weight Typical: 0.09 for large-cap

[Link]["beta[1]"] Persistence Typical: 0.90 — very sticky

Live Data and Automation

Function What It Does Example

[Link](sym).history(period) Download OHLCV from Yahoo [Link]("[Link]").history(peri

od="2y")

[Link].tz_localize(None) Remove timezone from index Required after every yfinance

download

[Link]().[Link]("16:05") Schedule job at 4:05 PM .do(daily_job)

[Link](...) Setup file+screen logging handlers=[FileHandler,

StreamHandler]

[Link](data,f,indent=2) Save dict to readable JSON Saves alerts and signals daily

Quant Pro | Advanced Quant Finance Course | Week 5 Page 32

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

Week 5 Concepts Summary

Concept Plain-English Meaning Key Number Tool

Market Regime Trending/sideways/volat ADX > 25 = trend Custom ADX function

ile — which one now?

p-value Probability result is < 0.05 = real [Link].ttest_1samp

random luck

Implied Vol Market forward vol IVR 0-100 [Link]

forecast in option
prices

Vol Skew OTM puts cost more — Skew slope BS reverse engineering
always in equity mkts

GARCH(1,1) Dynamic vol that alpha+beta < 1 arch library

updates daily with new
shocks

Live Pipeline Auto-download, compute, 4:05 PM daily yfinance + schedule

alert, log daily

Transaction Costs Real PnL after STT, RT ~0.23% Custom cost model
slippage, spread

Walk-Forward OOS validation on data WFE > 0.5 Rolling window loop
strategy never saw

Fea
Machine Learning for Finance — Done Properly

Quant Pro | Advanced Quant Finance Course | Week 5 Page 33

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

QUANT PRO
Advanced Quant Finance Course
WEEK 6 — Days 40 to 47
Feature Engineering · Time-Series CV · Model Evaluation · SHAP Interpretability · NLP Sentiment · Regime ML ·
Project 4

8 25+ 3 1
DAYS FEATURES ML MODELS PROJECT

Week 5 gave you production-ready tools — live data, real costs, statistical validation. Week 6 now rebuilds
the machine learning model from Day 26 of the original course completely from scratch. The Day 26 model
had three critical flaws: data leakage in features, a random train-test split that breaks temporal order, and
no way to understand why the model made any prediction. This week fixes all three — and adds sentiment
signals on top.

WEEK 6 AT A GLANCE
Day Topic Key Skill Project Output

40 Feature Engineering — No Lag shifts, rolling features, 25-feature set validated for
Leakage shift rule leakage

41 Time-Series TimeSeriesSplit, purging, CV results vs naive random

Cross-Validation embargoing split comparison

42 Three Models Compared LogReg, Random Forest, XGBoost Side-by-side model comparison
Properly calibrated on same data

43 Evaluation Beyond Accuracy Precision, recall, ROC-AUC, Model evaluation dashboard

threshold tuning

44 SHAP Interpretability Global importance, local SHAP summary + waterfall

explanations plots

45 NLP and Sentiment Signals VADER, FinBERT, shift rule for Sentiment feature added to ML
events pipeline

46 Regime-Aware ML HMM regime detection, Regime-aware signal with VIX

regime-conditioned models proxy

47 Project 4 — ML Alpha All Week 6 skills combined GitHub: ML pipeline + SHAP +

Pipeline end-to-end README

Quant Pro | Advanced Quant Finance Course | Week 5 Page 34

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

What Day 26 ML Had What Week 6 Fixes or Adds

Simple 80/20 random split — temporal order TimeSeriesSplit — train always before test in
broken time

Features used today's data to predict today Shift rule enforced — every feature uses
shift(1)

Reported accuracy only — no precision/recall/AUC Full evaluation: ROC-AUC, F1, calibration curve

No idea why the model made any prediction SHAP values — global and per-prediction
explanation

Same model regardless of market regime Regime detection — different model in different
market

Only price and technical features NLP sentiment from news headlines added as
feature

Quant Pro | Advanced Quant Finance Course | Week 5 Page 35

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

DAY

Feature Engineering — Building the Right Inputs

40 How to create 25 meaningful features without leaking future data into the past

Why Features Matter More Than Model Choice

In machine learning, garbage in equals garbage out. The choice between logistic regression and gradient
boosting matters far less than the quality of the features you feed them. A simple logistic regression with
excellent features beats gradient boosting with poor features almost every time. This day is about building
features that genuinely describe market conditions — without accidentally cheating by using future
information.

> REAL-WORLD ANALOGY

Imagine training a cricket predictor to guess if a batsman will score over 50 in tomorrow's match. Bad feature:
"did he score over 50 in tomorrow's match." That is data leakage — you used the answer as an input. Good
feature: "his average in the last 5 matches before today." This is the shift rule. Every single feature in your ML
model must be calculable using only data that existed before the prediction date. One leaked feature poisons
the entire model.

The Shift Rule — The Most Important Rule in Financial ML

THE SHIFT RULE

Every feature used to predict tomorrow must use .shift(1)

so it contains only data available up to and including today.

WRONG (data leakage):

df["RSI_feature"] = calculate_rsi(df["CLOSE"])
# RSI for day T uses close of day T to predict return on day T
# You cannot know day T close before day T ends!

CORRECT (no leakage):

df["RSI_feature"] = calculate_rsi(df["CLOSE"]).shift(1)
# RSI for day T now uses close of day T-1
# You knew this yesterday. Safe to use.

WRONG target definition:

df["Target"] = (df["Return"] > 0).astype(int)
# Predicting today's return using today's features = circular

CORRECT target definition:

df["Target"] = (df["Return"].shift(-1) > 0).astype(int)
# Predicting TOMORROW's return using today's features = valid

Quant Pro | Advanced Quant Finance Course | Week 5 Page 36

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

The 25 Features — Built Correctly

These features cover four categories: momentum (where is price going), volatility (how fast is it moving),
volume (how much interest is there), and market structure (where is price relative to key levels). All use
shift(1) before being added to the feature matrix.

Python Code — Day 40: Building the Full Feature Set

import pandas as pd
import numpy as np

df = pd.read_csv("[Link]")
df = df[df["SYMBOL"]=="TCS"].sort_values("DATE1").reset_index(drop=True)
df["DATE1"] = pd.to_datetime(df["DATE1"])

# ■■ RETURNS ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
df["ret_1d"] = df["CLOSE"].pct_change(1)
df["ret_5d"] = df["CLOSE"].pct_change(5)
df["ret_10d"] = df["CLOSE"].pct_change(10)
df["ret_20d"] = df["CLOSE"].pct_change(20)
df["ret_60d"] = df["CLOSE"].pct_change(60)

# ■■ VOLATILITY ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
df["vol_5d"] = df["ret_1d"].rolling(5).std()
df["vol_20d"] = df["ret_1d"].rolling(20).std()
df["vol_60d"] = df["ret_1d"].rolling(60).std()
df["vol_ratio"] = df["vol_5d"] / df["vol_20d"] # vol regime indicator

# ■■ MOVING AVERAGE STRUCTURE ■■■■■■■■■■■■■■■■■■■■■■■■■

df["sma20"] = df["CLOSE"].rolling(20).mean()
df["sma50"] = df["CLOSE"].rolling(50).mean()
df["ema20"] = df["CLOSE"].ewm(span=20, adjust=False).mean()
df["dist_sma20"] = (df["CLOSE"] - df["sma20"]) / df["sma20"] # % above/below SMA
df["dist_sma50"] = (df["CLOSE"] - df["sma50"]) / df["sma50"]
df["sma_cross"] = (df["sma20"] > df["sma50"]).astype(int)

# ■■ RSI ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
def rsi(s, p=14):
d = [Link]()
g = [Link](lower=0).ewm(com=p-1, min_periods=p).mean()
l = (-d).clip(lower=0).ewm(com=p-1, min_periods=p).mean()
return 100 - 100 / (1 + g / l)

df["rsi_14"] = rsi(df["CLOSE"], 14)

df["rsi_7"] = rsi(df["CLOSE"], 7)

# ■■ VOLUME ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
df["vol_chg_5d"] = df["TOTTRDQTY"].pct_change(5)
df["vol_chg_20d"] = df["TOTTRDQTY"].pct_change(20)
df["vol_vs_avg"] = df["TOTTRDQTY"] / df["TOTTRDQTY"].rolling(20).mean()

# ■■ CANDLE STRUCTURE ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■

df["body_size"] = (df["CLOSE"] - df["OPEN"]).abs() / df["OPEN"]
df["upper_wick"] = (df["HIGH"] - df[["CLOSE","OPEN"]].max(axis=1)) / df["OPEN"]

Quant Pro | Advanced Quant Finance Course | Week 5 Page 37

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

df["lower_wick"] = (df[["CLOSE","OPEN"]].min(axis=1) - df["LOW"]) / df["OPEN"]

# ■■ TARGET: will TOMORROW's return be positive? ■■■■■■

df["target"] = (df["CLOSE"].pct_change().shift(-1) > 0).astype(int)

# ■■ APPLY SHIFT(1) TO ALL FEATURES ■■■■■■■■■■■■■■■■■■

# This is the shift rule. Every feature gets shifted by 1.
# Now on day T, the feature contains data from day T-1.
FEATURE_COLS = [
"ret_1d","ret_5d","ret_10d","ret_20d","ret_60d",
"vol_5d","vol_20d","vol_60d","vol_ratio",
"dist_sma20","dist_sma50","sma_cross",
"rsi_14","rsi_7",
"vol_chg_5d","vol_chg_20d","vol_vs_avg",
"body_size","upper_wick","lower_wick",
]

for col in FEATURE_COLS:

df[col] = df[col].shift(1) # enforce shift rule

df = [Link]()
X = df[FEATURE_COLS].values
y = df["target"].values

print(f"Feature matrix: {[Link][0]} rows x {[Link][1]} features")

print(f"Target balance: {[Link]()*100:.1f}% UP days")
print("All features use shift(1) — no data leakage")

Day 40 Key Takeaways

• Data leakage is the single biggest mistake in financial ML — using future information to predict the past.
• The Shift Rule: every feature must use .shift(1) so it only contains data available before the prediction
date.
• Four feature categories: momentum (returns), volatility (std dev), volume (trading interest), structure
(candles, MAs).
• The target must also be defined correctly — tomorrow's return using shift(-1), not today's return.
• After building features, always check: can this value be calculated in real-time without knowing the
future?

Quant Pro | Advanced Quant Finance Course | Week 5 Page 38

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

DAY

Time-Series Cross-Validation

41 Why random train-test split is wrong for stock data — and the correct alternative

Why Random Split Breaks Everything in Finance

In a standard machine learning course you split data randomly into 80% train and 20% test. This is fine for
predicting house prices or classifying images. It is completely wrong for financial time series. Here is exactly
why: if you randomly shuffle and split daily stock data, your training set will contain data from 2024 while
your test set contains data from 2020. The model sees tomorrow's market conditions while learning about
yesterday. This is future leakage at the split level — even if your features are perfectly clean.

WHY RANDOM SPLIT IS WRONG FOR TIME-SERIES DATA

Your data: Jan 2020 ... Dec 2020 ... Jun 2022 ... Dec 2024

RANDOM SPLIT (wrong):

Train set: Jan-2020, Mar-2021, Nov-2022, Jul-2024 (random mix)
Test set: Feb-2020, Apr-2021, Dec-2022, Aug-2024 (random mix)

Problem: Model trained on Nov-2022 data is tested on Feb-2020.

It has seen the future during training. Results are inflated.

TIMESERIESSPLIT (correct):
Fold 1: Train Jan-Dec 2020 | Test Jan-Mar 2021
Fold 2: Train Jan 2020-Mar 2021 | Test Apr-Jun 2021
Fold 3: Train Jan 2020-Jun 2021 | Test Jul-Sep 2021
Fold 4: Train Jan 2020-Sep 2021 | Test Oct-Dec 2021
Fold 5: Train Jan 2020-Dec 2021 | Test Jan-Mar 2022

Train always ends BEFORE test begins. No future leakage.

Final score = average across all test folds.

Purging and Embargoing — Advanced Protection

Even with TimeSeriesSplit, there is a subtler leakage risk. If your features include a 5-day rolling return,
then the last 5 days of your training set overlap with the first 5 days of the test set. A sample in training and
a sample in test share the same raw price data. This is called temporal overlap leakage. The fix: purge the
last N days of training (remove them) and embargo the first N days of testing (do not use them for scoring).

PURGING AND EMBARGOING

Feature uses 20-day rolling window -> overlap risk = 20 days

BEFORE purging/embargoing:
Train: Jan 2020 ■■■■■■■■■■■■■■■■■■■ Dec 31 2021
Test: Jan 1 2022 ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■

Quant Pro | Advanced Quant Finance Course | Week 5 Page 39

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

Overlap: last 20 days of train share data with first 20 of test

AFTER purging/embargoing:
Train: Jan 2020 ■■■■■■■■■■■■■■ Dec 11 2021 (last 20 removed)
Embargo: Jan 1-20 2022 not scored (overlap zone)
Test scored: Jan 21 2022 onwards (clean zone)

Result: truly clean separation between train and test.

Performance estimate is now genuinely honest.

Python Code — Day 41: TimeSeriesSplit + Purging

import pandas as pd, numpy as np
from sklearn.model_selection import TimeSeriesSplit
from [Link] import RandomForestClassifier
from [Link] import StandardScaler
from [Link] import roc_auc_score, accuracy_score

# X, y from Day 40 feature engineering

# X = feature matrix (n_samples, n_features)
# y = target (0 = DOWN tomorrow, 1 = UP tomorrow)

PURGE_DAYS = 20 # match your longest rolling window

EMBARGO_DAYS = 5 # extra buffer after purge

tscv = TimeSeriesSplit(n_splits=5)
scaler = StandardScaler()

results = []

for fold, (train_idx, test_idx) in enumerate([Link](X), 1):

# Apply purging: remove last PURGE_DAYS from train
purge_start = train_idx[-1] - PURGE_DAYS
train_idx = train_idx[train_idx < purge_start]

# Apply embargoing: skip first EMBARGO_DAYS of test

test_idx = test_idx[EMBARGO_DAYS:]

if len(train_idx) < 100 or len(test_idx) < 20:

continue # not enough data for this fold

X_tr, X_te = X[train_idx], X[test_idx]

y_tr, y_te = y[train_idx], y[test_idx]

# Scale features (fit on train only — never on test)

X_tr = scaler.fit_transform(X_tr)
X_te = [Link](X_te)

model = RandomForestClassifier(n_estimators=100, random_state=42)

[Link](X_tr, y_tr)

prob = model.predict_proba(X_te)[:,1]
pred = (prob > 0.5).astype(int)

acc = accuracy_score(y_te, pred) * 100

auc = roc_auc_score(y_te, prob)

[Link]({"fold":fold, "acc":acc, "auc":auc,

"train_n":len(train_idx), "test_n":len(test_idx)})

Quant Pro | Advanced Quant Finance Course | Week 5 Page 40

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

print(f" Fold {fold}: Acc={acc:.1f}% AUC={auc:.3f}",

f" | Train={len(train_idx)} Test={len(test_idx)}")

import pandas as pd
res_df = [Link](results)
print(f"\nMean Accuracy : {res_df['acc'].mean():.1f}%")
print(f"Mean AUC : {res_df['auc'].mean():.3f}")
print(f"(Random baseline: Acc=50%, AUC=0.50)")

Day 41 Key Takeaways

• Random train-test split is wrong for time series — it allows training on future data, inflating all metrics.
• TimeSeriesSplit always ensures training data ends before test data begins — no future leakage at the
split level.
• Purging removes the last N rows from training where N equals your longest rolling feature window.
• Embargoing skips the first N rows of testing to avoid the overlap zone created by rolling feature
calculation.
• Always fit the scaler on training data only — then transform test data. Never fit_transform on combined
data.

Quant Pro | Advanced Quant Finance Course | Week 5 Page 41

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

DAY

Three ML Models Compared Properly

42 Logistic Regression, Random Forest, and XGBoost — what each does, when each wins

Why Three Models?

No single ML model is always best. Each has different strengths and makes different assumptions. A good
ML workflow always compares multiple models on the same data and picks the one that genuinely
generalises best — not the one that looks best on training data. This day builds all three, compares them
honestly, and explains exactly what each one is doing inside.

Model 1 — Logistic Regression

Logistic Regression is the simplest of the three. It fits a weighted sum of your features, passes it through a
sigmoid function, and outputs a probability between 0 and 1. It is fast, interpretable, and often competitive.
The feature coefficients directly tell you which features matter and in which direction.

LOGISTIC REGRESSION — HOW IT WORKS

Step 1: Calculate weighted sum of features

z = w1*RSI + w2*ret_5d + w3*vol_ratio + ... + bias

Step 2: Pass through sigmoid to get probability

P(UP tomorrow) = 1 / (1 + exp(-z))

Step 3: Classify based on threshold

If P > 0.5 -> predict UP (1)
If P < 0.5 -> predict DOWN (0)

STRENGTHS: Fast, interpretable, rarely overfits badly

WEAKNESSES: Cannot capture non-linear relationships
WHEN IT WINS: When signal is linear — e.g. higher RSI
consistently predicts higher returns (rare in markets)

Model 2 — Random Forest

Random Forest builds many decision trees, each trained on a random subset of data and features. The
final prediction is the majority vote across all trees. This ensemble approach reduces variance dramatically
compared to a single tree and handles non-linear relationships naturally.

RANDOM FOREST — HOW IT WORKS

Build 100 trees, each trained on:

- Random 80% of training rows (bootstrap sample)
- Random 60% of features at each split

Quant Pro | Advanced Quant Finance Course | Week 5 Page 42

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

Each tree votes: UP or DOWN

Final prediction: majority vote across all 100 trees

STRENGTHS: Handles non-linearity, robust to outliers,

built-in feature importance, rarely overfits badly
WEAKNESSES: Slower than LogReg, harder to interpret individual trees
WHEN IT WINS: Complex non-linear patterns, many features

Model 3 — XGBoost (Gradient Boosting)

XGBoost builds trees sequentially. Each new tree specifically corrects the errors made by all previous trees.
This makes it extremely powerful — and also prone to overfitting if you are not careful. It is typically the
highest accuracy model but needs more careful tuning.

XGBOOST — HOW IT WORKS

Tree 1: Predict from features. Get predictions and errors.

Tree 2: Fit a tree on the ERRORS from Tree 1.
Tree 3: Fit a tree on the ERRORS from Tree 1+2.
...repeat 100 times...
Final: sum of all 100 trees = prediction

Each tree corrects the mistakes of all previous trees.

Result: very low bias — captures subtle patterns.

STRENGTHS: Usually highest accuracy, handles mixed features

WEAKNESSES: Overfits easily, needs tuning, slow to train
WHEN IT WINS: Large datasets, complex interactions, competitions

Python Code — Day 42: Three Models Side by Side

import numpy as np, pandas as pd
from sklearn.linear_model import LogisticRegression
from [Link] import RandomForestClassifier, GradientBoostingClassifier
from [Link] import StandardScaler
from sklearn.model_selection import TimeSeriesSplit
from [Link] import roc_auc_score, accuracy_score
from [Link] import calibration_curve
import [Link] as plt

MODELS = {
"Logistic Regression" : LogisticRegression(max_iter=1000, C=0.1),
"Random Forest" : RandomForestClassifier(n_estimators=100,
max_depth=6, random_state=42),
"Gradient Boosting" : GradientBoostingClassifier(n_estimators=100,
max_depth=4,
learning_rate=0.05,
random_state=42),
}

tscv = TimeSeriesSplit(n_splits=5)

Quant Pro | Advanced Quant Finance Course | Week 5 Page 43

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

all_results = {name: [] for name in MODELS}

for fold, (tr_idx, te_idx) in enumerate([Link](X), 1):

# Purge and embargo
tr_idx = tr_idx[tr_idx < tr_idx[-1]-20]
te_idx = te_idx[5:]
if len(tr_idx) < 100: continue

X_tr, y_tr = X[tr_idx], y[tr_idx]

X_te, y_te = X[te_idx], y[te_idx]

sc = StandardScaler()
X_tr_s = sc.fit_transform(X_tr)
X_te_s = [Link](X_te)

for name, model in [Link]():

[Link](X_tr_s, y_tr)
prob = model.predict_proba(X_te_s)[:,1]
pred = (prob > 0.5).astype(int)
all_results[name].append({
"acc" : accuracy_score(y_te, pred)*100,
"auc" : roc_auc_score(y_te, prob),
})

print("=== MODEL COMPARISON (5-Fold TimeSeriesSplit) ===")

print(f" {"Model":<25} {"Acc":>8} {"AUC":>8}")
print("-" * 45)
for name, res in all_results.items():
df_r = [Link](res)
print(f" {name:<25} {df_r['acc'].mean():>7.1f}%",
f"{df_r['auc'].mean():>8.3f}")
print("-" * 45)
print(f" {"Random baseline":<25} {"50.0%":>8} {"0.500":>8}")

# Calibration: does "60% probability" actually mean 60%?

fig, axes = [Link](1,3,figsize=(14,5))
sc_f = StandardScaler()
X_tr_f = sc_f.fit_transform(X[:int(len(X)*0.7)])
X_te_f = sc_f.transform(X[int(len(X)*0.7):])
y_te_f = y[int(len(y)*0.7):]

for ax, (name, model) in zip(axes, [Link]()):

[Link](X_tr_f, y[:int(len(y)*0.7)])
prob = model.predict_proba(X_te_f)[:,1]
frac_pos, mean_pred = calibration_curve(y_te_f, prob, n_bins=8)
[Link](mean_pred, frac_pos, "o-", color="royalblue", lw=2)
[Link]([0,1],[0,1],"--",color="gray",lw=1,label="Perfect calibration")
ax.set_title(name, fontweight="bold", fontsize=9)
ax.set_xlabel("Predicted probability")
ax.set_ylabel("Actual fraction positive")
[Link](fontsize=7); [Link](True,alpha=0.3)

[Link]("Calibration Curves — Does 60% Mean 60%?",

Quant Pro | Advanced Quant Finance Course | Week 5 Page 44

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

fontweight="bold")
plt.tight_layout()
[Link]("calibration_curves.png", dpi=150, bbox_inches="tight")
[Link]()

Day 42 Key Takeaways

• Logistic Regression is fast and interpretable — its coefficients tell you exactly what each feature
contributes.
• Random Forest handles non-linearity naturally and is robust — a safe default for financial ML.
• Gradient Boosting is typically highest accuracy but overfits easily — always validate carefully with
time-series CV.
• Calibration curves show whether model probabilities are reliable. A model that says "70%" should be
right 70% of the time.
• Never pick the model that looks best on training data. Pick the one with best OOS performance across
CV folds.

Quant Pro | Advanced Quant Finance Course | Week 5 Page 45

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

DAY

Model Evaluation — Beyond Accuracy

43 Precision, recall, ROC-AUC, and why 55% accuracy can be worth millions

Why Accuracy is a Misleading Metric in Finance

Suppose the market goes UP 53% of days and DOWN 47% of days. A model that predicts UP every single
day without looking at any features would have 53% accuracy. That is useless — it has no predictive power
at all. Accuracy treats every error equally. In trading, errors are not equal. Missing a 5% gain and triggering
a 5% loss are completely different outcomes. You need better metrics.

THE FOUR METRICS YOU MUST KNOW

PREDICTED: DOWN PREDICTED: UP

ACTUAL: DOWN True Neg (TN) False Pos (FP)
ACTUAL: UP False Neg (FN) True Pos (TP)

Accuracy = (TP + TN) / All <- misleading alone

Precision = TP / (TP + FP) <- of all UP signals, how many were right?
Recall = TP / (TP + FN) <- of all actual UP days, how many caught?
F1 Score = 2 x Precision x Recall / (P + R) <- balance of both

IN TRADING:
High Precision = few false BUY signals (good for cost-sensitive strategies)
High Recall = catch most UP days (good if missing gains is costly)

ROC-AUC: ranking metric — does the model rank UP days above DOWN days?
AUC = 0.50: random. AUC = 0.55: small but real edge. AUC = 0.60: strong.
In finance, AUC of 0.55+ is genuinely useful and hard to achieve.

Threshold Tuning — Adjusting the Decision Boundary

By default, models predict UP when probability exceeds 0.50. But you can change this threshold. A higher
threshold (say 0.60) means the model only signals BUY when very confident — fewer trades but higher
precision. A lower threshold captures more UP days but generates more false signals. The right threshold
depends on your transaction cost structure.

Python Code — Day 43: Full Evaluation Dashboard

import numpy as np, pandas as pd
import [Link] as plt
from [Link] import (precision_score, recall_score, f1_score,
roc_auc_score, roc_curve, confusion_matrix, accuracy_score)
from [Link] import GradientBoostingClassifier
from [Link] import StandardScaler

Quant Pro | Advanced Quant Finance Course | Week 5 Page 46

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

# Use final fold from Day 42 for evaluation

split = int(len(X)*0.75)
sc = StandardScaler()
X_tr_s = sc.fit_transform(X[:split])
X_te_s = [Link](X[split:])
y_tr, y_te = y[:split], y[split:]

model = GradientBoostingClassifier(n_estimators=100,max_depth=4,
learning_rate=0.05,random_state=42)
[Link](X_tr_s, y_tr)
prob = model.predict_proba(X_te_s)[:,1]

# Evaluate at multiple thresholds

thresholds = [0.40, 0.45, 0.50, 0.55, 0.60, 0.65]
print(f" {"Threshold":>10} {"Acc":>7} {"Prec":>7} {"Rec":>7}",
f"{"F1":>7} {"Trades":>8}")
print("-"*55)
for thr in thresholds:
pred = (prob >= thr).astype(int)
n_trades = [Link]()
if n_trades == 0: continue
print(f" {thr:>10.2f}",
f"{accuracy_score(y_te,pred)*100:>6.1f}%",
f"{precision_score(y_te,pred,zero_division=0)*100:>6.1f}%",
f"{recall_score(y_te,pred)*100:>6.1f}%",
f"{f1_score(y_te,pred,zero_division=0)*100:>6.1f}%",
f"{n_trades:>8}")

auc = roc_auc_score(y_te, prob)

print(f"\nROC-AUC: {auc:.4f}")

# Plots
fig, axes = [Link](1,3,figsize=(14,5))

# 1. ROC Curve
fpr, tpr, _ = roc_curve(y_te, prob)
axes[0].plot(fpr, tpr, color="royalblue", lw=2, label=f"AUC={auc:.3f}")
axes[0].plot([0,1],[0,1],"--",color="gray")
axes[0].set_title("ROC Curve", fontweight="bold")
axes[0].set_xlabel("False Positive Rate")
axes[0].set_ylabel("True Positive Rate")
axes[0].legend(); axes[0].grid(True,alpha=0.3)

# 2. Precision-Recall vs Threshold
precs = [precision_score(y_te,(prob>=t).astype(int),zero_division=0) for t in thresholds]
recs = [recall_score(y_te,(prob>=t).astype(int)) for t in thresholds]
axes[1].plot(thresholds, precs, "o-", color="seagreen", lw=2, label="Precision")
axes[1].plot(thresholds, recs, "s-", color="tomato", lw=2, label="Recall")
axes[1].set_title("Precision & Recall vs Threshold", fontweight="bold")
axes[1].set_xlabel("Threshold")
axes[1].legend(); axes[1].grid(True,alpha=0.3)

# 3. Confusion Matrix at 0.55 threshold

Quant Pro | Advanced Quant Finance Course | Week 5 Page 47

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

pred_55 = (prob >= 0.55).astype(int)

cm = confusion_matrix(y_te, pred_55)
axes[2].imshow(cm, cmap="Blues")
axes[2].set_xticks([0,1]); axes[2].set_yticks([0,1])
axes[2].set_xticklabels(["Pred DOWN","Pred UP"])
axes[2].set_yticklabels(["Act DOWN","Act UP"])
for i in range(2):
for j in range(2):
axes[2].text(j,i,str(cm[i,j]),ha="center",va="center",
fontsize=14, fontweight="bold")
axes[2].set_title("Confusion Matrix (threshold=0.55)", fontweight="bold")

[Link]("Model Evaluation Dashboard", fontweight="bold")

plt.tight_layout()
[Link]("model_eval.png", dpi=150, bbox_inches="tight")
[Link]()

Day 43 Key Takeaways

• Accuracy is misleading when classes are imbalanced — a model predicting UP every day can have 53%
accuracy.
• ROC-AUC measures ranking quality — does the model score UP days above DOWN days? AUC 0.55+
is real edge in finance.
• Precision = how often your BUY signal is correct. Recall = how many UP days you capture. Choose
based on your costs.
• Threshold tuning lets you trade off precision vs recall — raise it to reduce false signals, lower it to
capture more.
• Always show a confusion matrix — it shows exactly which type of error your model makes most often.

Quant Pro | Advanced Quant Finance Course | Week 5 Page 48

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

DAY

SHAP — Making the Black Box Transparent

44 Understand exactly why your model made every single prediction it ever made

The Interpretability Problem

Your gradient boosting model is making predictions. But why? Which features drove the UP prediction on
Monday? Did it rely on RSI? Momentum? Volume? Without this understanding you cannot trust the model,
fix it when it breaks, or explain it to a risk manager or regulator. SHAP values solve this completely — for
every single prediction.

> REAL-WORLD ANALOGY

A judge announces a verdict: Guilty. Fine. But why? Which evidence mattered? Without reasoning, the verdict
is not useful — you cannot appeal it, learn from it, or trust the system. SHAP values are the reasoning behind
every ML verdict. They tell you: "For this specific day, the model predicted UP because RSI contributed
+2.3%, momentum contributed +1.8%, and volume worked against it at -0.9%." Every prediction, fully
explained.

What SHAP Values Actually Measure

SHAP VALUES — THE CONCEPT

Model base rate: market goes UP 53% of days -> base prediction = 53%

For a specific day, SHAP breaks down the prediction:

Base rate (53%) +
RSI=28 (oversold) -> +8.2% <- strong UP contribution
ret_5d=-3.2% (falling) -> -4.1% <- negative contribution
vol_ratio=1.8 (vol spike) -> -2.3% <- negative (vol rising = risk)
dist_sma20=-2.1% (below) -> +3.7% <- positive (mean reversion)
sma_cross=0 (bearish) -> -1.8% <- negative
... other features ... -> +1.1%
Final prediction = 58.8% probability of UP

SHAP values sum up to: final prediction - base rate

Total = +5.8% (model is slightly more bullish than baseline)

You can see EXACTLY which features drove this prediction.

Python Code — Day 44: SHAP Explanations

# Install: pip install shap --break-system-packages
import shap
import numpy as np, pandas as pd
import [Link] as plt

Quant Pro | Advanced Quant Finance Course | Week 5 Page 49

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

from [Link] import GradientBoostingClassifier

from [Link] import StandardScaler

# Fit model on training data

split = int(len(X)*0.75)
sc = StandardScaler()
X_tr_s = sc.fit_transform(X[:split])
X_te_s = [Link](X[split:])

model = GradientBoostingClassifier(n_estimators=100,max_depth=4,
learning_rate=0.05,random_state=42)
[Link](X_tr_s, y[:split])

# Compute SHAP values on test set

# TreeExplainer is fast and exact for tree-based models
explainer = [Link](model)
shap_values = explainer.shap_values(X_te_s)
# shap_values shape: (n_samples, n_features)
# Positive SHAP = feature pushes prediction towards UP
# Negative SHAP = feature pushes prediction towards DOWN

# ■■ PLOT 1: Global feature importance ■■■■■■■■■■■■■■■

# Shows which features matter most across all predictions
[Link](figsize=(10, 6))
shap.summary_plot(shap_values, X_te_s,
feature_names=FEATURE_COLS,
plot_type="bar",
show=False)
[Link]("SHAP Global Feature Importance", fontweight="bold")
plt.tight_layout()
[Link]("shap_global.png", dpi=150, bbox_inches="tight")
[Link]()

# ■■ PLOT 2: SHAP summary dot plot ■■■■■■■■■■■■■■■■■■■

# Each dot is one prediction — colour shows feature value
[Link](figsize=(10, 7))
shap.summary_plot(shap_values, X_te_s,
feature_names=FEATURE_COLS,
show=False)
[Link]("SHAP Summary — Feature Impact Distribution", fontweight="bold")
plt.tight_layout()
[Link]("shap_summary.png", dpi=150, bbox_inches="tight")
[Link]()

# ■■ PLOT 3: Waterfall for one specific prediction ■■■

# Shows exactly why the model predicted what it did for day 0
sample_idx = 0
sv = shap_values[sample_idx]
base = explainer.expected_value

# Sort features by absolute SHAP contribution

feat_imp = sorted(zip(FEATURE_COLS, sv), key=lambda x: abs(x[1]), reverse=True)
names = [f[0] for f in feat_imp[:10]]

Quant Pro | Advanced Quant Finance Course | Week 5 Page 50

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

values= [f[1] for f in feat_imp[:10]]

fig, ax = [Link](figsize=(10, 5))

colors = ["seagreen" if v > 0 else "tomato" for v in values]
[Link](names[::-1], values[::-1], color=colors[::-1])
[Link](0, color="black", lw=0.8)
ax.set_title(f"SHAP Waterfall — Prediction Explanation (Day {sample_idx})",
fontweight="bold")
ax.set_xlabel("SHAP Value (positive = pushes UP)")
[Link](True, alpha=0.3, axis="x")
plt.tight_layout()
[Link]("shap_waterfall.png", dpi=150, bbox_inches="tight")
[Link]()

print(f"Base rate : {base:.3f} ({base*100:.1f}% UP)")

print("Top 5 features driving this prediction:")
for name, val in feat_imp[:5]:
direction = "UP" if val > 0 else "DOWN"
print(f" {name:<20} {val:+.4f} -> pushes {direction}")

Day 44 Key Takeaways

• SHAP values explain every ML prediction — which features pushed it UP and which pushed it DOWN,
with exact magnitudes.
• Global SHAP importance (bar chart) shows which features matter most across all predictions in the
dataset.
• The summary dot plot shows how each feature affects predictions — high feature values on the right
side mean positive impact.
• Waterfall plots give per-prediction explanations — essential for risk managers and regulators who ask
"why this trade?"
• If SHAP shows your model relies heavily on vol_ratio or RSI, you can investigate whether that
relationship is economically valid.

Quant Pro | Advanced Quant Finance Course | Week 5 Page 51

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

DAY

NLP and Sentiment Signals

45 Reading news headlines and earnings calls to add a completely new dimension to your ML model

Why Text Data Matters in Finance

Price and volume data is what everybody sees. News and earnings call text is also public — but most
systematic traders do not process it in real-time. When a CEO says "we are seeing strong demand across
all segments" in an earnings call, that is a signal. When a headline says "RBI signals rate pause ahead of
budget," that is a signal. Sentiment features add a genuinely different dimension to your ML model that is
orthogonal to price-based features.

> REAL-WORLD ANALOGY

Imagine predicting whether a restaurant will be packed tonight using only temperature data from last night —
that is your price-based ML model. Now imagine also reading today's reviews and social media posts about
the restaurant. You have added a completely different source of information. Sentiment signals are the
reviews and social media of the stock market — they reflect human emotion and expectation, not just past
price movement.

Two Sentiment Tools — VADER and FinBERT

VADER vs FinBERT

VADER (rule-based, fast):

Pros: Instant, no GPU, works on any sentence
Output: compound score -1.0 (very negative) to +1.0 (very positive)
Best for: Twitter, headlines, short news snippets
Limit: Not trained on financial text specifically

FinBERT (deep learning, slower):

Pros: Trained specifically on financial news (Reuters, Bloomberg)
Output: positive / neutral / negative with probabilities
Best for: Earnings call transcripts, analyst reports, press releases
Limit: Needs transformers library, slightly slower

THE SHIFT RULE FOR SENTIMENT:

If the earnings call is at 6 PM after market close,
the signal applies to TOMORROW's open — shift(1) still required.
Never use same-day announcement to predict same-day return.

Python Code — Day 45: VADER Sentiment Pipeline

# Install: pip install vaderSentiment transformers --break-system-packages
from [Link] import SentimentIntensityAnalyzer

Quant Pro | Advanced Quant Finance Course | Week 5 Page 52

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

import pandas as pd, numpy as np

analyzer = SentimentIntensityAnalyzer()

# ■■ Example headlines (in practice: scrape from NSE, MoneyControl)

headlines = [
{"date":"2024-01-15", "symbol":"TCS",
"text":"TCS Q3 results beat estimates, strong deal wins announced"},
{"date":"2024-01-16", "symbol":"TCS",
"text":"TCS cautious on discretionary spending, margins under pressure"},
{"date":"2024-01-17", "symbol":"TCS",
"text":"Analysts upgrade TCS after strong guidance commentary"},
{"date":"2024-01-18", "symbol":"TCS",
"text":"TCS faces headwinds from global IT budget cuts"},
{"date":"2024-01-19", "symbol":"TCS",
"text":"TCS wins Rs 2000 crore deal from European banking client"},
]

# Score each headline

for h in headlines:
scores = analyzer.polarity_scores(h["text"])
h["compound"] = scores["compound"] # -1.0 to +1.0
h["positive"] = scores["pos"]
h["negative"] = scores["neg"]
print(f" {h['date']} Score={h['compound']:+.3f} {h['text'][:55]}...")

sent_df = [Link](headlines)
sent_df["date"] = pd.to_datetime(sent_df["date"])

# Group by date — if multiple headlines, take the mean

daily_sent = sent_df.groupby("date")["compound"].mean().reset_index()
daily_sent.columns = ["DATE1", "sentiment_raw"]

# Rolling 3-day average sentiment (smoother signal)

daily_sent["sentiment_3d"] = daily_sent["sentiment_raw"].rolling(3).mean()

# APPLY SHIFT RULE: sentiment from date T predicts return on T+1

daily_sent["sentiment_raw"] = daily_sent["sentiment_raw"].shift(1)
daily_sent["sentiment_3d"] = daily_sent["sentiment_3d"].shift(1)

# Merge with price data

df["DATE1"] = pd.to_datetime(df["DATE1"])
df = [Link](daily_sent, on="DATE1", how="left")
df["sentiment_raw"] = df["sentiment_raw"].fillna(0) # 0=neutral if no news
df["sentiment_3d"] = df["sentiment_3d"].fillna(0)

# Add to feature set

FEATURE_COLS_WITH_SENT = FEATURE_COLS + ["sentiment_raw", "sentiment_3d"]
X_with_sent = df[FEATURE_COLS_WITH_SENT].values

print(f"\nFeature set expanded: {len(FEATURE_COLS)} -> {len(FEATURE_COLS_WITH_SENT)}")

print("Sentiment features added with shift(1) applied")
print("\nSentiment Stats:")
print(f" Mean: {df['sentiment_raw'].mean():+.3f}")
print(f" Std: {df['sentiment_raw'].std():.3f}")

Quant Pro | Advanced Quant Finance Course | Week 5 Page 53

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

print(f" % Positive days (sent > 0.05): {(df['sentiment_raw']>0.05).mean()*100:.1f}%")

Day 45 Key Takeaways

• VADER is fast and rule-based — ideal for screening many headlines quickly. Compound score -1 to +1.
• FinBERT is trained on financial text — more accurate for earnings calls, analyst reports, and RBI
statements.
• The shift rule applies to sentiment too — an announcement after market close predicts the next day, not
the same day.
• Fill missing sentiment values with 0 (neutral) when no news is available — absence of news is not
bearish.
• Sentiment features add orthogonal information to price features — they capture human emotion that
prices lag behind.

Quant Pro | Advanced Quant Finance Course | Week 5 Page 54

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

DAY

Regime-Aware ML

46 The same model should not be used in a bull market and a crash — here is how to separate them

Why One Model Is Never Enough

Your ML model was trained on data from all market conditions mixed together. But the patterns that predict
UP days in a bull trend are completely different from the patterns that predict UP days in a volatile crash. A
single model trying to learn all regimes simultaneously ends up doing a mediocre job in all of them.
Regime-aware ML trains separate models for each regime and switches between them based on the
current regime.

> REAL-WORLD ANALOGY

A Formula 1 driver uses one set of tyres in dry conditions and a completely different set in wet conditions.
Using wet tyres on a dry track loses you three seconds per lap. The tyres are the same driver — but the
strategy is completely different based on the conditions. Your ML model is the driver. The regime is the
weather. The model parameters are the tyre choice.

Simple Regime Detection Using India VIX

The simplest and most practical regime detector for Indian markets is India VIX. Low VIX (below 15) means
calm bull/flat market. High VIX (above 20) means fearful, volatile market. You can use this as a proxy
without any complex model.

REGIME CLASSIFICATION USING VIX

VIX < 15 -> LOW VOL regime (calm, trending market)

VIX 15-20 -> NORMAL regime (typical conditions)
VIX > 20 -> HIGH VOL regime (volatile, fearful market)

Strategy:
1. Download India VIX history from NSE website
2. Label each day with its regime
3. Train three separate ML models — one per regime
4. On prediction day: check current VIX, pick the right model

Results:
Low-vol model -> learns patterns valid in calm uptrends
High-vol model -> learns patterns valid in volatile/crisis periods
Normal model -> catches the in-between

Each model sees only the data from its own regime.
The patterns it learns are regime-specific and more accurate.

Quant Pro | Advanced Quant Finance Course | Week 5 Page 55

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

Python Code — Day 46: Regime-Aware ML Pipeline

import pandas as pd, numpy as np
from [Link] import GradientBoostingClassifier
from [Link] import StandardScaler
from [Link] import roc_auc_score

# ■■ Simulate India VIX (in practice: download from NSE website)

[Link](42)
n = len(df)
vix_base = 15 + 3*[Link](n).cumsum()*0.05
vix_base = [Link](vix_base, 10, 35)
df["VIX"] = vix_base

# ■■ Classify regime
df["regime"] = "normal"
[Link][df["VIX"] < 15, "regime"] = "low_vol"
[Link][df["VIX"] > 20, "regime"] = "high_vol"

print("Regime distribution:")
print(df["regime"].value_counts())

# ■■ Train one model per regime

REGIMES = ["low_vol", "normal", "high_vol"]
regime_models = {}
regime_scalers = {}
regime_aucs = {}

for regime in REGIMES:

mask = df["regime"] == regime
X_reg = X[[Link]]
y_reg = y[[Link]]

if len(X_reg) < 60:

print(f" {regime}: too few samples ({len(X_reg)}), skipping")
continue

split = int(len(X_reg) * 0.75)

sc = StandardScaler()
X_tr_s = sc.fit_transform(X_reg[:split])
X_te_s = [Link](X_reg[split:])

model = GradientBoostingClassifier(n_estimators=80, max_depth=3,

random_state=42)
[Link](X_tr_s, y_reg[:split])

prob = model.predict_proba(X_te_s)[:,1]
auc = roc_auc_score(y_reg[split:], prob) if len(set(y_reg[split:])) > 1 else 0.5

regime_models[regime] = model
regime_scalers[regime] = sc
regime_aucs[regime] = auc
print(f" {regime:<12}: n={len(X_reg):<5} OOS AUC={auc:.3f}")

# ■■ Regime-aware prediction function

def predict_with_regime(features_today, current_vix):

Quant Pro | Advanced Quant Finance Course | Week 5 Page 56

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

"""
features_today: 1D array of feature values for today
current_vix: current India VIX level
Returns: probability of UP tomorrow
"""
if current_vix < 15: regime = "low_vol"
elif current_vix > 20: regime = "high_vol"
else: regime = "normal"

if regime not in regime_models:

regime = "normal" # fallback

sc = regime_scalers[regime]
model = regime_models[regime]
feat = [Link](features_today.reshape(1,-1))
return model.predict_proba(feat)[0,1], regime

# ■■ Example: predict for today

today_features = X[-1]
today_vix = float(df["VIX"].iloc[-1])
prob, regime = predict_with_regime(today_features, today_vix)
print(f"\nToday's VIX : {today_vix:.1f}")
print(f"Detected regime: {regime}")
print(f"P(UP tomorrow) : {prob:.3f} ({prob*100:.1f}%)")
print(f"Signal : {'BUY' if prob > 0.55 else 'SELL' if prob < 0.45 else 'HOLD'}")

Day 46 Key Takeaways

• Market regimes have different statistical properties — patterns that work in bull markets fail in crashes.
• India VIX is the simplest and most practical regime proxy for Indian markets — below 15 is calm, above
20 is fearful.
• Train separate models per regime — each learns the patterns specific to its own market conditions.
• The regime-aware predict function checks current VIX, picks the right model, and returns regime-specific
probabilities.
• Always have a fallback regime (normal) in case a regime has insufficient training data for a reliable
model.

Quant Pro | Advanced Quant Finance Course | Week 5 Page 57

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

DAY

Project 4 — Complete ML Alpha Pipeline

47 Feature engineering + CV + three models + SHAP + sentiment + regime — all in one system

Project 4 Overview
Today you connect every component from Week 6 into one complete, production-ready ML alpha pipeline.
This is the second major system you are building — it sits alongside your Week 5 live dashboard and feeds
it with ML-generated signals. When complete, your GitHub will show a system that any junior quant
researcher would be proud of.

Module 1 Feature Engineering — 22 technical features with shift(1) enforced on all

Module 2 Sentiment Features — VADER on recent headlines, 3-day rolling, shift(1) applied

Module 3 Regime Detection — VIX-based regime label appended to each row

Module 4 TimeSeriesSplit CV with purging — 5 folds, honest OOS evaluation

Module 5 Three models trained — LogReg, Random Forest, Gradient Boosting

Module 6 Model selection — pick best by OOS AUC across CV folds

Module 7 Regime-aware prediction — use correct model based on current VIX

Module 8 SHAP explanations — global importance + waterfall for latest prediction

Module 9 Signal integration — combine ML probability with ADX regime filter from Week 5

Module 10 GitHub upload — week6/ folder with plots, README, [Link]

Project 4 Master Code

# PROJECT 4 — ML ALPHA PIPELINE — Quant Pro Week 6
import pandas as pd, numpy as np
import [Link] as plt
import [Link] as gridspec
from sklearn.linear_model import LogisticRegression
from [Link] import RandomForestClassifier, GradientBoostingClassifier
from [Link] import StandardScaler
from sklearn.model_selection import TimeSeriesSplit
from [Link] import roc_auc_score, accuracy_score
from [Link] import SentimentIntensityAnalyzer
import shap, warnings
[Link]("ignore")

# ■■ 1. Load and prepare data ■■■■■■■■■■■■■■■■■■■■■■■■■

df = pd.read_csv("[Link]")

Quant Pro | Advanced Quant Finance Course | Week 5 Page 58

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

df = df[df["SYMBOL"]=="TCS"].sort_values("DATE1").reset_index(drop=True)
df["DATE1"] = pd.to_datetime(df["DATE1"])

# ■■ 2. Build features (all shift-rule compliant) ■■■■■

for n in [1,5,10,20,60]:
df[f"ret_{n}d"] = df["CLOSE"].pct_change(n).shift(1)
for n in [5,20,60]:
df[f"vol_{n}d"] = df["CLOSE"].pct_change().rolling(n).std().shift(1)
df["vol_ratio"] = (df["vol_5d"] / df["vol_20d"]).shift(1)

sma20 = df["CLOSE"].rolling(20).mean()
sma50 = df["CLOSE"].rolling(50).mean()
df["dist_sma20"] = ((df["CLOSE"]-sma20)/sma20).shift(1)
df["dist_sma50"] = ((df["CLOSE"]-sma50)/sma50).shift(1)
df["sma_cross"] = (sma20 > sma50).astype(int).shift(1)

def rsi(s,p=14):
d=[Link](); g=[Link](lower=0).ewm(com=p-1,min_periods=p).mean()
l=(-d).clip(lower=0).ewm(com=p-1,min_periods=p).mean()
return (100-100/(1+g/l)).shift(1)
df["rsi_14"] = rsi(df["CLOSE"],14)

df["vol_chg"] = df["TOTTRDQTY"].pct_change(5).shift(1)
df["vol_vs_avg"]= (df["TOTTRDQTY"]/df["TOTTRDQTY"].rolling(20).mean()).shift(1)

# ■■ 3. Sentiment features ■■■■■■■■■■■■■■■■■■■■■■■■■■■■

# (Placeholder: in practice scrape from NSE announcements)
[Link](42)
df["sentiment"] = [Link](0, 0.15, len(df))
df["sentiment"] = df["sentiment"].shift(1) # shift rule

# ■■ 4. Regime (VIX proxy) ■■■■■■■■■■■■■■■■■■■■■■■■■■■■

vix = 15 + 3*[Link](len(df)).cumsum()*0.05
df["VIX"] = [Link](vix, 10, 35)
df["regime"] = "normal"
[Link][df["VIX"] < 15, "regime"] = "low_vol"
[Link][df["VIX"] > 20, "regime"] = "high_vol"

# ■■ 5. Target and clean ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■

df["target"] = (df["CLOSE"].pct_change().shift(-1) > 0).astype(int)

FEATS = ["ret_1d","ret_5d","ret_10d","ret_20d","ret_60d",
"vol_5d","vol_20d","vol_60d","vol_ratio",
"dist_sma20","dist_sma50","sma_cross","rsi_14",
"vol_chg","vol_vs_avg","sentiment"]

df = [Link]()
X = df[FEATS].values
y = df["target"].values

# ■■ 6. TimeSeriesSplit CV with purging ■■■■■■■■■■■■■■■

tscv = TimeSeriesSplit(n_splits=5)
MODELS = {
"LogReg" : LogisticRegression(max_iter=1000,C=0.1),
"RF" : RandomForestClassifier(n_estimators=100,max_depth=6,random_state=42),

Quant Pro | Advanced Quant Finance Course | Week 5 Page 59

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

"GBM" : GradientBoostingClassifier(n_estimators=100,max_depth=4,
learning_rate=0.05,random_state=42),
}
cv_aucs = {n:[] for n in MODELS}

for _, (tr_idx, te_idx) in enumerate([Link](X)):

tr_idx = tr_idx[tr_idx < tr_idx[-1]-20]
te_idx = te_idx[5:]
if len(tr_idx)<80: continue
sc = StandardScaler()
Xtr = sc.fit_transform(X[tr_idx]); Xte = [Link](X[te_idx])
for nm, m in [Link]():
[Link](Xtr, y[tr_idx])
p = m.predict_proba(Xte)[:,1]
if len(set(y[te_idx]))>1:
cv_aucs[nm].append(roc_auc_score(y[te_idx],p))

# ■■ 7. Pick best model ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■

mean_aucs = {nm: [Link](v) for nm,v in cv_aucs.items() if v}
best_name = max(mean_aucs, key=mean_aucs.get)
print("=== CV RESULTS ===")
for nm, auc in mean_aucs.items():
print(f" {nm:<10}: AUC={auc:.4f}",
" <- BEST" if nm==best_name else "")

# ■■ 8. Train best model on full data ■■■■■■■■■■■■■■■■■

sc_final = StandardScaler()
X_scaled = sc_final.fit_transform(X)
best_model = MODELS[best_name]
best_model.fit(X_scaled, y)

# ■■ 9. SHAP explanations ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■

explainer = [Link](best_model)
shap_vals = explainer.shap_values(X_scaled[-252:]) # last year

# ■■ 10. Today's prediction ■■■■■■■■■■■■■■■■■■■■■■■■■■■

today_vix = float(df["VIX"].iloc[-1])
today_prob= float(best_model.predict_proba(X_scaled[-1:])[0,1])
regime = ("low_vol" if today_vix<15 else
"high_vol" if today_vix>20 else "normal")
signal = "BUY" if today_prob>0.55 else "SELL" if today_prob<0.45 else "HOLD"

print(f"\n=== TODAY'S ML SIGNAL ===")

print(f"Model : {best_name}")
print(f"VIX : {today_vix:.1f} ({regime} regime)")
print(f"P(UP) : {today_prob:.3f} ({today_prob*100:.1f}%)")
print(f"Signal : {signal}")

# ■■ 11. Dashboard ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■

fig = [Link](figsize=(14,10))
gs = [Link](2,2,hspace=0.4,wspace=0.35)

# Model comparison

Quant Pro | Advanced Quant Finance Course | Week 5 Page 60

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

ax1=fig.add_subplot(gs[0,0])
[Link](mean_aucs.keys(), mean_aucs.values(),
color=["royalblue","seagreen","tomato"])
[Link](0.5, color="gray",linestyle="--",lw=1, label="Random=0.50")
ax1.set_title("CV AUC by Model",fontweight="bold")
ax1.set_ylabel("ROC-AUC"); [Link](); [Link](True,alpha=0.3,axis="y")

# SHAP feature importance

ax2=fig.add_subplot(gs[0,1])
mean_abs_shap = [Link](shap_vals).mean(axis=0)
top_idx = [Link](mean_abs_shap)[-10:]
[Link]([FEATS[i] for i in top_idx],
mean_abs_shap[top_idx], color="royalblue")
ax2.set_title("Top 10 SHAP Features",fontweight="bold")
[Link](True,alpha=0.3,axis="x")

# Today prediction gauge

ax3=fig.add_subplot(gs[1,0])
[Link]("off")
[Link](0.5, 0.7, f"P(UP Tomorrow)",ha="center",va="center",
fontsize=14,transform=[Link])
color = "seagreen" if today_prob>0.55 else "tomato" if today_prob<0.45 else "gray"
[Link](0.5, 0.45, f"{today_prob*100:.1f}%",ha="center",va="center",
fontsize=36,fontweight="bold",color=color,transform=[Link])
[Link](0.5, 0.25, f"Signal: {signal} | Regime: {regime}",
ha="center",va="center",fontsize=12,transform=[Link])

# Rolling AUC
ax4=fig.add_subplot(gs[1,1])
probs_all = best_model.predict_proba(X_scaled)[:,1]
rolling_auc=[]
for i in range(60, len(y)):
if len(set(y[i-60:i]))>1:
rolling_auc.append(roc_auc_score(y[i-60:i], probs_all[i-60:i]))
else: rolling_auc.append(0.5)
[Link](rolling_auc, color="royalblue",lw=1.5)
[Link](0.5,color="gray",linestyle="--",lw=1)
ax4.set_title("60-Day Rolling AUC",fontweight="bold")
ax4.set_ylabel("AUC"); [Link](True,alpha=0.2)

[Link](f"ML Alpha Pipeline — {best_name} | OOS AUC={mean_aucs[best_name]:.3f}",

fontsize=13,fontweight="bold")
[Link]("week6_ml_dashboard.png",dpi=150,bbox_inches="tight")
[Link]()
print("\nSaved: week6_ml_dashboard.png")

Week 6 Completion Checklist

• I understand data leakage and the shift rule — every feature uses shift(1) before entering the model.

Quant Pro | Advanced Quant Finance Course | Week 5 Page 61

QUANT PRO WEEK 5 | Days 31-39

Advanced Quant Finance Course Production Foundation

• I use TimeSeriesSplit with purging and embargoing — no random splits, no future leakage at the split
level.
• I have trained and compared LogReg, Random Forest, and Gradient Boosting on the same CV setup.
• I use ROC-AUC as the primary metric — not accuracy — and understand precision vs recall tradeoffs.
• I can generate SHAP explanations and explain any model prediction in plain English to a risk manager.
• I have added sentiment features from news headlines using VADER with shift(1) applied.
• I use regime detection (VIX proxy) to select the right model for current market conditions.
• Project 4 ML pipeline is uploaded to GitHub with plots, README, and [Link].

# GOLDEN RULE
The ML model from Day 26 in the original course would fail in any serious interview because it had data
leakage, wrong CV, and no interpretability. The system you just built has none of those problems. This is what
separates a junior quant from a practitioner.

Gre
Options Mastery — From Formula to Trading

Quant Pro | Advanced Quant Finance Course | Week 5 Page 62