■ Probability

Complete Study
· Statistics · Hypothesis Guide
· Machine Learning

UNIT 3 — 12 Marks UNIT 2 — 4 Marks

Data, Big Data, Preprocessing, Transformations, ML Workflow, Supervised Learning,

Descriptive Stats, Probability, Bayes Theorem, KNN, Decision Tree, SVM,
Bayesian Networks, Hypothesis Testing, Naive Bayes, Linear/Logistic/Polynomial
Concept Learning, Hypothesis Space, Regression, Random Forest,
Find-S, Bias-Variance, Regression Confusion Matrix, Python Implementation

Unit # Topic

U3 1 Understanding Data — Types, Big Data 6 V's

U3 2 Big Data Architecture & ML Process

U3 3 Problem Definition (T-E-P) & Data Collection

U3 4 Data Preprocessing — Cleaning, Binning, Sampling, Scaling

U3 5 Data Transformations — Min-Max & Z-Score (with numericals)

U3 6 Data Types & Descriptive Statistics

U3 7 Univariate, Bivariate & Multivariate Analysis (with numericals)

U3 8 Probability — Marginal, Joint, Conditional

U3 9 Random Variables, PDF, CDF, Gaussian Distribution

U3 10 Bayes Theorem & Bayesian Networks (with 3 solved examples)

U3 11 Hypothesis & Hypothesis Testing — Z, t, Chi-Square (with all numericals)

U3 12 Concept Learning, Hypothesis Space, Find-S Algorithm

U3 13 Induction Bias, Bias-Variance Trade-off

U3 14 Python — Preprocessing & Hypothesis Space

U2 15 ML Workflow — Train/Val/Test Split, Forms of Learning

U2 16 KNN, Decision Tree, SVM, Naive Bayes

U2 17 Linear Regression, Multiple & Polynomial Regression (with numericals)

U2 18 Validation Metrics — MAE, MSE, RMSE (with numerical)

U2 19 Logistic Regression & Random Forest (with numerical)

U2 20 Confusion Matrix & Evaluation Metrics (with numerical)

U2 21 Python — Complete ML Pipeline

UNIT 3 — 12 Marks

■ Topic 1: Understanding Data & Big Data

1.1 What is Data?
Data is a collection of facts. All facts are data. Data can be:
• Human interpretable — can be understood directly by humans
• Diffused data — data spread across multiple sources
• Operational data — data used in day-to-day operations
• Non-operational data — historical or reference data

1.2 Big Data — Definition

Big Data is a larger dataset whose volume is much larger than 'Small data'. It is characterized by 6 V's:

V Name Description Example

1 Volume The sheer amount/size of data generated every second

Terabytes of Facebook posts daily

2 Velocity The speed at which data is generated, collected and processed

Real-time stock prices, sensor data

3 Variety Different types and formats of data (structured, unstructured,

Text, images,
semi-structured)
videos, JSON, tables

4 Veracity Trustworthiness/accuracy of the data — handling uncertainty

Fake news, noisy sensor readings

5 Validity Relevance and correctness of data for its intended use

Outdated records being irrelevant

6 Value The worth/usefulness extracted from the data after processing

Insights driving business decisions

1.3 Types of Data in Big Data

Type Description Example

Structured Organized in rows and columns; predefined schema; easily

Relational
searchable
databases, Excel sheets, SQL tables

Unstructured No predefined format or schema; hard to analyze withText

traditional
files, images,
tools videos, audio, social media posts

Semi-structured Mix of structured and unstructured; uses tags/markersJSON,

to separate
XML, elements
HTML, Emails with metadata

■ Topic 2: Big Data Architecture & Machine Learning Process

2.1 Big Data Architecture Layers
There are four main Big Data architecture layers:

Layer Name Function

1 Data Ingestion Collecting and importing raw data from various sources (databases, APIs, sensors, files)

2 Data Processing Cleaning, transforming and analyzing the raw data (batch or stream processing)

3 Data Storage Storing processed data in data lakes, warehouses, NoSQL/SQL databases

4 Data Visualization Presenting insights via dashboards, charts, reports

2.2 Big Data Processing Cycle

The Big Data processing cycle involves the following steps:
• Step 1: Data collection — gathering raw data from sources
• Step 2: Data preprocessing — cleaning and transforming data
• Step 3: Applications of ML algorithm — applying learning models
• Step 4: Interpretation & Visualization — understanding and displaying results

2.3 Process of Machine Learning

ML is the process that starts with defining the data and ends with a model with some defined level of accuracy. The
steps are:

Step Stage Description

1 Define the Problem Identify task T, experience E, and performance measure P

2 Data Collection Gather relevant, high-quality data from reliable sources

3 Data Preparation Clean, preprocess, normalize, encode — make data model-ready

4 Split Data Divide into Training, Validation, and Test sets

5 Algorithm Selection Choose best algorithm based on problem type (classification/regression)

6 Performance Evaluation
Measure accuracy, precision, recall, etc. on test data

■ Topic 3: Problem Definition (T-E-P Framework) & Data Collection

3.1 Defining the Problem: T-E-P Framework
A problem in ML is formally defined by three components:

Component Full Name Definition Example (Classify Human Image)

T Task The specific thing the model must do Classify an image: human or not human

E Experience The training data the model learns fromImages labeled 'human' or 'not human'

P Performance How we measure how well the model performs

Error rate — % of wrong predictions; lower = better accura
■ Note: Lower error rate leads to higher accuracy. The model improves by minimizing error on training experience.

3.2 Data Collection

Data can be collected from the following sources:
• Open/public data sources (Kaggle, UCI ML Repository, [Link])
• Social media (Twitter API, Facebook Graph API)
• Academic research databases
• Government or institutional data portals
Properties of Good Data:

Property Meaning

Timeliness Data is up-to-date and collected within a relevant time frame

Relevancy Data is directly related to the problem being solved

Accuracy Data correctly represents the real-world values

Reliability Data is consistent and can be depended upon

Knowledge The data collector understands what the data represents

■ Topic 4: Data Preprocessing

4.1 Why Preprocess? — Dirty Data
In the real world, available data is 'dirty'. Dirty data means:
• Incomplete data — missing attributes or attribute values
• Outlier data — data that deviates significantly from normal values
• Inconsistent values — conflicting data across records
• Inaccurate data — data that does not reflect reality
• Missing values — NULL or blank fields
• Duplicate data — same record appearing multiple times

4.2 Cleaning
Cleaning involves identifying and rectifying errors or inconsistencies:
• Handling missing values: Replace with mean/median/mode, or remove the record
• Removing duplicates: Identify and drop repeated records
• Correcting inconsistent data: Standardize formats (e.g., date formats, capitalization)
• Dealing with outliers: Use binning, smoothing, or removal techniques

4.3 Binning Techniques (Smoothing Outliers)

Binning groups sorted data into 'bins' and replaces values to smooth noise.

Method How it works Example result for Bin = {12, 14, 19}

Smoothing by Bin Mean mean= (12+14+19)/3 = 15 → {15, 15, 15}

Replace each value in bin with the bin'sMean

Smoothing by Bin Median median= 14 → {14, 14, 14}

Replace each value in bin with the bin'sMedian

Smoothing by Bin Boundaries Replace each value with closest boundary

Min=12,
(min or max of→bin)
Max=19 {12, 12, 19}
NUMERICAL — S = (12, 14, 19, 22, 24, 26, 28, 31, 32) — Apply Binning Techniques:
Sort & split into 3 equal bins:
Bin 1 = {12, 14, 19} Bin 2 = {22, 24, 26} Bin 3 = {28, 31, 32}

Smoothing by Bin Means:

Bin1 mean = (12+14+19)/3 = 15 → {15, 15, 15}
Bin2 mean = (22+24+26)/3 = 24 → {24, 24, 24}
Bin3 mean = (28+31+32)/3 = 30.33 → {30.33, 30.33, 30.33}

Smoothing by Bin Medians:

Bin1 median = 14 → {14, 14, 14}
Bin2 median = 24 → {24, 24, 24}
Bin3 median = 31 → {31, 31, 31}

Smoothing by Bin Boundaries (Min/Max):

Bin1: min=12, max=19 → 12→12, 14→12, 19→19 → {12, 12, 19}
Bin2: min=22, max=26 → 22→22, 24→22, 26→26 → {22, 22, 26}
Bin3: min=28, max=32 → 28→28, 31→32, 32→32 → {28, 32, 32}

4.4 Formatting
• Converting categorical data: Label Encoding or One-Hot Encoding
• Date and time handling: Convert to standard formats (YYYY-MM-DD)

4.5 Sampling
Technique Description

Random Sampling Select a random subset of data — equal probability for all records

Over-sampling Increase samples of minority class (e.g., in imbalanced datasets)

Under-sampling Reduce samples of majority class to balance the dataset

Bootstrapping Random sampling WITH replacement — used for estimating statistics

4.6 Decomposition (Dimensionality Reduction)

• PCA (Principal Component Analysis): Transforms data into a smaller set of uncorrelated components
capturing maximum variance
• SVD (Singular Value Decomposition): Decomposes matrix into U × Σ × V^T; used in recommendation
systems and NLP

4.7 Scaling Methods

Method Formula When to Use

Min-Max Normalization V' = (V - Vmin)/(Vmax - Vmin) When you need bounded output [0,1]; sensitive to outliers
 Z-Score Standardization Z = (V - µ)/σ When data is normally distributed; SVM, LR algorithms

Robust Scaling V' = (V - Q2)/(Q3 - Q1) When data has many outliers; uses IQR instead of range

■ Topic 5: Data Transformations — Min-Max & Z-Score

5.1 Why Transform?
Data transformations perform operations like normalization to improve the performance of data mining and ML
algorithms. Without transformation, features with larger ranges dominate the model.

5.2 Min-Max Normalization

Normalizes each variable V to a new range (say 0 to 1) by subtracting the minimum and dividing by the range.
V' = (V - V_min) / (V_max - V_min) [For range 0 to 1]
V' = [(V - V_min) / (V_max - V_min)] × (new_max - new_min) + new_min [For any
range]
NUMERICAL — V = {88, 90, 92, 94}, map to range [0, 1]:
V_min = 88, V_max = 94, Range = 94 - 88 = 6

V=88 → (88-88)/6 = 0/6 = 0.000

V=90 → (90-88)/6 = 2/6 = 0.333
V=92 → (92-88)/6 = 4/6 = 0.667
V=94 → (94-88)/6 = 6/6 = 1.000

Result: {0.0, 0.333, 0.667, 1.0}

5.3 Z-Score Normalization

Works by subtracting the mean and dividing by the standard deviation. Output has mean=0 and std=1.
Z = (V - µ) / σ where µ = mean, σ = standard deviation
NUMERICAL — V = {10, 20, 30}, convert to z-score:
Step 1: Mean µ = (10+20+30)/3 = 60/3 = 20
Step 2: Variance σ² = [(10-20)²+(20-20)²+(30-20)²]/3
= [100 + 0 + 100]/3 = 200/3 = 66.67
Step 3: Std dev σ = √66.67 = 8.165

Z(10) = (10-20)/8.165 = -10/8.165 = -1.22

Z(20) = (20-20)/8.165 = 0/8.165 = 0.00
Z(30) = (30-20)/8.165 = 10/8.165 = +1.22

Result: {-1.22, 0.00, +1.22} — mean=0, std=1 ✓

■ Note: Min-Max is sensitive to outliers. Z-Score is preferred when data follows a Gaussian distribution. Robust Scaling
(uses IQR) is best when there are many outliers.

■ Topic 6: Data Types & Descriptive Statistics

6.1 Classification of Data Types
Category Sub-type Description Example

Qualitative
Nominal Labels with NO natural order Color, Gender, Blood Type
(Categorical)

Ordinal Categories with a natural ORDER Education Level, Ratings (1-5 stars)

Quantitative
Discrete Countable; takes integer values No. of students, No. of cars
(Numerical)

Continuous Measurable; takes any value in a rangeHeight, Weight, Temperature, Time

Classification by number of variables:

Type No. of Variables Description

Univariate 1 Analysis of a single variable alone

Bivariate 2 Analysis of relationship between two variables

Multivariate 3+ Analysis of three or more variables together

6.2 Measures of Central Tendency

Measure Definition Formula Best Used When

Mean µ = Σx■ / N
Sum of all values divided by count Data is symmetric, no extreme outliers

Median Middle value in a sorted datasetMiddle value (or avg of 2 middle)

Data is skewed or has outliers

Mode Most frequently occurring value Most common value Categorical data; finding most common item

6.3 Measures of Dispersion

Dispersion measures how spread out data is around the central tendency.
Variance σ² = Σ(x■ - µ)² / N
Standard Deviation σ = √Variance = √[Σ(x■ - µ)² / N]
Low variance → data points clustered near mean. High variance → data points spread far from mean.

■ Topic 7: Univariate, Bivariate & Multivariate Statistics

7.1 Univariate Analysis
Univariate analysis has only one variable. The goal is to describe the distribution of that variable.

Aspect Details

Statistics used Mean, Median, Mode, Variance, Standard Deviation, Range, Quartiles

Visualization Histogram, Box plot, Bar chart, Pie chart, Frequency polygon

Purpose Understand distribution, detect outliers, summarize data

7.2 Bivariate Analysis

Bivariate analysis involves two variables. It deals with causes and relationships between them.
• Covariance: Measures how two variables change together.
Cov(X,Y) = Σ[(x■ - x■)(y■ - ■)] / N
• Interpretation: Positive cov → both increase together. Negative cov → one increases, other decreases. Zero
→ no linear relationship.
• Correlation (Pearson's r): Normalized covariance. Range: -1 to +1.
r = Cov(X,Y) / (σX × σY)
• r = +1 → perfect positive | r = -1 → perfect negative | r = 0 → no linear relationship
NUMERICAL — X = (1,2,3,4,5) and Y = (1,4,9,16,25):

x■ = (1+2+3+4+5)/5 = 15/5 = 3
■ = (1+4+9+16+25)/5 = 55/5 = 11

Cov(X,Y) = [(1-3)(1-11) + (2-3)(4-11) + (3-3)(9-11) + (4-3)(16-11) + (5-3)(25-11)] / 5

= [(-2)(-10) + (-1)(-7) + (0)(-2) + (1)(5) + (2)(14)] / 5
= [20 + 7 + 0 + 5 + 28] / 5 = 60/5 = 12

σX = √[((1-3)²+(2-3)²+(3-3)²+(4-3)²+(5-3)²)/5] = √[(4+1+0+1+4)/5] = √2 = 1.414

σY = √[((1-11)²+(4-11)²+(9-11)²+(16-11)²+(25-11)²)/5]
= √[(100+49+4+25+196)/5] = √(374/5) = √74.8 = 8.649
 r = Cov(X,Y)/(σX × σY) = 12 / (1.414 × 8.649) = 12/12.23 ≈ 0.981

Interpretation: r = 0.981 → Very strong positive correlation (Y increases strongly with

X)

7.3 Multivariate Analysis

When data involves three or more variables, it is multivariate.
Techniques: Regression analysis, Path analysis, Factor analysis, Cluster analysis, MANOVA (Multivariate Analysis
of Variance).
Visualization: 3D scatter plots, Pair plots, Heat maps, Parallel coordinates.

■ Topic 8: Probability — Marginal, Joint & Conditional

8.1 Why Probability in ML?
Statistics in ML is used to analyze data and find unseen patterns. Probability distributions are used to:
• Calculate confidence intervals for parameters
• Calculate critical regions for hypothesis tests
• Power classification algorithms like Naive Bayes

8.2 Types of Probability

Type Definition Formula Example

Marginal Probability Probability of an event irrespective

P(A) Σ P(A,B)
of =the outcome of anotherP(Male)
variableregardless of rank

Joint Probability Probability of two events occurring

P(A∩B)
simultaneously
= P(A) × P(B) [if independent]
P(Rank-1 AND Male)

Conditional Probability Probability of event A given that P(A|B)

event B= has
P(A∩B)
occurred
/ P(B) P(Rank-1 | Male)

MARGINAL PROBABILITY EXAMPLE:

Suppose a table has 100 employees: 60 Male, 40 Female.
P(Male) = 60/100 = 0.6 — This is marginal probability (ignores rank)

JOINT PROBABILITY EXAMPLE:

10 employees are Rank-1 Officers, of which 7 are Male.
P(Rank-1 AND Male) = 7/100 = 0.07

CONDITIONAL PROBABILITY EXAMPLE:

P(Rank-1 | Male) = P(Rank-1 AND Male) / P(Male) = 0.07 / 0.60 = 0.117
i.e., given the employee is male, there is 11.7% chance they are Rank-1.

■ Topic 9: Random Variables, PDF, CDF & Gaussian Distribution

9.1 Random Variable (RV)
A Random Variable X is a process by which a real number x(s) is assigned to each possible outcome of a statistical
experiment.

Type Description Example

Discrete RV Takes countable, finite values Number of heads in 5 coin tosses: {0,1,2,3,4,5}

Continuous RV Takes any real value in an interval [a,b]

Height of students: any value between 140-200 cm

9.2 Moments of a Random Variable

The nth moment of a random variable X is defined as the expected value of X^n: E[X^n]
The first moment (n=1) is the Mean (expected value): E[X] = µ
The second moment (n=2) is the Mean Square Value: E[X²]
Central Moments are moments about the mean. The nth central moment = E[(X-µ)^n]
The second central moment (n=2) is the VARIANCE: σ² = E[(X-µ)²]
Standard Deviation σ = √Variance
9.3 Cumulative Distribution Function (CDF)
The CDF of a RV X is defined as the probability that X takes values less than or equal to x:
F(x) = P(X ≤ x)
Properties of CDF:
• F(-∞) = 0 and F(+∞) = 1
• CDF is a non-decreasing function
• P(a < X ≤ b) = F(b) - F(a)

9.4 Probability Density Function (PDF)

The PDF f(x) is the derivative of CDF: f(x) = dF(x)/dx. It describes the relative likelihood of the RV taking a given
value.
Properties of PDF:
• f(x) ≥ 0 for all values of x (non-negative)
• Area under the PDF curve is always unity: ∫f(x)dx = 1
• P(a ≤ X ≤ b) = ∫[a to b] f(x) dx

9.5 Gaussian (Normal) Distribution

The Gaussian Distribution (also called Normal Distribution) is the most important probability distribution in statistics
and ML.
f(x) = (1 / σ√2π) × e^[-(x-µ)²/(2σ²)]
Properties of Gaussian PDF:
• Bell-shaped, symmetric curve
• Peak value occurs at x = µ (mean)
• Even symmetry around the mean value
• Completely defined by two parameters: mean µ and standard deviation σ
• ~68% data within 1σ, ~95% within 2σ, ~99.7% within 3σ (Empirical Rule)
■ Topic 10: Bayes Theorem & Bayesian Networks
10.1 Bayes Theorem — Introduction
Bayes Theorem was given by Thomas Bayes in the 17th century. It is a method to determine conditional
probabilities — the probability of one event occurring given that another has already occurred.
It allows a model to update/revise its predictions as new evidence becomes available. Widely used in ML,
especially in classification problems.
P(A|B) = [P(B|A) × P(A)] / P(B)
Posterior = (Likelihood × Prior) / Evidence
Symbol Term Definition

P(A|B) Posterior Updated probability of hypothesis A after observing evidence B

P(B|A) Likelihood Probability of evidence B occurring if hypothesis A is true

P(A) Prior Initial probability of hypothesis A before seeing evidence

P(B) Marginal / Evidence Total probability of evidence B under ALL possible hypotheses

SOLVED EXAMPLE 1 — Disease Diagnosis:

Given: P(Disease) = 0.01, P(No Disease) = 0.99
True Positive Rate P(+|Disease) = 0.90
False Positive Rate P(+|No Disease) = 0.10
Find: P(Disease | Test Positive) = ?

Step 1: Total evidence P(+) = P(+|D)×P(D) + P(+|¬D)×P(¬D)

= (0.90×0.01) + (0.10×0.99)
= 0.009 + 0.099 = 0.108

Step 2: Posterior = P(+|D)×P(D) / P(+)

= (0.90 × 0.01) / 0.108
= 0.009 / 0.108 ≈ 0.0833 = 8.33%

Insight: Even with 90% accurate test, only 8.3% chance of actually having disease!
(because disease is rare — prior is only 1%). This is the Base Rate Fallacy.
SOLVED EXAMPLE 2 — Man Speaks Lies:
P(lie) = 1/4, P(truth) = 3/4
He throws a die and reports it is a SIX.
P(reporting 6 | actually 6) = 3/4 [he tells truth]
P(reporting 6 | not actually 6) = (1/4)×(1/5) = 1/20 [lies and picks another of 5
numbers]

P(B) = P(rep 6|actual 6)×P(6) + P(rep 6|not 6)×P(not 6)

= (3/4)(1/6) + (1/20)(5/6)
= 3/24 + 5/120 = 15/120 + 5/120 = 20/120 = 1/6

P(actually 6 | reports 6) = [(3/4)(1/6)] / (1/6) = 3/4

Answer: Probability it is actually a six = 3/4

SOLVED EXAMPLE 3 — Meningitis:
P(meningitis) = 1/30000, P(stiff neck) = 0.02
P(stiff neck | meningitis) = 0.80

P(meningitis | stiff neck) = P(stiff neck|meningitis) × P(meningitis) / P(stiff neck)

= 0.80 × (1/30000) / 0.02
= (0.80/30000) / 0.02
 = 0.0000267 / 0.02
= 0.001333 = 0.133%

Answer: Only 0.133% chance of meningitis even with a stiff neck.

10.2 Bayesian Networks

A Bayesian Network (BN) is a probabilistic graphical model that represents probabilistic relationships among
variables using a Directed Acyclic Graph (DAG).

Component Description

Nodes Represent random variables (e.g., Burglary, Disease, Weather, Sensor readings)

Edges (Arrows) Represent conditional dependencies between variables (Parent → Child)

CPT Conditional Probability Table — defines P(node | parent nodes) for each node

Applications of Bayesian Networks:

• Medical Diagnosis (inferring disease from symptoms)
• Decision Support Systems
• Risk Assessment
• Machine Learning (probabilistic classifiers)
• Natural Language Processing (spam detection)
Advantages of Bayesian Networks:
• Modeling Uncertainty: Explicitly handles uncertain information using probabilities
• Visual Representation: DAG makes relationships visually intuitive and interpretable
• Learning from Data: Parameters (CPTs) can be learned from training data
BAYESIAN NETWORK EXAMPLE — Harry's Burglary Alarm:
Setup:
B=Burglary, E=Earthquake, A=Alarm, D=David calls, S=Sophia calls
Alarm triggers due to Burglary OR Earthquake
David: Always calls when alarm sounds (but sometimes confused by phone ringing)
Sophia: Sometimes misses alarm (listens to loud music)

Query: P(A=T, B=F, E=F, D=T, S=T) = ?

Using Joint Probability with CPTs:
= P(B=F) × P(E=F) × P(A=T|B=F,E=F) × P(D=T|A=T) × P(S=T|A=T)
= 0.999 × 0.998 × 0.001 × 0.91 × 0.70
≈ 0.000637

This represents the joint probability of the specific scenario occurring.

■ Topic 11: Hypothesis & Hypothesis Testing

11.1 What is a Hypothesis?
A hypothesis is an assumption or prediction based on some evidence that can be tested. In supervised ML, a
hypothesis is the function the model learns to map inputs to outputs.

Type Symbol Meaning Example

Null Hypothesis H0 Default assumption; states no effect, noµdifference

= 100 (population mean is 100)

Alternative Hypothesis H1 or Ha What you want to prove; contradicts H0µ ≠ 100 (population mean differs from 100)

11.2 Types of Hypothesis Tests

Category Tests When to Use
 Parametric Z-test, t-test, F-test Data is normally distributed; makes assumptions about population parameters

Non-Parametric Chi-Square test Data is categorical; no assumptions about distribution

11.3 Significance Level (α)

The significance level α is the threshold probability for rejecting H0. Common values are 5% (α=0.05) and 1%
(α=0.01).
If p-value < α → Reject H0 (result is statistically significant)
If p-value ≥ α → Fail to Reject H0 (insufficient evidence)

11.4 Z-Test
Used when: Population standard deviation σ is KNOWN, sample size n is LARGE (n ≥ 30), data follows normal
distribution.
Z = (X■ - µ) / (σ / √n)
where X■ = sample mean, µ = population mean, σ = population std dev, n = sample size
Critical values: α=0.05 two-tailed → ±1.96 | α=0.05 one-tailed → 1.645
Z-TEST EXAMPLE 1 — Vaccine Immunity:
Given: σ=20, n=40, X■=96.25, µ=100, α=0.05 (two-tailed → Z_critical=±1.96)
Z = (96.25 - 100) / (20/√40) = -3.75 / (20/6.324) = -3.75 / 3.162 = -1.186
|Z| = 1.186 < 1.96 → FAIL TO REJECT H0
Conclusion: Immunity level is NOT significantly different from population mean of 100.

Z-TEST EXAMPLE 2 — GATE Scores (One-Tailed):

Given: σ=9, n=40, X■=303.8, µ=300, α=0.05 (one-tailed → Z_critical=1.645)
Z = (303.8 - 300) / (9/√40) = 3.8 / (9/6.324) = 3.8 / 1.423 = 2.67
|Z| = 2.67 > 1.645 → REJECT H0
Conclusion: Mean GATE score IS significantly greater than 300. Claim supported.

Z-TEST EXAMPLE 3 — Two-Sample Z-Test (Teacher's Claim):

Section A: n1=60, X■1=22.1, σ1=4.8
Section B: n2=40, X■2=18.8, σ2=8.1
α=0.05 (one-tailed → Z_critical=1.645)
Z = (X■1-X■2) / √(σ1²/n1 + σ2²/n2)
= (22.1-18.8) / √(4.8²/60 + 8.1²/40)
= 3.3 / √(0.384 + 1.640)
= 3.3 / √2.024 = 3.3/1.423 = 2.32
Z=2.32 > 1.645 → REJECT H0. Teacher's claim IS supported.

11.5 t-Test (One-Sample)

Used when: Population σ is UNKNOWN, sample size is SMALL (n < 30).
t = (X■ - µ) / (s / √n) Degrees of freedom df = n - 1
where s = sample standard deviation
t-TEST EXAMPLE — Student Marks:
Data: 9.5,10,8,7,11,7,6.5,8.5,10.5,12 n=10, µ_population=12, α=0.05, df=9
X■ = (9.5+10+8+7+11+7+6.5+8.5+10.5+12)/10 = 90/10 = 9.0
s² = Σ(xi-X■)²/(n-1)
= [(0.25+1+1+4+4+4+6.25+0.25+2.25+9)] / 9 = 32/9 = 3.556
s = √3.556 = 1.886
t = (9.0 - 12) / (1.886/√10) = -3.0 / (1.886/3.162) = -3.0/0.596 = -5.03
t_critical(df=9, α=0.05, two-tailed) = ±2.262
|t| = 5.03 > 2.262 → REJECT H0
Conclusion: Student mean IS significantly different from population mean of 12.

11.6 Independent Two-Sample t-Test

Compares means of two independent groups A and B when population σ is unknown.
 t = (X■A - X■B) / √[s²(1/N1 + 1/N2)] df = N1+N2-2
where s² is the pooled variance of both samples.

11.7 Chi-Square Test (Non-Parametric)

Chi-Square test measures statistical significance between observed (O) and expected (E) frequencies. Each
observation is independent and follows normal distribution.
χ² = Σ [(O - E)² / E] df = C - 1 (C = number of categories)
If χ² > χ²_critical → Reject H0 (significant difference exists)
The Chi-Square test detects data duplication and helps remove redundancy.
Chi-Square EXAMPLE — ML Course Registration:
50 Boys and 50 Girls. Registered: Boys=35, Girls=20. Not Registered: Boys=15, Girls=30.
Expected (if no gender difference): Each group should register equally → Expected=27.5
each

For Registered Boys: O=35, E=27.5 → (35-27.5)²/27.5 = 56.25/27.5 = 2.045

For Registered Girls: O=20, E=27.5 → (20-27.5)²/27.5 = 56.25/27.5 = 2.045
For Not Reg Boys: O=15, E=22.5 → (15-22.5)²/22.5 = 56.25/22.5 = 2.5
For Not Reg Girls: O=30, E=22.5 → (30-22.5)²/22.5 = 56.25/22.5 = 2.5

χ² = 2.045+2.045+2.5+2.5 = 9.09
df = 2-1 = 1, χ²_critical(0.05, df=1) = 3.841
9.09 > 3.841 → REJECT H0. Significant difference exists between boys and girls.

■ Topic 12: Concept Learning, Hypothesis Space & Find-S Algorithm

12.1 Concept Learning
Concept Learning is the process of acquiring knowledge about categories, ideas, or things based on shared
features. Shared features are common characteristics present in all instances of a category.
Purpose: Group similar objects together and distinguish them from other categories.
Importance: Essential for problem-solving, categorization, and language learning.
Concept learning requires three things:

Component Description Example

Input Training dataset — instances labeled with theirImages

concept/category
labeled 'Elephant' or 'Not Elephant'

Output Target function f(x) — maps input x to concept'Has

output
trunk,
y; finds
largecommon
ears, grey, heavy' → Elephant
features

Test New unlabeled instances to test the learned model

A new image to classify
■ Note: Formally: 'Given a set of hypotheses, the learner searches through the hypothesis space to identify the best
hypothesis that matches the target concept.'

12.2 Hypothesis Space (H)

The Hypothesis Space H is the set of ALL possible legal hypotheses. Also known as the hypothesis set. Supervised
ML algorithms search through H to find the best hypothesis h* that maps inputs to outputs.

12.3 Searching the Hypothesis Space

Method Direction Description Reasoning Type Example

Specialization
Top-Down Starts with most general hypothesis,
Deductive All birds have wings → This is a bird → It ha
narrows down
(General→Specific)

Generalization
Bottom-Up Starts with most specific, broadens to cover more Seen 5 white swans → All swans are white
Inductive
(Specific→General)

12.4 Find-S Algorithm

Find-S is a simple ML algorithm for concept learning. It starts with the most specific hypothesis and generalizes it
based on positive training examples only.
Key Rules:
• Starts with the MOST SPECIFIC hypothesis h = <∅, ∅, ∅, ...>
• For each POSITIVE example: generalize h to match it
• Negative examples are ignored
• If an attribute value in h matches the example → keep it
• If attribute value does NOT match → replace with '?' (generalize)
• Return the final h as the learned concept
Find-S EXAMPLE — PlayTennis concept:
Attributes: Sky, Temp, Humidity, Wind, PlayTennis
Initial h = <∅, ∅, ∅, ∅> (most specific — matches nothing)

Training examples:
Ex1: Sunny, Hot, High, Strong → YES (positive)
Ex2: Sunny, Hot, High, Weak → YES (positive)
Ex3: Cloudy, Hot, High, Strong → NO (negative) → IGNORE
Ex4: Sunny, Cool, Normal, Weak → YES (positive)

Processing:
After Ex1: h = (direct copy)
After Ex2: h = (Wind: Strong≠Weak → generalize to ?)
After Ex3: SKIP (negative example)
After Ex4: h = (Temp: Hot≠Cool, Humidity: High≠Normal → ?)

Final hypothesis: h =
Meaning: It is a PlayTennis day whenever Sky=Sunny, regardless of other attributes.

■ Topic 13: Induction Bias & Bias-Variance Trade-off

13.1 Bias
Bias is the difference between the model's expected prediction and the true value.

Type Description Effect

High Bias Model is too simple; fails to capture underlyingPoor on BOTH training and test data → UNDERFITTING
patterns

Low Bias wellalso capture noise → can lead to overfitting

Model is complex; captures underlying patternsMay

13.2 Variance
Variance measures how much the model's predictions change when trained on different subsets of the data.

Type Description Effect

High Variance Model is too complex; captures noise along with

Good on training, poor on test data → OVERFITTING
patterns

Low Variance Model is simple and stable May underfit → poor on both

13.3 Bias-Variance Trade-off

There is a fundamental trade-off. The goal is to find the optimal model complexity that minimizes BOTH bias and
variance (total error):
Total Error = Bias² + Variance + Irreducible Noise
Model Complexity Bias Variance Problem Solution

Too simple High Low Underfitting Increase complexity, add features

Too complex Low High Overfitting Regularization (L1/L2), more data, pruning
 Optimal Medium Medium Balanced This is the goal!

13.4 Inductive Bias

Inductive bias is the set of assumptions a learning algorithm uses to generalize beyond training data to unseen
instances. Without bias, a model cannot generalize.
• Hypothesis Space Bias: Restricts H to a subset of all possible hypotheses (e.g., only linear functions)
• Preference Bias (Occam's Razor): Among equally good hypotheses, prefer the simpler one
• Find-S bias: Considers only the most specific consistent hypothesis

■ Topic 14: Python — Preprocessing & Hypothesis Space

Min-Max, Z-Score & Encoding
import numpy as np
import pandas as pd
from [Link] import MinMaxScaler, StandardScaler, LabelEncoder, OneHotEncoder
from [Link] import SimpleImputer

# Min-Max Normalization
data = [Link]([[88],[90],[92],[94]])
scaler = MinMaxScaler(feature_range=(0,1))
print(scaler.fit_transform(data)) # [[0.],[0.333],[0.667],[1.0]]

# Z-Score Standardization
data2 = [Link]([[10],[20],[30]])
std_scaler = StandardScaler()
print(std_scaler.fit_transform(data2)) # [[-1.22],[0.0],[1.22]]

# Handle Missing Values

df = [Link]({'Age': [25, None, 35], 'Salary': [50000, 60000, None]})
imputer = SimpleImputer(strategy='mean')
df_clean = [Link](imputer.fit_transform(df), columns=[Link])

# Label Encoding (for ordinal categorical)

le = LabelEncoder()
encoded = le.fit_transform(['Red','Blue','Green','Red']) # [2,0,1,2]

# One-Hot Encoding (for nominal categorical)

ohe = OneHotEncoder()
result = ohe.fit_transform([['Red'],['Blue'],['Green']]).toarray()
UNIT 2 — 4 Marks

■ Topic 15: ML Workflow — Train/Validation/Test & Forms of Learning

15.1 Data Split Strategy
For training the model, data needs to be divided into 3 parts:

Split Typical Size Purpose Key Rule

Training Data 60–70% Model LEARNS patterns from this dataNever evaluate final performance here

Validation Data 10–15% Tune hyperparameters and select best Used

modelDURING development

Test Data 15–20% FINAL unbiased evaluation of the modelUse ONLY ONCE at the end; never for tuning
■ Note: Algorithm selection depends on the problem definition. E.g., classifying emails as spam/not spam requires a
classification algorithm that takes input and gives output SPAM / Not SPAM.

15.2 Forms of Learning

Form Description Algorithm Examples

Supervised Learning Learns from LABELED data; maps inputs

KNN,
to outputs
Decision Tree, SVM, Naive Bayes, Linear Regression

Unsupervised Learning Learns from UNLABELED data; finds hidden

K-Means
patterns/groupings
Clustering, PCA, DBSCAN

Semi-Supervised Mix of labeled + unlabeled data Self-training, Label Propagation

Reinforcement Learning Agent takes actions, receives rewards/penalties

Q-Learning,
to learn
Deepoptimal
RL, AlphaGo
policy

15.3 Supervised Learning — Types

Type Output Algorithms

Classification Discrete class label (Spam/Not Spam,

KNN, Decision
Cat/Dog)Tree, SVM, Naive Bayes, Logistic Regression, Random Forest

Regression Continuous numerical value (price,

Linear
temperature)
Regression, Polynomial Regression, Random Forest Regression

■ Topic 16: Classification Algorithms — KNN, Decision Tree, SVM, Naive Bayes
16.1 K-Nearest Neighbors (KNN)
Non-parametric, lazy learner (no explicit training phase). Classifies based on majority vote of K nearest neighbors.
Euclidean Distance = √[Σ(x■ - y■)²]
Steps:
• Choose value of K
• Calculate Euclidean distance from query point to ALL training points
• Select K nearest neighbors
• Assign the majority class among K neighbors
Aspect Details

Small K Low bias, high variance → Overfitting

Large K High bias, low variance → Underfitting

Best K Found via cross-validation; typically odd numbers to avoid ties

Pros Simple, no training time, handles multi-class naturally

Cons Slow prediction on large datasets, sensitive to irrelevant features & scale
from [Link] import KNeighborsClassifier
knn = KNeighborsClassifier(n_neighbors=5)
[Link](X_train, y_train)
predictions = [Link](X_test)

16.2 Decision Tree Classifier

Tree-structured model. Each internal node = a feature/attribute. Each branch = decision. Each leaf = class label.
Entropy H(S) = -Σ p■ × log■(p■)
Information Gain IG = H(parent) - Σ[weighted H(children)]
Gini Impurity = 1 - Σ(p■)²
Criterion Formula Used In

Information Gain IG = H(S) - Σ|Sv|/|S| × H(Sv) ID3 algorithm

Gini Impurity G = 1 - Σp■² CART algorithm (sklearn default)

Gain Ratio GR = IG / SplitInfo C4.5 algorithm — handles multi-valued attributes better

Pros: Highly interpretable, no feature scaling needed, handles both types of data.
Cons: Prone to overfitting; deep trees overfit — use max_depth or pruning.
from [Link] import DecisionTreeClassifier
dt = DecisionTreeClassifier(criterion='gini', max_depth=5, min_samples_split=2)
[Link](X_train, y_train)

16.3 Support Vector Machine (SVM)

Finds the optimal hyperplane that maximally separates classes. The decision boundary is determined by support
vectors — data points closest to the hyperplane.
Margin = 2 / ||w|| SVM maximizes this margin
Concept Explanation

Hyperplane Decision boundary in n-dimensional space: w·x + b = 0

Support Vectors Data points closest to the hyperplane; they define the margin

Kernel Trick Maps data to higher dimensions to make non-linear data separable

Linear Kernel K(x,y) = x·y — for linearly separable data

RBF Kernel K(x,y) = exp(-γ||x-y||²) — most commonly used; handles non-linear

C Parameter Controls margin width vs misclassification. High C → small margin, less error
Pros: Very effective in high-dimensional spaces, robust to overfitting with right C.
Cons: Does not scale well to very large datasets; requires feature scaling.
from [Link] import SVC
svm = SVC(kernel='rbf', C=1.0, gamma='scale')
[Link](X_train, y_train)

16.4 Naive Bayes Classifier

Probabilistic classifier based on Bayes theorem. Called 'Naive' because it assumes ALL features are conditionally
INDEPENDENT given the class.
P(y|X) ∝ P(y) × Π P(x■|y) for each feature x■
Classify: assign class y* = argmax P(y) × Π P(x■|y)

Type Assumption Use Case

Gaussian NB Features follow normal distribution Continuous numerical data

Multinomial NB Features are counts/frequencies Text classification (word counts)

Bernoulli NB Features are binary (0 or 1) Document classification (word presence)

Pros: Fast, works well with small data, excellent for text classification.
Cons: The independence assumption rarely holds; poor probability estimates.
from sklearn.naive_bayes import GaussianNB
nb = GaussianNB()
[Link](X_train, y_train)
 proba = nb.predict_proba(X_test)

■ Topic 17: Regression — Linear, Multiple & Polynomial

17.1 What is Regression?
Regression analysis is used for prediction and forecasting. It models the relationship between independent
variable(s) x and dependent variable y.
Applications: Sales forecasting, bond valuation, insurance premiums, real estate prices.
Regression finds:
• Relationship between variables
• Nature of relationship (linear/non-linear)
• Relevance of attributes
• Contribution of each attribute
Limitations of Regression: Sensitive to outliers, needs adequate number of cases, problems with missing data
and multicollinearity.

17.2 Simple Linear Regression

Fits a straight line through scatter data points. Models relationship between ONE independent variable x and
dependent variable y.
y = a■ + a■x + ε
a■ = intercept (bias), a■ = slope (regression coefficient), ε = error in prediction
OLS (Ordinary Least Squares): Minimizes sum of squared errors to find best-fit line.
a■ = [n·Σx■y■ - Σx■·Σy■] / [n·Σx■² - (Σx■)²]
a■ = (Σy■ - a■·Σx■) / n = ■ - a■·x■
NUMERICAL — Weekly Sales Data:
X (Week): 1, 2, 3, 4, 5
Y (Sales in 000s): 1.2, 1.8, 2.6, 3.2, 3.8

n=5, Σx=15, Σy=12.6

Σxy = (1×1.2)+(2×1.8)+(3×2.6)+(4×3.2)+(5×3.8) = 1.2+3.6+7.8+12.8+19.0 = 44.4
Σx² = 1+4+9+16+25 = 55

a■ = [5×44.4 - 15×12.6] / [5×55 - 15²]

= [222 - 189] / [275 - 225] = 33/50 = 0.66

a■ = (12.6 - 0.66×15) / 5 = (12.6 - 9.9) / 5 = 2.7/5 = 0.54

Regression equation: y = 0.54 + 0.66x

Predict Week 7: y = 0.54 + 0.66×7 = 0.54 + 4.62 = 5.16 (Thousands)

Predict Week 9: y = 0.54 + 0.66×9 = 0.54 + 5.94 = 6.48 (Thousands)

17.3 Multiple Linear Regression

Models the relationship between ONE dependent variable and TWO or more independent variables.
y = a■ + a■x■ + a■x■ + ... + a■x■
Assumptions: Linear relationship, Normally distributed residuals, Little or no multicollinearity (features not highly
correlated with each other)
NUMERICAL — House Price Prediction:
Data: Size x1 ([Link].), Bedrooms x2, Price y (Rs.)
 1000, 2, 50,00,000
1200, 3, 60,00,000
1500, 4, 75,00,000
Model: y = a■ + a■×Size + a■×Bedrooms
(Solve using matrix method or sklearn — sklearn approach below)

from sklearn.linear_model import LinearRegression

X = [[1000,2],[1200,3],[1500,4]]
y = [5000000, 6000000, 7500000]
model = LinearRegression().fit(X, y)
print(model.coef_, model.intercept_)

17.4 Polynomial Regression

Models the relationship as an nth degree polynomial. Used when the relationship between x and y is NON-LINEAR.
y = b■ + b■x + b■x² + b■x³ + ... + b■x■
Polynomial regression is a special case of Multiple Linear Regression. The dataset used for training is of non-linear
nature.
Need: If a linear model is applied on non-linear data, it gives poor predictions. Polynomial regression solves this.
NUMERICAL — Employee Salary (Degree 2):
Experience (x): 2, 4, 6 years | Salary (y): 3,00,000 | 4,50,000 | 6,00,000
Model: y = b■ + b■x + b■x²

from [Link] import PolynomialFeatures

from sklearn.linear_model import LinearRegression
X = [[2],[4],[6]]
y = [300000, 450000, 600000]
poly = PolynomialFeatures(degree=2)
X_poly = poly.fit_transform(X) # adds x² column
model = LinearRegression().fit(X_poly, y)
# Predict for 5 years:
X_pred = [Link]([[5]])
print([Link](X_pred))

■ Topic 18: Regression Validation Metrics

Metric Formula Interpretation

Standard Error Std dev of residuals (■ - y) 0 = perfect fit; higher = worse fit

MAE Σ|y■ - ■■| / n Mean of absolute errors; not sensitive to outliers

MSE Σ(y■ - ■■)² / n Always positive; penalizes large errors more than MAE

RMSE √MSE Same unit as target; lower is better

Relative MSE MSE / Var(y) 0=perfect; 0-1=good; >1=model is bad

CV (Coeff. Variation) RMSE / ■ × 100% Normalized RMSE; lower is better

NUMERICAL — Validation Metrics Example:
Two fresh items I6, I7: actual = [80, 75], predicted = [75, 85]

Residuals: e6 = 80-75 = 5, e7 = 75-85 = -10

MAE = (|5| + |-10|) / 2 = (5+10)/2 = 7.5

MSE = (5² + (-10)²) / 2 = (25+100)/2 = 62.5
RMSE = √62.5 = 7.906
 ■_actual = (80+75)/2 = 77.5
Var(y) = ((80-77.5)² + (75-77.5)²)/2 = (6.25+6.25)/2 = 6.25
Relative MSE = MSE/Var(y) = 62.5/6.25 = 10.0 → >1 so model is NOT good for this mini
example!
CV = RMSE/■ = 7.906/77.5 = 0.102 = 10.2%

■ Topic 19: Logistic Regression & Random Forest

19.1 Logistic Regression
Logistic regression predicts the output of a categorical dependent variable. Used for classification problems.
Instead of fitting a line, it fits an S-shaped logistic (sigmoid) function.
Sigmoid: P(y=1|x) = 1 / (1 + e^-(a■ + a■x))
Output is a probability between 0 and 1. If P ≥ 0.5 → Class 1; If P < 0.5 → Class 0.
Use cases: Spam detection (yes/no), admission prediction (admitted/not), pass/fail.

Type Classes Example

Binomial 2 classes (0 or 1) Pass/Fail, Spam/Not Spam, Yes/No

Multinomial 3+ unordered classes Cat / Dog / Sheep

Ordinal 3+ ordered classes Low / Medium / High

NUMERICAL — Student Admission:
a■ = 1, a■ = 8, marks x = 0.60 (normalized)

z = a■ + a■×x = 1 + 8×0.60 = 1 + 4.8 = 5.8

P(pass) = 1 / (1 + e^(-5.8))
= 1 / (1 + 0.00304)
= 1 / 1.00304
≈ 0.997 = 99.7%

Since P=0.997 > 0.5 threshold → Class = PASS (Student is admitted)

19.2 Random Forest

An ensemble method that builds multiple decision trees on random subsets of data and features, then combines
predictions. Based on Bagging (Bootstrap Aggregating).

Concept Explanation

Bagging Each tree trained on a bootstrap sample (random sample WITH replacement)

Feature Randomness At each split, only a random subset of features is considered (√n features)

Classification Final prediction = majority VOTE across all trees

Regression Final prediction = AVERAGE prediction across all trees

Feature Importance Measures how much each feature reduces impurity across all trees
Pros: Reduces overfitting vs single decision tree, handles high-dimensional data, robust to outliers, provides feature
importance.
Cons: Less interpretable than a single tree, slower to train with many trees.
from [Link] import RandomForestClassifier, RandomForestRegressor
rf_cls = RandomForestClassifier(n_estimators=100, max_depth=5, random_state=42)
rf_cls.fit(X_train, y_train)
# Feature importances
print(dict(zip(feature_names, rf_cls.feature_importances_)))
■ Topic 20: Confusion Matrix & Evaluation Metrics
A confusion matrix is a summary table of a classifier's predictions vs actual values.

Predicted: Positive Predicted: Negative

TP — True Positive FN — False Negative

Actual: Positive
(Correct positive prediction) (Missed positive → Type II Error)

FP — False Positive TN — True Negative

Actual: Negative
(Wrong positive → Type I Error) (Correct negative prediction)

Metric Formula What it Measures

Accuracy (TP+TN) / (TP+TN+FP+FN) Overall fraction of correct predictions

Precision TP / (TP+FP) Of predicted positives, how many are actually positive? (avoid FP)

Recall (Sensitivity) TP / (TP+FN) Of actual positives, how many were correctly found? (avoid FN)

Specificity TN / (TN+FP) Of actual negatives, how many were correctly identified?

F1 Score 2×(Precision×Recall)/(Precision+Recall)
Harmonic mean of Precision & Recall; good when classes are imbalance
NUMERICAL — Confusion Matrix Evaluation:
Given: TP=50, FP=10, FN=5, TN=100 (Total = 165)

Accuracy = (50+100)/(50+100+10+5) = 150/165 = 90.9%

Precision = 50/(50+10) = 50/60 = 83.3% (when model predicts +, right 83% of time)
Recall = 50/(50+5) = 50/55 = 90.9% (model finds 90.9% of all actual positives)
Specificity = 100/(100+10) = 100/110 = 90.9% (correctly identifies 90.9% of negatives)
F1 Score = 2×(0.833×0.909)/(0.833+0.909)
= 2×0.757/1.742 = 1.514/1.742 = 86.9%

Interpretation: High recall means we're missing very few actual positives.
Lower precision means some false alarms (healthy people flagged as sick).

■ Topic 21: Python — Complete ML Implementation

Complete Classification + Regression Pipeline
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from [Link] import StandardScaler, MinMaxScaler
from [Link] import KNeighborsClassifier
from [Link] import DecisionTreeClassifier
from [Link] import SVC
from sklearn.naive_bayes import GaussianNB
from [Link] import RandomForestClassifier
from sklearn.linear_model import LinearRegression, LogisticRegression
from [Link] import (confusion_matrix, classification_report,
accuracy_score, mean_absolute_error,
mean_squared_error)

# 1. Load Data
df = pd.read_csv('[Link]')
X = [Link]('target', axis=1)
y = df['target']

# 2. Split
X_train, X_test, y_train, y_test = train_test_split(
 X, y, test_size=0.2, random_state=42)

# 3. Scale
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = [Link](X_test)

# 4. Train all classifiers

classifiers = {
'KNN': KNeighborsClassifier(n_neighbors=5),
'DTree': DecisionTreeClassifier(criterion='gini', max_depth=5),
'SVM': SVC(kernel='rbf', C=1.0),
'NaiveBayes': GaussianNB(),
'RF': RandomForestClassifier(n_estimators=100)
}
for name, clf in [Link]():
[Link](X_train, y_train)
pred = [Link](X_test)
print(f'{name} Accuracy: {accuracy_score(y_test, pred):.4f}')
print(confusion_matrix(y_test, pred))
print(classification_report(y_test, pred))

# 5. Linear Regression
lr = LinearRegression()
[Link](X_train, y_train)
y_pred = [Link](X_test)
print('MAE:', mean_absolute_error(y_test, y_pred))
print('MSE:', mean_squared_error(y_test, y_pred))
print('RMSE:', [Link](mean_squared_error(y_test, y_pred)))

# 6. Logistic Regression
log_reg = LogisticRegression()
log_reg.fit(X_train, y_train)
print('Probabilities:', log_reg.predict_proba(X_test[:3]))

# 7. Polynomial Regression
from [Link] import PolynomialFeatures
poly = PolynomialFeatures(degree=2)
X_poly_train = poly.fit_transform(X_train)
X_poly_test = [Link](X_test)
poly_reg = LinearRegression().fit(X_poly_train, y_train)
y_poly_pred = poly_reg.predict(X_poly_test)

■ FINAL QUICK REFERENCE — Formula & Algorithm Cheat Sheet

Topic Key Formula / Fact

Big Data 6V's Volume, Velocity, Variety, Veracity, Validity, Value

Min-Max V' = (V - Vmin)/(Vmax - Vmin)

Z-Score Z = (V - µ)/σ

Covariance Cov(X,Y) = Σ(xi-x■)(yi-■)/N

Correlation r = Cov(X,Y)/(σX·σY) range: -1 to +1

Variance σ² = Σ(xi-µ)²/N
Bayes Theorem P(A|B) = P(B|A)·P(A)/P(B)

Z-Test Z = (X■-µ)/(σ/√n), critical: ±1.96 (α=0.05)

One-Sample t-Test t = (X■-µ)/(s/√n), df=n-1

Chi-Square χ² = Σ(O-E)²/E, df=C-1

Linear Regression y = a■ + a■x; a■ = [nΣxy-ΣxΣy]/[nΣx²-(Σx)²]

Multiple Regression y = a■ + a■x■ + a■x■

Polynomial Regression y = b■+b■x+b■x²+...+b■x■

Logistic Regression P = 1/(1+e^-z), z = a■+a■x

KNN Majority vote of K nearest (Euclidean distance)

Decision Tree Split by max IG or min Gini impurity

SVM Maximize margin = 2/||w||; uses kernels

Naive Bayes P(y|X) ∝ P(y) × Π P(xi|y)

Random Forest Ensemble of DTs; vote/average; bagging + random features

Accuracy (TP+TN)/(TP+TN+FP+FN)

Precision TP/(TP+FP)

Recall TP/(TP+FN)

F1 Score 2×P×R/(P+R)

MAE Σ|yi-■i|/n

RMSE √[Σ(yi-■i)²/n]

Find-S Start specific h=<∅>, generalize on + examples only

Bias-Variance Total Error = Bias² + Variance + Noise

Entropy H(S) = -Σ p■ log■(p■)

Gini G = 1 - Σp■²

PDF Properties f(x)≥0; ∫f(x)dx=1; peak at mean for Gaussian

Gaussian PDF f(x) = (1/σ√2π) × e^[-(x-µ)²/2σ²]

■ All the best for your exam! Focus on EVERY solved numerical and formula above.