Data Science

Exam Preparation Notes

Comprehensive notes with numericals, theory, and scenario-based Q&A

Naive Bayes ✓ KNN ✓ Decision Tree ✓ K-Means ✓ Recommender Systems ✓ Cosine Similarity ✓

Preprocessing ✓ Scenario Q&A ✓

01 Data Preprocessing
Before training any machine learning model, raw data must be cleaned and prepared. This is the most important step — "garbage in, garbage
out". A bad dataset will ruin even the best algorithm.

THE 4 MAIN STEPS

1. Data Quality Assessment 2. Data Cleaning
Look for mismatched types, mixed values, outliers, missing data. Fix missing values, remove noise, handle duplicates.
3. Data Transformation 4. Data Reduction
Normalize, encode, aggregate, select features. Reduce dimensions, remove unneeded attributes.

Data Quality Issues

PROBLEM WHAT IT MEANS SOLUTION

Missing Values Some cells are empty (NaN) Fill with mean/median or drop
rows
Outliers Values too far from normal (e.g., score = 0% when all others are Detect and remove or transform
80%+)

Mismatched Income in USD from one source, EUR from another Convert to single format
Types

Noisy Data Random errors, irrelevant data points Binning, regression, clustering

Duplicates Same row appears multiple times Remove duplicates

Transformation Techniques

NORMALIZATION ENCODING CATEGORICAL DATA

Scales all values to a fixed range like 0–1 so no feature Machine learning needs numbers, not words.
dominates due to large numbers. Label Encoding: cat→0, dog→1, moose→2
Example: House price ($50k–$500k) and number of rooms (2– One-Hot Encoding: Creates separate columns (cat=1/0,
10) — rooms would be ignored without scaling. dog=1/0). Avoids false ordering.

BINNING FEATURE SCALING (STANDARDSCALER)

Groups continuous values into ranges. Transforms data to have mean=0, std=1. Ensures fair
Example: Income: $0–35k, $35k–75k, $75k+. Reduces noise. comparison between features of different scales.

Python Preprocessing Code

5-mark code potential

import numpy as np
import pandas as pd
from [Link] import LabelEncoder, OneHotEncoder, StandardScaler
from [Link] import SimpleImputer
from sklearn.model_selection import train_test_split

# Load dataset
dataset = pd.read_csv('[Link]')

# Separate features (X) and target (y)

X = [Link][:, :-1].values # All columns except last
y = [Link][:, -1].values # Last column

# Handle missing values — fill with mean

imputer = SimpleImputer(missing_values=[Link], strategy='mean')
X[:, 1:3] = imputer.fit_transform(X[:, 1:3])

# Encode categorical column

le = LabelEncoder()
X[:, 0] = le.fit_transform(X[:, 0])

# One-Hot Encode to remove false ordering

from [Link] import OneHotEncoder
from [Link] import ColumnTransformer
ct = ColumnTransformer([('encoder', OneHotEncoder(), [0])], remainder='passthrough')
X = ct.fit_transform(X)

# Train/Test Split (80/20)

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)

# Feature Scaling
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = [Link](X_test) # Only transform, never fit test set
02 Feature Engineering & Selection
Feature Engineering = creating new useful columns from existing data.
Feature Selection = choosing only the most relevant columns for the model.

Example of Feature Engineering: In a house price dataset, if you have total_price and area_sqft, you can create a new column:
price_per_sqft = total_price / area_sqft. This new feature can reveal errors (e.g., if one house shows $500/sqft while others show
$5000, it's likely an outlier/error).

Feature Selection Methods

METHOD HOW IT WORKS EXAMPLE

Filter Apply a statistical score to each feature before training. Fast but ignores Chi-squared test, Correlation
feature interactions.

Wrapper Use the ML model itself to try different subsets and find the best. Slower Recursive Feature Elimination
but more accurate. (RFE)

Embedded Feature selection happens inside the algorithm during training. LASSO, Ridge Regression,
Decision Trees

KEY DIFFERENCE: FEATURE SELECTION VS DIMENSIONALITY REDUCTION

Feature Selection: Keeps or removes original features as-is.
Dimensionality Reduction (PCA): Creates NEW combined features from old ones. Original meaning may be lost.

FEATURE IMPORTANCE (TREE-BASED)

You can get a score for each feature using tree classifiers. Higher score = more important to prediction.
from [Link] import ExtraTreesClassifier
Features with low importance can be dropped to simplify the model.
03 Decision Trees
A Decision Tree is like a flowchart — it splits data using questions until it reaches a final answer. Think of it as "20 questions" for predicting
outcomes.

Key Terms:
— Root Node: The top node, the first feature we split on
— Internal Node: Each question/decision point
— Leaf Node: Final answer (no more branches)
— Splitting: Dividing data at a node
— Pruning: Cutting unimportant branches to avoid overfitting

How to Pick the Best Feature to Split On?

We use Information Gain — split on the feature that gives the most information (reduces the most entropy/disorder).

ENTROPY FORMULA

Entropy(S) = -p₊ log₂(p₊) - p₋ log₂(p₋)

Where p₊ = fraction of positive examples, p₋ = fraction of negative examples.

Information Gain = Entropy(Parent) - Weighted Avg Entropy(Children)

📊 NUMERICAL EXAMPLE PLAY TENNIS DATASET — EXAM FAVORITE

We have 14 days of weather data. We want to predict: Will we play tennis?

Dataset Summary: 9 days YES (+), 5 days NO (−)

Entropy(S) = −(9/14)log₂(9/14) − (5/14)log₂(5/14) = 0.940

Step 1 — Calculate Entropy for each feature branch:

Feature Split Count Entropy of Split

Sunny 2+, 3− 0.971

Outlook Overcast 4+, 0− 0.000
Rain 3+, 2− 0.971
High 3+, 4− 0.985
Humidity
Normal 6+, 1− 0.592
Weak 6+, 2− 0.811
Wind
Strong 3+, 3− 1.000

Step 2 — Calculate Information Gain:

Gain(Outlook) = 0.940 − [(5/14)×0.971 + (4/14)×0 + (5/14)×0.971] = 0.246

Gain(Humidity) = 0.940 − [(7/14)×0.985 + (7/14)×0.592] = 0.151

Gain(Wind) = 0.940 − [(8/14)×0.811 + (6/14)×1.000] = 0.048

Gain(Temperature) = 0.029

Step 3 — Pick best split: Outlook wins (highest gain = 0.246). Split root on Outlook.
Step 4 — After Outlook split:
— Overcast → Always YES (leaf, entropy=0)
— Sunny branch (2+, 3−): Next split on Humidity: Normal→YES, High→NO
— Rain branch (3+, 2−): Next split on Wind: Weak→YES, Strong→NO

Final Tree: Outlook → if Overcast: Yes. If Sunny: check Humidity (Normal=Yes, High=No). If Rain: check Wind (Weak=Yes,
Strong=No).
04 K-Means Clustering
K-Means is an unsupervised algorithm — there are no labels. It groups data into K clusters based on similarity. Points close to each other end
up in the same group.

Algorithm Steps:
1 Choose K (number of clusters). Randomly place K centroids.

2 Assign each data point to the nearest centroid (using Euclidean distance).

3 Recalculate centroid = mean of all points in the cluster.

4 Repeat steps 2–3 until centroids stop moving (convergence).

EUCLIDEAN DISTANCE FORMULA

d = √[ (x₂−x₁)² + (y₂−y₁)² ] This is used to find which centroid each point is

closest to.
📊 NUMERICAL EXAMPLE K-MEANS WITH K=2

Data Points: A(1,1), B(2,1), C(4,3), D(5,4)

Initial Centroids: C1=(1,1), C2=(5,4)

Iteration 1 — Assign points to nearest centroid:

Point Dist to C1(1,1) Dist to C2(5,4) Cluster

A(1,1) 0.00 5.00 C1

B(2,1) 1.00 4.24 C1
C(4,3) 3.61 1.41 C2
D(5,4) 5.00 0.00 C2

Update Centroids:

New C1 = mean of {A,B} = ((1+2)/2, (1+1)/2) = (1.5, 1.0)

New C2 = mean of {C,D} = ((4+5)/2, (3+4)/2) = (4.5, 3.5)

Iteration 2 — Reassign with updated centroids:

Point Dist to C1(1.5,1) Dist to C2(4.5,3.5) Cluster

A(1,1) 0.50 4.61 C1

B(2,1) 0.50 3.54 C1
C(4,3) 3.20 0.71 C2
D(5,4) 4.61 0.71 C2

Clusters unchanged → Algorithm Converged!

Result: Cluster 1 = {A, B}, Cluster 2 = {C, D}

IMPORTANT NOTES
— K-Means is unsupervised — no labels needed
— You must choose K beforehand (use Elbow Method to find best K)
— Feature Scaling is important — large-scale features dominate distance
— Sensitive to initial centroid placement (can get stuck in local minimum)
05 K-Nearest Neighbors (KNN)
KNN is a supervised classification algorithm. To classify a new point, look at the K nearest training points and take a majority vote.

How it works (simple explanation):

Imagine you move to a new city. To predict if you'll like a neighborhood, you ask your K nearest neighbors what they think. Most of
them agree → that's your prediction.

EUCLIDEAN DISTANCE

d(p, q) = √[ Σ (pᵢ − qᵢ)² ] For 2D: d = √[(x₂−x₁)² + (y₂−y₁)²]

📊 NUMERICAL EXAMPLE K=3 CLASSIFICATION

Problem: Classify new point P(3, 4) using K=3 from the training data below.

Point X Y Class

A 1 2 Red
B 2 3 Red
C 3 1 Blue
D 5 5 Blue
E 4 4 Red

Step 1 — Calculate distances from P(3,4) to all points:

Point Distance Class

A(1,2) √[(3−1)²+(4−2)²] = √8 = 2.83 Red

B(2,3) √[(3−2)²+(4−3)²] = √2 = 1.41 Red
C(3,1) √[(3−3)²+(4−1)²] = √9 = 3.00 Blue
D(5,5) √[(3−5)²+(4−5)²] = √5 = 2.24 Blue
E(4,4) √[(3−4)²+(4−4)²] = √1 = 1.00 Red

Step 2 — Sort by distance, pick K=3 nearest:

E (1.00, Red), B (1.41, Red), D (2.24, Blue)
Step 3 — Majority vote: Red=2, Blue=1 → P is classified as RED ✓
CHOOSING K PROS & CONS
— Small K → fits training data well but noisy (overfitting) — ✓ Simple, no training phase
— Large K → smoother boundary but may miss patterns — ✓ Works well for small datasets
— Common choice: K = √n (n = total points) — ✗ Slow for large data (calculates distance to all points)

— Always pick an odd K to avoid ties — ✗ Sensitive to irrelevant features and scale
06 Naive Bayes
Naive Bayes is a probabilistic classification algorithm based on Bayes' Theorem. It's called "naive" because it assumes all features are
independent of each other (which is rarely true but works surprisingly well in practice).

BAYES' THEOREM

P(Class | Features) = [ P(Features | Class) × P(Class) ] / P(Features)

In plain English:
Probability of a class given features = (Likelihood of features in that class × Prior
probability of class) / Evidence

We compare this for each class and pick the one with highest probability.
📊 NUMERICAL EXAMPLE SPAM EMAIL CLASSIFICATION

Problem: An email has features: "Buy"=Yes, "Cheap"=Yes, "Meeting"=No. Is it Spam or Not Spam?
Training Data (10 emails):

Email Buy Cheap Meeting Class

1 Yes Yes No Spam
2 Yes No No Spam
3 Yes Yes No Spam
4 No No Yes Not Spam
5 No No Yes Not Spam
6 Yes No Yes Not Spam
7 No No No Not Spam
8 Yes Yes No Spam
9 No Yes No Spam
10 No No Yes Not Spam

Step 1 — Prior Probabilities:

P(Spam) = 5/10 = 0.5 P(Not Spam) = 5/10 = 0.5

Step 2 — Likelihoods:

Feature P(feature|Spam) P(feature|Not Spam)

Buy = Yes 4/5 = 0.8 2/5 = 0.4

Cheap = Yes 4/5 = 0.8 1/5 = 0.2
Meeting = No 5/5 = 1.0 2/5 = 0.4

Step 3 — Calculate Posterior (ignore denominator, same for both):

P(Spam | X) ∝ 0.5 × 0.8 × 0.8 × 1.0 = 0.320

P(Not Spam | X) ∝ 0.5 × 0.4 × 0.2 × 0.4 = 0.016

Result: 0.320 > 0.016 → Email is classified as SPAM ✓

WHEN TO USE NAIVE BAYES LAPLACE SMOOTHING
— Text classification (spam, sentiment) If a word never appeared in training for a class, P=0 kills the
entire probability. Add 1 to all counts (Laplace smoothing) to
— When features are mostly independent
avoid zero probabilities:
— When training data is small
— Real-time prediction (very fast) P = (count + 1) / (total + V) Where V =
number of distinct values.
07 Recommender Systems
Recommender systems automatically suggest items to users based on their preferences or behavior. Used by Netflix, Amazon, Spotify, YouTube,
etc.

Types of Recommender Systems Exam Topic

TYPE HOW IT WORKS REAL EXAMPLE ADVANTAGE LIMITATION

Collaborative Find users with similar Netflix: recommends No need to know Cold start problem
Filtering behavior/taste and recommend based on viewing product details. Finds (new users/items
what they liked. "Users like you history of similar hidden patterns. have no history)
also liked…" users
Content-Based Analyze features of items the user Spotify: recommends Works even with one Limited to things
Filtering liked and recommend similar songs similar to ones user. No need for similar to what user
items. Uses ML to classify you play other users' data. already likes (no
interesting vs uninteresting discovery)
items.

Knowledge- Uses explicit knowledge about Financial advisor Works when no Hard to build and
Based user needs and product features. tools, real estate history is available. maintain the
Decision trees, case-based search Highly accurate if knowledge base
reasoning. rules are good.

Cosine Similarity
Cosine similarity measures how similar two items are regardless of size. It measures the angle between two vectors, not the distance.

COSINE SIMILARITY FORMULA

cos(θ) = (A · B) / (|A| × |B|) Where A·B = dot product, |A| and |B| = magnitudes.

Output range: 0 to 1
0 = completely different, 1 = identical
📊 COSINE SIMILARITY EXAMPLE MOVIE RECOMMENDATION

Problem: Two movies are represented as feature vectors. How similar are they?
Movie A (Action, Romance, Comedy): [3, 1, 0]
Movie B (Action, Romance, Comedy): [2, 3, 1]

Step 1 — Dot product (A · B):

A · B = (3×2) + (1×3) + (0×1) = 6 + 3 + 0 = 9

Step 2 — Magnitudes:

|A| = √(3² + 1² + 0²) = √10 = 3.162 |B| = √(2² + 3² + 1²) = √14 = 3.742

Step 3 — Cosine Similarity:

cos(θ) = 9 / (3.162 × 3.742) = 9 / 11.83 = 0.761

Result: 0.761 — These movies are quite similar (76% similarity). They would be good recommendations for each other.

CONTENT-BASED FILTERING USING COSINE SIMILARITY (FROM LECTURE CODE)

The movie recommendation system works by:
1 Combine features: keywords + cast + genres + director into one string per movie

2 Use CountVectorizer to convert text into a numeric matrix

3 Compute cosine similarity matrix between all movies

4 Given user's liked movie, find top-N most similar movies by sorting the cosine scores
08 PCA — Principal Component Analysis
PCA is a dimensionality reduction technique. When you have too many features (columns), PCA reduces them to fewer "principal
components" while keeping most of the information.

Why PCA?
— Too many features → model trains slowly, harder to visualize
— Some features carry the same information (correlated)
— PCA finds the directions of maximum variance and projects data onto them

KEY CONCEPTS EXPLAINED VARIANCE RATIO

— PC1 = direction of highest variance in data Tells you how much original information each PC keeps.
If PC1 = 0.85 → it captures 85% of the data's variance.
— PC2 = second highest, perpendicular to PC1
Use enough PCs to retain 95%+ variance.
— Principal Components are uncorrelated (orthogonal)
— PCA creates NEW features (linear combinations of originals)

PCA IN PYTHON (5-MARK CODE)

from [Link] import PCA

from [Link] import StandardScaler

# Always scale before PCA!

sc = StandardScaler()
X_scaled = sc.fit_transform(X)

# Apply PCA — keep 95% of variance

pca = PCA(n_components=0.95)
X_pca = pca.fit_transform(X_scaled)

# See how much each component explains

print(pca.explained_variance_ratio_)

PCA vs Feature Selection: PCA creates new combined features (original meaning may be lost). Feature Selection keeps original
features and just removes the less useful ones.
09 Theory Topics (Lecture 10 & 11)

Supervised vs Unsupervised Learning

TYPE HAS LABELS? GOAL ALGORITHMS

Supervised Yes — every sample has a Learn a mapping from input to Decision Tree, KNN, Naive Bayes, SVM,
correct answer output Logistic Regression
Unsupervised No — raw data only Find patterns/structure in data K-Means, PCA, Autoencoders

Semi- Few labels Use small labeled + large Self-training, Co-training

supervised unlabeled data
Reinforcement Reward signal Learn by trial and error Q-Learning, Deep Q-Networks

Classification vs Regression vs Clustering

CLASSIFICATION REGRESSION
Output is a category/class. Output is a continuous number.
Examples: Spam/Not Spam, Disease/Healthy, Cat/Dog. Examples: House price, temperature, stock price.
Algorithms: KNN, Decision Tree, Naive Bayes. Algorithms: Linear Regression, Decision Tree Regressor.

CLUSTERING OVERFITTING VS UNDERFITTING

Unsupervised — group similar items together without labels. Overfitting: Model memorizes training data, fails on new data
Examples: Customer segmentation, document grouping. (too complex).
Algorithm: K-Means. Underfitting: Model too simple, can't capture patterns.
Fix: More data, cross-validation, pruning, regularization.

Confusion Matrix & Accuracy

METRIC FORMULA MEANING

Accuracy (TP + TN) / Total Overall correct predictions

Precision TP / (TP + FP) Of all predicted positives, how many are actually positive?

Recall TP / (TP + FN) Of all actual positives, how many did we catch?
F1 Score 2 × (Precision × Recall) / (Precision + Recall) Balance of Precision and Recall

TP=True Positive, TN=True Negative, FP=False Positive, FN=False Negative

10 Scenario-Based Questions & Answers
These questions describe a real-world problem and ask: Which algorithm would you use and why? No numericals — focus on justification.

ANSWER FRAMEWORK
State: (1) What type of problem it is, (2) Which algorithm, (3) Why this algorithm fits, (4) What recommendation system type if
applicable.

SCENARIO 1
Problem: An e-commerce company wants to suggest products to customers based on their past purchases and browsing history. Which
algorithm and recommendation system would you use?

ANSWER
Type: Recommendation problem.
Recommendation System: Collaborative Filtering — because we have purchase/browsing history from many users. We can
find users with similar behavior and recommend products they liked.
ML Algorithm: KNN (K-Nearest Neighbors) can be used to find similar users based on purchase patterns (Euclidean or cosine
similarity).
Justification: Collaborative filtering works well when you have a large user base with historical interactions. It doesn't need to
know product details — just behavior patterns.

SCENARIO 2
Problem: A hospital wants to automatically classify incoming patient emails as "Urgent" or "Non-Urgent" based on keywords. Which
algorithm would you use?

ANSWER
Type: Binary classification problem with text data.
Algorithm: Naive Bayes — it is the standard choice for text classification.
Justification: Naive Bayes calculates the probability of each class given the words in the email. It works extremely well for text, is
fast, and handles high-dimensional word features naturally. Even with limited training data, it performs well.

SCENARIO 3
Problem: A bank has thousands of customers and wants to segment them into groups (e.g., high-value, low-value, at-risk) without any
predefined labels. Which algorithm would you use?
 ANSWER
Type: Unsupervised clustering problem (no labels).
Algorithm: K-Means Clustering.
Justification: Since there are no predefined categories, we cannot use supervised learning. K-Means groups customers by
similarity in spending, frequency, balance, etc. Each cluster represents a customer segment. The number of segments K can be
chosen using the Elbow method.

SCENARIO 4
Problem: A streaming service wants to recommend movies to a new user who has only watched 2 movies so far. Which recommendation
system is best?

ANSWER
Challenge: Cold start problem — very little user history.
Recommendation System: Content-Based Filtering — because we can analyze the 2 movies the user watched (genre, director,
cast, keywords) and recommend similar ones.
Algorithm: Cosine Similarity on movie feature vectors (from CountVectorizer on combined features).
Justification: Collaborative filtering fails here because there's no sufficient behavioral history. Content-based filtering works
from the first interaction by matching item features.

SCENARIO 5
Problem: A dataset has 500 features (columns). Training is very slow and accuracy is poor. How would you handle this?

ANSWER
Problem: Curse of dimensionality — too many features cause overfitting and slow training.
Solution 1: Apply PCA to reduce to fewer principal components while keeping 95%+ variance.
Solution 2: Apply Feature Selection (Filter/Wrapper/Embedded methods) to remove irrelevant features.
Justification: PCA is preferred when features are correlated. Feature selection is preferred when interpretability matters. Always
scale features before PCA.

SCENARIO 6
Problem: A doctor wants to predict whether a tumor is malignant or benign based on 10 patient measurements. The data is labeled.
Which algorithm would you choose?
 ANSWER
Type: Binary supervised classification.
Best Algorithms: Decision Tree or KNN.
Decision Tree preferred because: It is interpretable — doctors need to understand WHY a prediction is made. Decision trees
provide clear if-then rules based on feature thresholds.
KNN alternative: Good for small datasets, simple to implement, but less interpretable and slow at test time.

SCENARIO 7
Problem: A user is looking for a financial advisor recommendation system. They specify their income range, risk tolerance, and
investment goals. Which system type is best?

ANSWER
Recommendation System: Knowledge-Based System.
Justification: Here we have explicit user requirements (income, risk, goals) rather than historical behavior. Knowledge-based
systems use decision rules/trees and case-based reasoning to match user requirements to products. No previous user data is
needed — it reasons from the specified constraints.
11 Quick Code Reference (5-mark questions)

Maximum 5-mark code questions. Know these templates well — they are short and predictable.

KNN Classifier

from [Link] import KNeighborsClassifier

from sklearn.model_selection import train_test_split
from [Link] import accuracy_score

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

knn = KNeighborsClassifier(n_neighbors=3)
[Link](X_train, y_train)
y_pred = [Link](X_test)
print("Accuracy:", accuracy_score(y_test, y_pred))

Naive Bayes Classifier

from sklearn.naive_bayes import GaussianNB

from [Link] import accuracy_score

model = GaussianNB()
[Link](X_train, y_train)
y_pred = [Link](X_test)
print("Accuracy:", accuracy_score(y_test, y_pred))

Decision Tree Classifier

from [Link] import DecisionTreeClassifier

dt = DecisionTreeClassifier(criterion='entropy', random_state=0)
[Link](X_train, y_train)
y_pred = [Link](X_test)
print("Accuracy:", accuracy_score(y_test, y_pred))

K-Means Clustering

from [Link] import KMeans

import [Link] as plt

kmeans = KMeans(n_clusters=3, random_state=0)

[Link](X)
labels = kmeans.labels_
centers = kmeans.cluster_centers_

[Link](X[:,0], X[:,1], c=labels, cmap='viridis')

[Link](centers[:,0], centers[:,1], c='red', marker='X', s=200)
[Link]()

Cosine Similarity (Movie Recommender)

from sklearn.feature_extraction.text import CountVectorizer

from [Link] import cosine_similarity
import pandas as pd

df = pd.read_csv('[Link]')
features = ['keywords', 'cast', 'genres', 'director']
for f in features:
df[f] = df[f].fillna('')

df['combined'] = df[features].apply(lambda row: ' '.join(row), axis=1)

cv = CountVectorizer()
matrix = cv.fit_transform(df['combined'])
cosine_sim = cosine_similarity(matrix)

# Get recommendations for a movie

idx = df[df['title'] == 'Movie Name'].index[0]
sim_scores = list(enumerate(cosine_sim[idx]))
sim_scores = sorted(sim_scores, key=lambda x: x[1], reverse=True)[1:6]
for i, score in sim_scores:
print(df['title'].iloc[i], round(score, 3))

PCA

from [Link] import PCA

from [Link] import StandardScaler

sc = StandardScaler()
X_scaled = sc.fit_transform(X_train)

pca = PCA(n_components=2) # reduce to 2 components

X_pca = pca.fit_transform(X_scaled)
print("Variance explained:", pca.explained_variance_ratio_)
FINAL EXAM CHEAT SHEET

ALGORITHM TYPE LABELED? OUTPUT KEY CONCEPT

KNN Supervised Yes Class Distance to K neighbors, majority

vote

Naive Bayes Supervised Yes Class Bayes theorem, P(Class|Features)

Decision Tree Supervised Yes Class/Value Information Gain, Entropy

K-Means Unsupervised No Clusters Centroid update, Euclidean
distance

PCA Dimensionality No New Maximum variance directions

Reduction features
Collaborative Recommendation Behavior Items Similar user behavior
Filtering

Content-Based Recommendation Item Items Similar item features

features

Knowledge-Based Recommendation Rules Items Explicit user requirements