UNIT 1

What is Machine Learning?

Machine Learning (ML) is a way to teach computers to learn from data instead of giving them xed rules.
• Normally, we write step-by-step instructions for a computer.
• But in ML, we just give the data, and it learns patterns from them.
• Later, it can use those patterns to make decisions or predictions on new data.
👉 Example: If you give a computer thousands of photos of cats and dogs, it can learn to recognize whether
a new photo is a cat or a dog.

Why is Machine Learning Important?

1. Automation – Computers can do tasks without human help. (Ex: spam email detection)
2. Better Decisions – Helps in analyzing large data to make smart choices. (Ex: Net ix recommending
movies)
3. Accuracy Improves Over Time – The more data it sees, the better it becomes.
4. Saves Time & Effort – Machines can process things faster than humans.
5. Used Everywhere – Healthcare (disease detection), Banking (fraud detection), E-commerce
(recommendations), Self-driving cars, etc.
Comparison: Machine Learning vs Traditional Programming
👉 Machine Learning (ML):
• ML tries to build its own rules/logic by looking at input data and the correct answers.
• It means the computer learns automatically from examples.
• The more data it gets, the better it learns and improves over time.
👉 Traditional Programming:
• Here, the programmer must write all the rules/logic manually.
• The computer only follows those xed rules.
• It does not learn or improve on its own – if rules change, programmer must change code
🔹 Data Formats in Machine Learning
In ML, data can be stored and processed in different formats depending on the type of problem. The most common formats are:
1. Tabular / Structured Data
fi
fl
fi
 • Data arranged in rows and columns (like Excel or SQL tables).
• Each row = one sample, each column = one feature.
• Example:

Point Arti cial Intelligence (AI) Machine Learning (ML)

1 Broad eld to make machines smart Subset of AI that learns from data
2 Focuses on decision-making & problem-solving Focuses on learning & prediction
3 Includes reasoning, planning, creativity Mainly pattern recognition & data learning
4 Goal: simulate human intelligence Goal: improve automatically with data
5 Works with rules + data Works mostly with data only
6 Example: Chatbot, self-driving car Example: Spam lter, recommendation system

Age Height Weight Label

• 25 170 65 Healthy
• 30 165 70 Unhealthy
• Used in: Regression, Classi cation.
2. Text Data
• Data in the form of sentences, documents, words.
• Needs preprocessing like tokenization, stop-word removal, embeddings.
• Example:
"Machine learning is amazing"
• Used in: Sentiment Analysis, Chatbots, NLP tasks.
3. Image Data
• Data stored as pixels in 2D (grayscale) or 3D (RGB).
• Each pixel has a value (0–255).
• Example:
◦ Grayscale digit image (28×28 pixels in MNIST).
◦ Color image (width × height × 3 channels).
• Used in: Computer Vision, Face Recognition.
4. Audio Data
• Data in the form of sound signals or waveforms.
• Usually converted into spectrograms or feature vectors (like MFCC).
• Example: Voice commands (“Ok Google”).
• Used in: Speech Recognition, Music classi cation.
5. Video Data
• Sequence of images (frames) with audio.
• Very large in size, needs feature extraction.
• Example: Activity recognition in CCTV.
• Used in: Object detection, Self-driving cars.
Supervised Learning
👉 It is a type of Machine Learning where the model learns from labeled data (data that already has
answers).
fi
fi
fi
fi
fi
 • Think of it like a teacher supervising a student. The teacher gives the correct answers, and the
student learns how to solve similar problems in the future.
Key Points
1. Labeled Data → Input + Correct Output are already given.
Example:
◦ Input: Picture of a cat 🐱
◦ Output (label): "Cat"
2. Goal → Train the machine to predict the output for new, unseen inputs.
3. Types:
◦ Classi cation → Predicting a category/class.
Example: Is this email Spam or Not Spam?
◦ Regression → Predicting a number.
Example: Predicting house price based on size, location, etc.
Diagram
Training Data (Input + Output) ---> Machine Learning Model --->
Prediction
Example:
[Study Hours → 2, Marks → 50]
[Study Hours → 4, Marks → 70]
[Study Hours → 6, Marks → 90]
↓
Model Learns Relationship
↓
Input: Study Hours = 5 → Model predicts Marks ≈ 80
✅ Simple Example in Real Life:
• If you show the computer many dog 🐶 and cat 🐱 pictures with their labels,
later it can correctly say "This is a dog" or "This is a cat" when you give a new picture.

1. Classi
cation
• De nition: Classi cation is a supervised learning task where the goal is to predict a discrete label or category.
• Output: Categorical ( nite classes).
• Example:
◦ Email → Spam or Not Spam
◦ Tumor → Benign or Malignant
◦ Weather → Sunny, Cloudy, Rainy
Diagram (simple view):

Input Data → Model → Predicted Class (Category)

2. Regression
• De nition: Regression is a supervised learning task where the goal is to predict a continuous value.
• Output: Numerical (real values).
• Example:
◦ Predicting house price based on size & location
◦ Predicting temperature tomorrow
◦ Predicting stock market price
Diagram (simple view):
Input Data → Model → Predicted Value (Number)

Unsupervised Learning
👉 In unsupervised learning, the computer is given only data (input) but no answers (output labels).
It has to nd hidden patterns, groups, or structures on its own.
✅ Key Points:
• No teacher (no labels).
• The system learns by itself.
• Mainly used for clustering (grouping) and association ( nding relations).
fi
fi
fi
fi
fi
fi
fi
fi
 📌 Examples:
• Grouping customers in a shop based on their buying habits.
• Grouping students by study patterns.
• Market basket analysis → “People who buy bread also buy butter.”
🔎 Diagram (simple):
Data (no labels) ---> ML Algorithm ---> Groups / Patterns
Semi-Supervised Learning
👉 In semi-supervised learning, the computer is given a small amount of labeled data (with answers) and
a large amount of unlabeled data (without answers).
It uses the small labeled part to guide learning from the bigger unlabeled part.
✅ Key Points:
• Mix of Supervised + Unsupervised.
• Uses few labeled data + many unlabeled data.
• Helpful when labeling data is expensive or time-consuming.
📌 Examples:
• Google Photos: Only a few photos are tagged (labeled), the rest are auto-grouped.
• Medical images: Only some X-rays are labeled by doctors, but the system learns from many
unlabeled scans.
• Spam ltering: Few labeled emails + many unlabeled emails.
🔎 Diagram (simple):
Small Labeled Data + Large Unlabeled Data
↓
Semi-Supervised Algorithm
↓
Better Predictions / Groups

Feature Supervised Learning Unsupervised Learning Semi-Supervised Learning

Data
Labeled data only Unlabeled data only Mix of labeled + unlabeled
Used
Improve learning using few labels with many unlabeled
Goal Predict outcomes (classi cation/regression) Find hidden patterns/grouping
samples

Known & measurable (e.g., spam or not Unknown patterns (e.g., Better accuracy than unsupervised, less labeling than
Output
spam) customer segments) supervised

Algorith Linear Regression, Decision Trees, SVM, K-Means, Hierarchical Self-training, Semi-supervised SVM, Graph-based
ms Neural Networks Clustering, PCA methods

Use Market segmentation, anomaly Medical diagnosis (few labeled records, many
Email ltering, price prediction
Cases detection unlabeled), Speech recognition

Reinforcement Learning
🔹 De nition
Reinforcement Learning is a type of machine learning where an agent learns to make decisions by
interacting with an environment to achieve a goal.
It learns through trial and error, receiving feedback in the form of rewards (positive) or penalties
(negative).

🔹 Key Components
1. Agent – The learner/decision maker (e.g., robot, software).
fi
fi
fi
fi
 2. Environment – The system the agent interacts with.
3. State (S) – Current situation of the agent in the environment.
4. Action (A) – Choices available to the agent.
5. Reward (R) – Feedback from the environment after an action.
6. Policy (π) – Strategy that de nes the agent’s actions in each state.
7. Value Function – Estimates how good a state or action is for long-term rewards.

🔹 Process Flow
1. Agent observes the current state (S).
2. Agent chooses an action (A) based on its policy.
3. Environment responds with a new state (S') and a reward (R).
4. Agent updates its policy to maximize future rewards.

🔹 Types of Reinforcement Learning

1. Positive Reinforcement – Adding a reward after correct action.
2. Negative Reinforcement – Removing an undesirable outcome when action is correct.

🔹 Applications
• Game playing (e.g., AlphaGo, Chess, Atari games)
• Robotics (navigation, manipulation)
• Self-driving cars
• Recommendation systems
• Healthcare (treatment strategies)

Geometric Model (in Machine Learning & AI)

• A geometric model is a way of representing data or problems using shapes, points, and spaces
(geometry).
• It sees data as points in space (like on a graph with x, y, z axes).
• The model then uses lines, planes, or curves to separate or group these points.
Examples
1. Classi cation:
◦ Imagine two types of fruits (apples & oranges) plotted as points based on weight and color
intensity.
◦ A straight line (or curve) can be drawn to separate apples from oranges. That line is part of a
geometric model.
2. Regression:
◦ If we plot house price vs. house size, the model nds the best line/curve that ts the points.
Importance
• Helps us visualize complex data.
• Very useful in pattern recognition, image processing, and 3D modeling.

🔹 Geometric Models
• De nition:
Geometric models are a type of statistical learning model that use geometric concepts (like distance,
similarity, and regions in space) to make predictions.
• They represent data points as vectors in a multi-dimensional space.
• The model then uses geometry (distances, boundaries, regions) to classify or predict outcomes.
🔹 Types of Geometric Models
1. Instance-Based Models
• Predictions are made based on similarity to training examples.
• No explicit formula, model remembers training data.
• Example Algorithms:
fi
fi
fi
fi
fi
 ◦ k-Nearest Neighbors (kNN): Predicts label based on majority of k closest neighbors.
◦ Radial Basis Function methods.
👉 Example: If we want to classify a new fruit, we look at the nearest known fruits in feature space (weight, color,
sweetness).
2. Linear Models
• Use straight lines (2D), planes (3D), or hyperplanes (higher dimensions) to separate data.
• Very fast and simple.
• Example Algorithms:
◦ Linear Regression (predict continuous values)
◦ Logistic Regression (binary classi cation)
◦ Support Vector Machines (SVM with linear kernel)
👉 Example: Separating spam vs. non-spam emails using a line/hyperplane in feature space.
3. Non-Linear Models (Kernel Methods)
• Extend linear models to work in a transformed space (using kernels).
• Allow more complex, curved decision boundaries.
• Example Algorithm:
◦ SVM with polynomial/RBF kernel
👉 Example: Classifying data that cannot be separated with a straight line (like concentric circles).

Probabilistic Models in Machine Learning

👉 De nition:
Probabilistic models are machine learning models that use probability theory to deal with uncertainty in
predictions.
Instead of giving only a xed output, they give the probability of different outcomes.

Key Points:
1. Based on statistics and probability theory.
2. Represent data and predictions as probability distributions.
3. Useful when the data has noise or uncertainty.
4. They help in making predictions like “What is the probability that this email is spam?”

Examples of Probabilistic Models:

• Naive Bayes Classi er (used in spam ltering).
• Hidden Markov Models (HMMs) (used in speech recognition).
• Bayesian Networks (used in medical diagnosis).
• Gaussian Mixture Models (GMMs) (used in clustering).
For example:
📧 Email = “Free money offer” → Spam (90%), Not Spam (10%)
Parametric Models
• De nition: Parametric models are models that summarize data using a xed number of
parameters.
Once the parameters are learned from the training data, the model does not grow in complexity, no
matter how large the dataset becomes.
• Key Idea:
◦ The model assumes a speci c functional form (like a straight line or curve).
◦ It tries to t the data into this form by learning the parameters.
◦ Example: In linear regression, the line is represented as y = mx + c. The parameters are
m (slope) and c (intercept).
• Advantages:
◦ Simple, fast, and less expensive.
◦ Requires less data to train compared to non-parametric models.
• Disadvantages:
fi
fi
fi
fi
fi
fi
fi
fi
fi
 ◦ Can be less exible because the model is restricted by its xed form.
◦ May perform poorly if the data does not t the assumed function.
• Examples:
◦ Linear Regression
◦ Logistic Regression
◦ Naïve Bayes
◦ Perceptron
Non-Parametric Models
• These models do not make strong assumptions about the form of the data distribution.
• They do not x the number of parameters before training.
• Instead, the complexity grows with the data size (more data → more model complexity).
• Useful when the relationship between input and output is very complex or unknown.

Key Features
1. No xed parameters → Model adjusts based on the dataset.
2. Flexible → Can t more complex patterns in data.
3. Require more data & computation compared to parametric models.
4. Often used when data does not follow a clear mathematical equation.
❌ Disadvantages:
• Slower, needs more data & computation.
Examples
• K-Nearest Neighbors (KNN)
• Decision Trees
• Random Forests
• Support Vector Machines

Aspect Parametric Models Non-Parametric Models

Assumes a xed number of parameters to Does not assume a xed number of parameters;
De nition
de ne the model model complexity grows with data
Strong assumptions about data distribution
Assumption Few or no assumptions about data distribution
(e.g., Normal distribution)

Flexibility Less exible, may not t complex data well More exible, can adapt to complex patterns

Linear Regression, Logistic Regression, k-Nearest Neighbors (kNN), Decision Trees,

Examples
Naive Bayes Random Forests

Training Speed Faster training (since fewer parameters) Slower training (complex computations with data)

Data
Works well with smaller datasets Requires large amounts of data to perform well
Requirement

Interpretability Easy to interpret Harder to interpret due to complexity

Risk of
Lower (simpler models) Higher (very exible, may over t small data)
Over tting

🔹 What is Statistical Learning?

• Statistical learning is a framework in Machine Learning that uses statistics + algorithms to nd
patterns in data.
• It focuses on the relationship between input (X) and output (Y).
fi
fi
fi
fl
fi
fl
fi
fl
fl
fi
fi
fi
fi
fi
fi
fi
fi
 • Used for prediction, classi cation, and inference.

🔹 Main Approaches in Statistical Learning:

1. Supervised Learning
◦ Data has inputs (features X) and outputs (labels Y).
◦ Goal: Learn a function to predict Y from X.
◦ Examples:
▪ Linear Regression (predict prices)
▪ Logistic Regression (predict spam / not spam)
▪ Support Vector Machines
2. Unsupervised Learning
◦ Data has only inputs (X), no outputs.
◦ Goal: Find hidden patterns or groupings.
◦ Examples:
▪ Clustering (K-Means, Hierarchical Clustering)
▪ Dimensionality Reduction (PCA)
3. Semi-Supervised Learning
◦ Uses some labeled data + a lot of unlabeled data.
◦ Useful when labeling is expensive.
◦ Example: Web page classi cation with few labeled pages.
4. Reinforcement Learning (Statistical Decision Making)
◦ Learns by trial and error using rewards & penalties.
◦ Example: Self-driving cars, game-playing bots.

🔹 Key Characteristics:
• Uses probability & statistics (e.g., Bayes’ theorem, regression).
• Deals with uncertainty in data.
• Helps in both prediction (future outcomes) and inference (understanding data relationships).

🔹 Applications of ML in Data Science

1. Predictive Analytics
◦ Predict future outcomes from past data.
◦ Example: Predicting sales, stock prices, or customer churn.
2. Classi cation
◦ Categorizing data into prede ned classes.
◦ Example: Email spam detection, disease diagnosis.
3. Clustering
◦ Grouping similar data points without labels.
◦ Example: Customer segmentation in marketing.
4. Recommendation Systems
◦ Suggest items based on user behavior.
◦ Example: Net ix recommending movies, Amazon product suggestions.
5. Natural Language Processing (NLP)
◦ Analyzing and understanding human language.
◦ Example: Sentiment analysis on tweets, chatbots.
6. Computer Vision
◦ Extracting insights from images/videos.
◦ Example: Face recognition, medical image analysis.
7. Anomaly Detection
◦ Finding unusual patterns or fraud.
◦ Example: Fraud detection in banking, network intrusion detection.
fi
fl
fi
fi
fi
How a Machine Learning Model Works?
A machine learning (ML) model works by learning patterns from data and then using these patterns to
make predictions or decisions on new, unseen data.
Steps Involved in Building an ML Model
1. Data Collection
• Gather raw data from different sources (databases, sensors, les, APIs, etc.).
• Example: Collecting images of cats and dogs.
2. Data Preprocessing (Cleaning)
• Handle missing values.
• Remove duplicates.
• Convert data into a usable format (numbers, categories, etc.).
• Example: Converting “Yes/No” into 1/0.
3. Feature Selection / Extraction
• Choose the most important variables (features) that affect the output.
• Example: For house price prediction → select features like size, location, number of rooms.
4. Splitting the Dataset
• Split the dataset into:
◦ Training set (70–80%) → to train the model.
◦ Testing set (20–30%) → to check accuracy.
5. Choosing a Model (Algorithm)
• Select a suitable ML algorithm based on the problem:
◦ Linear Regression (for prediction of numbers),
◦ Decision Trees,
◦ Neural Networks,
◦ SVM, etc.
6. Training the Model
• Give the training data to the chosen algorithm.
• The model learns the relationship between input (X) and output (Y).
• Example: Learning how house size affects price.
7. Model Evaluation
• Test the model on unseen test data.
• Measure performance using metrics like:
◦ Accuracy,
◦ Precision/Recall,
◦ Mean Squared Error, etc.
8. Model Optimization (Tuning)
• Improve model performance using:
◦ Hyperparameter tuning,
◦ Feature engineering,
◦ Cross-validation.
9. Deployment
• Use the trained model in real life for predictions.
• Example: A spam email lter running in Gmail.
Grouping and Grading Models
1. Grouping Models
• Meaning: These models divide data into groups or clusters based on similarities.
• Purpose: To nd hidden patterns and structure in data without prede ned labels.
• Used in: Unsupervised Learning (no output labels).
• Examples:
◦ Customer segmentation (grouping customers by purchase behavior).
◦ Document clustering (grouping news articles by topic).
fi
fi
fi
fi
 • Techniques:
◦ K-Means Clustering
◦ Hierarchical Clustering

2. Grading Models
• Meaning: These models assign a grade, score, or category to data.
• Purpose: To classify or rank data into ordered levels or classes.
• Used in: Supervised Learning (we already know labels/grades).
• Examples:
◦ Student grading (A, B, C, D, Fail).
◦ Credit scoring (Good, Average, Bad).
◦ Spam classi cation (Spam/Not Spam).
• Techniques:
◦ Logistic Regression
◦ Decision Trees
◦ Support Vector Machines (SVM)

UNIT 2

Feature Engineering – Simple Explanation

• De nition: Feature engineering is the process of creating, selecting, or transforming input
variables (features) in your data to help a machine learning model perform better.
• Why it is important: Good features make it easier for the model to learn patterns and give better
predictions.
• Steps in Feature Engineering:
1. Feature Creation – Make new features from existing data.
▪ Example: From “Date of Birth,” create “Age.”
2. Feature Transformation – Change features to a better form.
▪ Example: Convert text categories into numbers.
3. Feature Selection – Keep only the important features and remove useless ones.
▪ Example: Ignore columns that don’t affect the outcome.
4. Feature Scaling/Normalization – Make features on a similar scale.
▪ Example: Height in cm and Weight in kg → scale both to 0–1.
• Goal: To make the data more meaningful so the model can learn faster and predict accurately.

Feature Selection – Simple Explanation

• De nition: Feature selection is the process of choosing only the most important features from
your data and removing the irrelevant or less useful ones.
• Why it is important:
1. Makes the model faster and simpler.
2. Reduces over tting (model learning noise instead of patterns).
3. Improves accuracy.
• Types of Feature Selection:
1. Filter Methods – Select features based on statistics (like correlation, chi-square).
2. Wrapper Methods – Test different combinations of features to see which works best.
3. Embedded Methods – Features are selected during model training (like in decision trees).
• Example: If predicting house prices, you might remove features like “house color” if it doesn’t
affect the price.
Handling Missing Values
De nition: Sometimes, data is incomplete and some values are missing. Handling missing values means
lling them or removing them so the model can work properly.
fi
fi
fi
fi
fi
fi
 • Why it is important:
◦ Missing data can make models inaccurate.
◦ Some algorithms cannot handle missing values.
• Ways to Handle Missing Values:
◦ Remove Missing Data – Delete rows or columns with missing values.
▪ Use when there are very few missing values.
◦ Fill with Mean/Median/Mode – Replace missing numbers with the average, middle, or
most common value.
▪ Example: Fill missing ages with the average age.
◦ Predict Missing Values – Use a model to predict the missing value based on other data.
◦ Use Special Value – Sometimes replace missing values with 0 or “Unknown”.
• Goal: Ensure the dataset is complete and ready for analysis.

Dimensionality Reduction in Machine Learning

• Dimensionality: In ML, this means the number of features (input variables) in your dataset.
• When there are too many features compared to the number of observations, some algorithms
struggle to learn properly. This problem is called the “Curse of Dimensionality.”
• Dimensionality Reduction: Techniques that reduce the number of features while keeping
important information.
Common Techniques:
1. Principal Component Analysis (PCA):
◦ Used for continuous data.
◦ Combines features in a way that features with most variance become principal
components.
2. Linear Discriminant Analysis (LDA):
◦ Projects data to maximize separability between classes.
3. Generalized Discriminant Analysis (GDA):
◦ Works well for non-linear features.
Advantages:
• Reduces storage space.
• Reduces computation time.
• Helps make faster and simpler models.
Disadvantages:
• May cause some data loss.

Principal Component Analysis (PCA) – Simple Explanation

• De nition: PCA is a technique to reduce the number of features in a dataset while keeping most of
the important information.
• It transforms the original features into new features called principal components. These
components capture the most variance (differences) in the data.
• Why use PCA:
1. Reduces computation time.
2. Helps visualize high-dimensional data.
3. Removes redundant or less important features.
• How it works:
1. Find the variance of each feature.
2. Identify components where data varies the most.
3. Project data onto these principal components.
• Example:
1. Original dataset: 10 features.
2. PCA reduces it to 2 or 3 important components without losing much information.
• Advantages:
1. Reduces storage and computation.
2. Helps in better visualization.
• Disadvantages:
fi
 1. Can lose some information.
2. Principal components are harder to interpret than original features.
Kernel PCA – Simple Explanation
• De nition: Kernel PCA is an extension of PCA that can handle non-linear data.
• While normal PCA works well for linear relationships, Kernel PCA can capture complex patterns
in data.
• How it works:
1. Use a kernel function (like RBF or polynomial) to transform data into a higher-
dimensional space.
2. Perform PCA in this new space.
3. Extract principal components that capture most variance.
• Why use Kernel PCA:
1. Works with non-linear relationships in data.
2. Can improve model performance when linear PCA is not enough.
• Advantages:
1. Captures complex patterns.
2. Reduces dimensions while keeping important information.
• Disadvantages:
1. Computationally expensive for large datasets.
2. Choosing the right kernel can be tricky.
Local Binary Pattern (LBP) – Simple Explanation
• De nition: LBP is a simple and ef cient feature extraction method used in image processing,
especially for texture analysis and face recognition.
• How it works:
1. For each pixel in an image, compare it with its neighboring pixels.
2. If a neighbor pixel’s value is greater or equal, mark it as 1; otherwise, mark it as 0.
3. Combine these 1s and 0s into a binary number (local binary pattern).
4. Use these patterns to describe the texture of the image.
• Advantages:
1. Simple and fast to compute.
2. Works well for texture classi cation.
3. Robust to changes in lighting.
• Disadvantages:
1. Only captures local information.
2. Sensitive to noise in the image.
• Example: Used in face recognition systems to identify patterns in facial textures.

Filtering Method – Feature Selection Technique (Simple Explanation)

• De nition: Filtering is a feature selection method where we choose features based on statistical
measures before training the model.
• It does not use any machine learning model; it just looks at the data itself.
• How it works:
1. Calculate scores for each feature using statistical tests (like correlation, chi-square,
ANOVA).
2. Rank features based on these scores.
3. Select top features and remove irrelevant ones.
• Advantages:
1. Fast and simple.
2. Works well for large datasets.
3. Reduces over tting.
• Disadvantages:
1. Ignores feature interactions (how features work together).
2. May miss important features if they alone do not show strong correlation.
• Example:
1. Predicting house prices: Keep features like size, location; remove features like house color
if it has no correlation with price.
fi
fi
fi
fi
fi
fi
 Wrapper Methods – Types
1. Greedy Search:
◦ Follows a path that seems best at the moment.
◦ Chooses features step by step based on immediate performance.
◦ Example: Recursive Feature Elimination (RFE) – removes least important features
recursively.
2. Non-Greedy Search:
◦ Looks at all possible feature subsets before choosing the best combination.
◦ Can nd a globally best solution.
◦ Examples: Genetic Algorithms (GA), Simulated Annealing (SA).

🔹 Wrapper Method (Feature Selection)

• A feature selection technique where we select subsets of features and test them by actually
training and evaluating a model.
• It “wraps” the model around different feature sets and chooses the best one.
• Unlike Filter methods (which use statistical tests), wrapper methods rely on model performance
(accuracy, error, etc.).

🔹 Steps
1. Select a subset of features.
2. Train a model on these features.
3. Evaluate model performance (e.g., accuracy).
4. Repeat with different subsets.
5. Choose the subset that gives the best result.

🔹 Types
• Forward Selection → Start with none, add features one by one.
• Backward Elimination → Start with all, remove features one by one.
• Recursive Feature Elimination (RFE) → Repeatedly build model and remove least important
features.

🔹 Examples
• Using k-Nearest Neighbors (kNN) or Decision Trees to test feature subsets.
• RFE with SVM or Logistic Regression.
✅ Advantages:
• More accurate than lter methods (since it considers model performance).
❌ Disadvantages:
• Computationally expensive (slow for large datasets).

🔹 Matrix Factorization
• Matrix factorization means breaking a big matrix into two small matrices.
• In recommender systems, the big matrix is the user–item rating table (who rated what).
• Since many ratings are missing, we break it into two parts:
◦ One shows user preferences
◦ Other shows item features
• By multiplying them back, we can guess the missing ratings.
👉 Example: If a user likes Action and Comedy movies, and another user with similar taste liked a new
Action movie, matrix factorization can predict that the rst user will also like it.
fi
fi
fi
 🔹 Content-Based Filtering
• In this method, the system recommends items that are similar to what the user liked before.
• It uses item features like genre, actors, description, keywords, etc.
• A user pro le is created based on past likes.
• Then the system nds items with similar content.
👉 Example:
If you watched and liked “Mission Impossible” (Action, Tom Cruise), the system may recommend:
• “Edge of Tomorrow” (Action + Tom Cruise)
• “Minority Report” (Sci-Fi + Tom Cruise)
Because these movies have similar features.

Why do you need categorical variable encoding?

• In datasets, many features are categorical (like colors: Red, Blue, Green or Gender: Male, Female).
fi
fi
 • Machine learning models work with numbers, not words or labels.

• So, categorical values must be converted into numeric form → this process is called categorical
variable encoding.

One-Hot Encoding (OHE)

• De nition: One-hot encoding is a method to convert categorical values into binary vectors (0s and
1s).

• Each category gets a new column with values 1 (present) or 0 (not present).

Example:

Suppose we have a column Color with values:

Red, Blue, Green, Red

After One-Hot Encoding:

Color Red Blue Green

Red 1 0 0
Blue 0 1 0
Green 0 0 1
Red 1 0 0
fi