Unit 4:

Introduction to Machine Learning:

Machine Learning (ML) is a branch of Artificial Intelligence that focuses on building systems that can
learn from data and improve their performance over time without being explicitly programmed.

What is machine learning:

Machine Learning is the process of teaching computers to recognize patterns, make decisions, and
predict outcomes based on data.

Instead of writing detailed step-by-step instructions, we provide data and algorithms that allow the
computer to learn on its own.

Example:

• Email spam filtering

• Movie recommendations

Definitions:

Types of learning:

Supervised:

Supervised Learning is a type of machine learning where the model is trained on labeled data (input
+ correct output).
The goal is to learn a mapping from inputs to outputs so it can predict results for new data.
➔ Think of it like learning with a teacher who gives both questions and correct answers.
How it works
• Input data (features) + Output labels
• Model learns patterns
• Used for:
o Classification (e.g., spam or not spam)
o Regression (e.g., predicting price)
Real-world Examples
1. Email Spam Detection
o Classifies emails as spam or not spam.
2. House Price Prediction
o Predicts price based on size, location, etc.
3. Medical Diagnosis
o Predicts diseases from patient data.
4. Credit Scoring
o Banks decide whether to approve loans.
5. Image Recognition
o Identifies objects (cats, cars, people) in images.
Key Tasks
1. Classification

1
 o Assigns data into categories
o Example: spam vs not spam
2. Regression
o Predicts continuous values
o Example: house price prediction
3. Ranking / Scoring
o Orders items based on importance
o Example: search engine results
4. Object Detection
o Identifies and locates objects in images
5. Time Series Prediction
o Predicts future values based on past data
o Example: stock prices, weather forecasting

Unsupervised:

Unsupervised Learning is where the model is trained on unlabeled data (no correct answers
provided).
The goal is to find hidden patterns or structures in the data.
➔ Think of it like learning without a teacher—finding patterns on your own.
How it works
• Only input data (no labels)
• Finds:
o Clusters (grouping similar items)
o Associations (relationships)
Real-world Examples
1. Customer Segmentation
o Businesses group customers by behaviour.
2. Market Basket Analysis
o Finds items frequently bought together.
3. Anomaly Detection (Fraud Detection)
o Detects unusual transactions.
4. Document Clustering
o Groups similar articles/news automatically.
5. Recommendation Systems (partially)
o Groups users/items with similar interests.
Key Tasks
1. Clustering
o Groups similar data points
o Example: customer segmentation
2. Association Rule Learning
o Finds relationships between variables
o Example: “people who buy X also buy Y”
3. Dimensionality Reduction
o Reduces number of features while keeping important info
o Example: data visualization
4. Anomaly Detection

2
 o Identifies unusual or rare data points
o Example: fraud detection
5. Density Estimation
o Models data distribution
o Example: probability modeling

Reinforcement learning (basic ideas):

Reinforcement Learning (RL) is a type of learning where an agent learns by interacting with an
environment and receiving rewards or penalties.
➔ Think of it like learning by trial and error.
How it works
• Agent takes actions
• Environment responds
• Receives:
o Reward (good action)
o Penalty (bad action)
• Goal: maximize total reward over time
Real-world Examples
1. Game Playing AI
o AI learns to play chess or video games.
2. Self-driving Cars
o Learns driving decisions from environment feedback.
3. Robotics
o Robots learn to walk or pick objects.
4. Online Ad Optimization
o Chooses best ads to maximize clicks.
5. Recommendation Systems (advanced)
o Improves suggestions based on user interaction.
Key Tasks
1. Policy Learning
o Learns the best strategy (set of actions)
2. Sequential Decision Making
o Makes decisions step-by-step over time
3. Control Tasks
o Controls systems dynamically
o Example: robotics, self-driving
4. Exploration vs Exploitation
o Balances trying new actions vs using known ones
5. Reward Optimization
o Maximizes long-term rewards

3
Classification:

Classification is a supervised learning task where the model assigns input data to predefined
categories (classes).
Key Points
• Uses labeled data
• Output is discrete (e.g., Yes/No, Spam/Not Spam)
• Widely used in decision-making systems
• Can be binary or multi-class
• Evaluated using accuracy, precision, recall
Types of Classification:
1. Binary Classification
• Involves two classes only
• Output: Yes/No, True/False
• Example: Spam vs Not Spam
2. Multi-class Classification
• Involves more than two classes
• Each input belongs to one class
• Example: Classifying animals (cat, dog, horse)
3. Multi-label Classification
• An input can belong to multiple classes at the same time
• Example: An image tagged as both “car” and “person”

Main Algorithms
1. Logistic Regression
A statistical algorithm used to predict the probability of a class using a logistic (sigmoid) function.
Points:
• Outputs probabilities between 0 and 1
• Works well for binary classification
• Simple and fast
• Assumes linear relationship
• Interpretable model
Real-world Examples:
• Email spam detection
• Disease prediction
• Credit approval

2. Decision Tree
A tree-based model that splits data based on conditions to classify outcomes.
Points:
• Easy to understand and visualize
• Handles both numerical & categorical data
• Prone to overfitting
• Uses rules (if–else structure)
• No need for data scaling
Real-world Examples:
• Loan approval systems
• Customer churn prediction

4
 • Medical diagnosis

3. K-Nearest Neighbors (KNN)

Classifies data based on the majority class of its nearest neighbors.
Points:
• Simple and intuitive
• No training phase (lazy learning)
• Sensitive to noise
• Requires distance calculation
• Works well for small datasets
Real-world Examples:
• Image classification
• Recommendation systems
• Pattern recognition

Regression:
Regression predicts continuous numerical values based on input features.
Key Points
• Output is continuous (e.g., price, temperature)
• Uses labeled data
• Helps in forecasting
• Measures relationship between variables
• Evaluated using MSE, RMSE
Types of Regression:
1. Linear Regression
• Relationship between variables is linear
• Example: Salary vs experience
2. Multiple Regression
• Uses multiple input variables
• Example: House price based on size, location, rooms
3. Polynomial Regression
• Models non-linear relationships using polynomial terms
• Example: Growth trends
4. Ridge and Lasso Regression
• Used to reduce overfitting
• Adds penalty terms to model
• Example: High-dimensional data problems

Main Algorithms
1. Linear Regression
Models the relationship between input and output using a straight line.
Points:
• Simple and widely used
• Assumes linear relationship
• Fast computation
• Sensitive to outliers
• Easy to interpret

5
Real-world Examples:
• House price prediction
• Sales forecasting
• Salary prediction

2. Decision Tree Regression

Uses tree structure to predict continuous values.
Points:
• Handles non-linear data
• No need for normalization
• Can overfit easily
• Splits data into regions
• Flexible model
Real-world Examples:
• Demand forecasting
• Energy consumption prediction
• Risk analysis

3. Support Vector Regression (SVR)

Uses support vector machines to predict continuous values within a margin.
Points:
• Works well for complex data
• Uses kernel functions
• Robust to outliers
• High accuracy
• Computationally expensive
Real-world Examples:
• Stock price prediction
• Traffic prediction
• Financial forecasting

Clustering:

Clustering is an unsupervised learning task that groups similar data points together.
Key Points
• No labeled data
• Finds hidden patterns
• Groups based on similarity
• Used in segmentation
• Results depend on distance measures
Types of Clustering:
1. Partitioning Clustering
• Divides data into K clusters
• Example: K-Means
2. Hierarchical Clustering
• Creates a tree of clusters (dendrogram)
• Example: Agglomerative clustering
3. Density-Based Clustering

6
 • Forms clusters based on data density
• Example: DBSCAN
4. Grid-Based Clustering
• Divides data into grids and clusters them
• Example: STING

Main Algorithms
1. K-Means
Divides data into K clusters based on nearest mean.
Points:
• Simple and fast
• Requires number of clusters (K)
• Sensitive to initial values
• Works best with spherical clusters
• Scales well to large data
Real-world Examples:
• Customer segmentation
• Image compression
• Market analysis

2. Hierarchical Clustering
Builds a tree of clusters (dendrogram).
Points:
• No need to specify clusters initially
• Works for small datasets
• Produces hierarchy
• Computationally expensive
• Easy to visualize
Real-world Examples:
• Gene analysis
• Document clustering
• Social network analysis

3. DBSCAN
Groups data based on density and identifies noise points.
Points:
• Detects arbitrary shaped clusters
• Handles noise well
• No need to specify cluster number
• Sensitive to parameters
• Works well with spatial data
Real-world Examples:
• Fraud detection
• Geographic data analysis
• Anomaly detection

7
Association:

Association finds relationships or patterns between variables in large datasets.

Key Points
• Used in market basket analysis
• Discovers frequent itemsets
• Works on transactional data
• Uses support & confidence
• Helps in recommendation systems
Types of Association:
1. Boolean Association Rules
• Works with binary data (0 or 1)
• Example: Bought or not bought
2. Quantitative Association Rules
• Uses numerical values
• Example: Age 20–30 buys product A
3. Single-Dimensional Association
• Involves one attribute
• Example: Product purchase patterns
4. Multi-Dimensional Association
• Involves multiple attributes
• Example: Age + income + product purchase

Main Algorithms
1. Apriori Algorithm
Finds frequent itemsets using a bottom-up approach.
Points:
• Uses support threshold
• Generates candidate sets
• Prunes infrequent items
• Easy to understand
• Computationally expensive
Real-world Examples:
• Retail shopping analysis
• Product bundling
• Cross-selling

2. FP-Growth
Uses a tree structure (FP-tree) to find frequent patterns efficiently.
Points:
• Faster than Apriori
• No candidate generation
• Uses compact data structure
• Efficient for large data
• Reduces computation
Real-world Examples:
• E-commerce recommendations
• Clickstream analysis

8
 • Inventory planning

3. Eclat Algorithm
Uses depth-first search and vertical data format to find frequent itemsets.
Points:
• Faster for dense datasets
• Uses intersection operations
• Memory efficient
• Works well with smaller datasets
• Depth-first approach
Real-world Examples:
• Telecom usage analysis
• Bioinformatics
• Market basket analysis

Basic learning algorithms overview

Applications:

1. Applications of Classification

1. Email Spam Detection

Email services use classification algorithms to categorize emails as spam or not spam based on
features like keywords and sender details. For example, if a model is trained on 10,000 emails, it may
classify a new email with 95% probability as spam if it contains suspicious words. This helps reduce
unwanted messages in inboxes.

2. Medical Diagnosis

Classification models help doctors identify diseases by analyzing patient data such as symptoms and
test results. For instance, a model trained on 5,000 patient records may classify a new patient as
having diabetes with 90% accuracy. This improves early detection and treatment.

3. Credit Risk Assessment

Banks use classification to decide whether a customer is a high-risk or low-risk borrower. For
example, based on income, credit score, and past history, a system may classify a customer with a
credit score of 750 as low risk. This helps banks reduce loan defaults.

4. Image Recognition

Classification algorithms identify objects in images by learning from labeled datasets. For example, a
model trained on 50,000 images can classify whether an image contains a cat or dog with 98%
accuracy. This is widely used in security and automation systems.

5. Customer Churn Prediction

Companies use classification to predict whether a customer will leave their service. For instance, out
of 1,000 customers, the model may predict 200 customers are likely to churn. Businesses can then
take action to retain them.

9
2. Applications of Regression

1. House Price Prediction

Regression models estimate house prices based on features like area, location, and number of
rooms. For example, a house of 1200 sq. ft. in a city area may be predicted to cost ₹50 lakhs. This
helps buyers and sellers make informed decisions.

2. Sales Forecasting

Businesses use regression to predict future sales based on past trends. For instance, if sales were ₹1
lakh, ₹1.2 lakh, and ₹1.5 lakh in previous months, the model may predict ₹1.8 lakh next month. This
helps in planning inventory and resources.

3. Weather Forecasting

Regression models predict weather conditions such as temperature and rainfall. For example, based
on historical data, the model may predict tomorrow’s temperature as 32°C. This helps in agriculture
and disaster management.

4. Stock Market Prediction

Regression is used to estimate stock prices based on past data. For example, a stock priced at ₹500
today may be predicted to reach ₹520 tomorrow. Investors use this information for decision-making.

5. Demand Prediction

Businesses predict product demand to manage supply efficiently. For instance, a store may predict
that 500 units of a product will be needed next week. This prevents overstocking or shortages.

3. Applications of Clustering

1. Customer Segmentation

Clustering groups customers based on behavior like spending habits. For example, a company may
divide 1,000 customers into 3 clusters: high spenders, medium spenders, and low spenders. This
helps in targeted marketing.

2. Market Research

Companies use clustering to identify different market segments. For instance, survey data from 2,000
people may be grouped into 5 clusters based on preferences. This helps businesses design better
products.

3. Image Segmentation

Clustering divides an image into meaningful parts. For example, an image with 1 million pixels can be
grouped into 4 clusters representing different objects. This is useful in medical imaging and computer
vision.

4. Document Clustering

Clustering groups similar documents based on content. For example, 10,000 articles may be grouped
into categories like sports, politics, and technology. This improves search and organization.

10
5. Social Network Analysis

Clustering identifies communities in social networks. For instance, among 5,000 users, clusters may
represent friend groups or shared interests. This helps in recommendation systems.

4. Applications of Association

1. Market Basket Analysis

Association finds items that are frequently bought together. For example, out of 1,000 transactions,
300 customers buy bread and butter together. Stores use this to place products nearby.

2. Product Recommendation Systems

E-commerce platforms suggest products based on associations. For instance, if 70% of users who
buy a phone also buy a case, the system recommends a case. This increases sales.

3. Cross-Selling Strategies

Businesses use association rules to suggest additional products. For example, if a customer buys a
laptop worth ₹50,000, the system may suggest accessories like a mouse or bag. This increases
revenue.

4. Inventory Management

Association helps businesses manage stock efficiently. For instance, if two products are often bought
together in 40% of transactions, both should be stocked together. This prevents shortages.

5. Customer Behavior Analysis

Association helps understand buying patterns of customers. For example, data from 5,000 purchases
may reveal that customers buying snacks often buy soft drinks too. This helps in marketing strategies.

MACHINE LEARNING

Supervised Learning

Classification

• Naive Bayes

• Decision Tree

• Random Forest

• K-Nearest Neighbors (KNN)

• Support Vector Machine (SVM)

• Logistic Regression

• Neural Networks

Regression

• Linear Regression

11
 • Polynomial Regression

• Ridge Regression

• Lasso Regression

• Decision Tree Regressor

• Random Forest Regressor

• Support Vector Regression (SVR)

Unsupervised Learning

Clustering

• K-Means

• Hierarchical Clustering

• DBSCAN

• Mean Shift

Dimensionality Reduction

• PCA (Principal Component Analysis)

• LDA (Linear Discriminant Analysis)

• t-SNE

• Autoencoders

Association Rule Learning

• Apriori Algorithm

• Eclat Algorithm

• FP-Growth

Semi-Supervised Learning

Classification (limited labels)

• Self-Training

• Label Propagation

• Label Spreading

• Semi-Supervised SVM

Regression (limited labels)

• Semi-Supervised Regression Models

12
 • Graph-Based Methods

Reinforcement Learning

Policy Learning

• Policy Gradient

• REINFORCE

Value Learning

• Q-Learning

• Deep Q Network (DQN)

Control / Decision Making

• SARSA

• Actor-Critic Methods

• Monte Carlo Methods

13