INTRODUCTION TO

AI AND APPLICATIONS
1BAIA103

.
Outline
Module 3: Machine Learning
Reema Thareja, Artificial Intelligence: Beyond Classical AI,Pearson Education, 2023.

1. Machine Learning in AI 4. Clustering Techniques

Overview of Machine Learning Techniques Introduction to Clustering
Introduction to Machine Learning Models Types of Clustering Algorithms

2. Regression Analysis in Machine Learning 5. Neural Networks

Basics of Regression Basics of Neural Networks
Linear and Non-Linear Regression Techniques Types and Applications of Neural Networks

3. Classification Techniques
Overview of Classification Algorithms
Naïve Bayes Classification
Support Vector Machine (SVM)
Techniques in AI
Machine Learning Model

Machine Learning (ML) involves teaching a computer program to improve its performance at a
specific task through experience (data).

Tom M Mitchell:
“A computer program is said to learn from experience E with respect to some class of tasks T and
performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.”

This definition leads to understanding Machine Learning Models in three key components:
[Link] (T)
2. Experience (E)
3. Performance (P).
Example of the ML Model:

To build a model to predict house prices:

• Task (T): Predict the price of a house based on features like size, location, and number of
rooms.
• Experience (E): We provide the model with historical data on house sales (e.g., size, location,
price).
• Performance (P): The model’s performance is measured by how accurately it predicts the price
of new houses.

To build a model for A handwriting recognition:

• Task (T): recognizing and classifying handwritten words within images
• Training experience (E): a database of handwritten words with given classifications
• Performance measure (P): percent of words correctly classified
Types of Machine Learning Algorithms
 Labelled and Unlabelled Data
Labelled Data:
 Labelled and Unlabelled Data
Unlabelled Data:
Classification
Classification is a supervised learning method. The input attributes of the classification
algorithms are called independent variables. The target attribute is called label or variable.

Stages of Classification
• Training stage: The learning
algorithm takes a labelled
dataset and starts learning.
After the training set, samples
are processed and the model is
generated.
• Testing Stage: The constructed
model is tested with test or
unknown sample and assigned a
label. This is the classification
process.
Regression Algorithm
A Regression Algorithm is used in machine learning to predict a real value (a continuous
number) based on the data it has learned from.

How Regression Works:

[Link] Phase: During training, the
algorithm is provided with labeled data that
includes input variables (features) and their
corresponding real values (output).
Example: You have data on house features (size,
location, number of rooms) and their prices. The
model learns the relationship between these
features and the price.

[Link]: After learning, the model can

predict the real value (continuous output) for
new, unseen data.
Example: Given the size, location, and number
of rooms of a new house, the model can predict
its price.
Unsupervised Machine Learning
1. Clustering
Examples –
• Customer Segmentation: Unsupervised learning can help businesses group customers with
similar purchasing behaviors for targeted marketing campaigns.
• Image Segmentation: It can be used in image processing to automatically group pixels that
form certain objects or regions in an image.
• Anomaly Detection: It helps detect outliers or unusual data points in a dataset, which can be
useful for fraud detection or network security.
• Location-based Decisions: Cell Tower Placement
2. Association Analysis
It is used in data mining to find relationships or patterns between different items in large
datasets. The goal is to discover rules that describe these patterns, showing how items or
events are associated with each other.

How Association Analysis Works:

[Link] Relationships:
Association analysis tries to find patterns that describe how different items or actions are linked together.
Example: If customers frequently buy bread, they might also buy butter. The analysis finds that these two products
are often purchased together.
[Link] Association Rules:
The main goal is to create association rules in the form of:
"If X happens, then Y happens".
These rules help businesses understand which items tend to be bought together.
Example: "If a customer buys a laptop (X), then they are likely to buy a mouse (Y)."
This rule can be used to recommend related products to customers.
Applications of Association Analysis:
[Link] and E-commerce:
Businesses use association analysis to discover which products are often bought
together.
Example: In a supermarket, if many customers buy diapers and baby wipes together,
the store might place these items next to each other to increase sales.
[Link] Basket Analysis:
Commonly used in retail, this technique finds associations between
products purchased together. Example: "If a customer buys shampoo, they
are likely to buy conditioner."
[Link] Systems:
Used in online platforms (like Amazon or Netflix) to suggest products or movies based
on what others with similar preferences have liked.
Example: "Customers who bought this book also bought these other books."
3. Dimensionality reduction
• Dimensionality Reduction is a technique used to simplify a dataset by reducing the number of
features (variables) without losing important information.
• This is especially useful when you have datasets with a large number of features, making it
difficult to process and analyze the data effectively.
Image Source: [Link]
Why is Dimensionality Reduction Important?
[Link] Data:
When datasets have too many features (variables), it can be overwhelming and computationally
expensive to work with them.
Dimensionality reduction reduces the number of features, making the dataset more manageable.
[Link] Complexity:
Fewer features make models easier to train, faster to process, and often lead to better performance.
Reducing features also helps to avoid problems like overfitting, where a model is too complex and
doesn’t generalize well to new data.
[Link] Visualization:
With too many features, it's hard to visualize the data. Dimensionality reduction makes it easier to
visualize the dataset in 2D or 3D.
4. Outlier Detection
Outlier detection, also known as anomaly detection, is a technique used to find rare or unusual events in a
dataset that do not follow the normal pattern. These events or observations are called outliers and can
indicate important issues like fraud, errors, or new trends.

How Outlier Detection Works:

[Link] Anomalies:
Outlier detection identifies data points that differ significantly from the rest of the data.
Example: In credit card transactions, a sudden large withdrawal made in an unusual location could be
flagged as an anomaly (possible fraud).
[Link] (KNN):
K-Nearest Neighbors (KNN) is a common technique used to detect anomalies by checking the distance
between data points. If a data point is far from its neighbors, it's considered an outlier.
Example: In a KNN-based anomaly detection, if a customer transaction is much higher than the usual range,
it may be flagged as suspicious.
Applications of Outlier Detection:
[Link] Detection:
In banking or credit card systems, detecting unusual transactions
(like purchases in unexpected locations) is done using anomaly
detection.
[Link]:
Detecting rare diseases or unusual symptoms in patients’ data can be
done using anomaly detection.
[Link]:
In a production line, detecting defects or malfunctioning machinery
through unusual patterns in data can prevent larger issues.
Semi-Supervised Learning
Semi-Supervised Learning is a type of machine learning that combines the strengths of
supervised and unsupervised learning. It uses both labeled data (data with known
answers) and unlabeled data (data without known answers) to train the machine.
Reinforcement Learning (RL)
Reinforcement Learning (RL) is a type of machine learning where an agent learns
how to make decisions by interacting with an environment.

Unlike supervised learning, where the model is trained on labeled data, RL

involves an agent that learns by trial and error, with the goal of maximizing
rewards and minimizing penalties.
Example: Reinforcement Learning in a Game

A robot needs to reach a diamond (reward)

while avoiding fires (penalties) in a game.

The robot starts at a random point and tries different

paths.
Right path = Reward (diamond)
Wrong path = Penalty (fire).

The robot learns over time by repeating the process

and adjusting its behavior to maximize rewards and
avoid penalties.

Final Goal: The robot learns the best path to take in

the game and successfully reaches the diamond,
maximizing its total reward.
How Reinforcement Learning Works:
[Link]-Environment Interaction:
An agent interacts with its environment by taking actions (decisions) and receiving feedback in the
form of rewards or penalties.
Example: In a game scenario, the agent might be a robot and the environment is the game world.
[Link] and Penalties:
Rewards are given for performing correct actions, while penalties are given for incorrect
actions. The agent’s goal is to maximize its total reward over time by learning from
experience.
Example: If a robot picks the right path in the game (avoiding fire), it earns a reward (like a
diamond). If it chooses the wrong path (touches fire), it loses some reward.
[Link] and Error:
The agent starts with no knowledge and learns by trying different actions, observing the
results, and adjusting its strategy.
Over time, the agent learns which actions lead to the highest rewards and which actions lead to
penalties.
Regression Analysis in Machine Learning

Regression Analysis is a statistical method used to study the relationship between a dependent
(target) variable and one or more independent (predictor) variables. The primary goal of regression
analysis is to predict the value of the dependent variable based on the given predictors.

How Regression Analysis Works:

[Link] Relationships:
• Regression analysis aims to determine how changes in the independent variables affect the
dependent variable.
• Example: A company may want to find out how advertising expenditure affects sales.
[Link] Continuous Values:
• Unlike classification, which predicts discrete categories, regression predicts continuous values
like price, salary, or temperature.
• Example: Predicting the price of a house based on its size, location, and age.
Types of Regression:
[Link] Regression:
The simplest form of regression where the relationship between the dependent and independent
variables is assumed to be linear (a straight line).
Equation for Linear Regression:
Y=b0+b1X+e
b0 is the intercept, b1 is the coefficient of the predictor variable (X), and e is the error term.
Example: To predict a company's sales based only on its advertising expenses on television.
Sales = (Intercept) + (Coefficient) * TV Advertising
[Link] Regression:
When there are multiple independent variables, the equation extends to:
Y=b0+b1X1+b2X2+e
This is used when there are multiple predictors.
To predict a company's sales using a more comprehensive set of factors: TV ad spending, radio ad
spending, and newspaper ad spending
Sales = (Coefficient1 * TV Ad Spending) + (Coefficient2 * Radio Ad Spending) + (Coefficient3 *
Newspaper Ad Spending) + (Intercept)
Image Source: [Link]
Classification Techniques

• Classification is a type of machine learning task where the goal is to predict

which category or class an observation belongs to.
• The model is trained using labeled data, where each input is already tagged
with the correct class or category. Based on this, the model learns the
patterns and uses them to predict the class of new, unseen data
Key Classification Algorithms:
Decision Trees:
How It Works: Decision trees split the data into branches based on feature values, creating a
tree-like structure. Each leaf node represents a predicted class.
Example: A decision tree might classify an email as spam or not spam by asking questions like "Is
the sender known?" and "Does the subject contain certain keywords?"
Random Forest:
How It Works: A random forest is an ensemble of multiple decision trees. Each tree gives a
prediction, and the final class is determined by a majority vote.
Example: Random forests are often used for more accurate classification, such as predicting
whether a customer will buy a product based on their demographic information.
K-Nearest Neighbors (KNN):
How It Works: KNN classifies data points based on the majority class of their nearest neighbors in
the feature space.
Example: If a new email has features similar to several spam emails, KNN will classify it as spam.
Support Vector Machines (SVM):
How It Works: SVM finds the hyperplane that best separates the data into different classes.
It maximizes the margin between the classes to ensure the best separation.
Key Classification Algorithms:
Naive Bayes:
How It Works: Naive Bayes is based on Bayes' Theorem and assumes that features are independent.
It calculates the probability of each class and assigns the class with the highest probability.
Example: Naive Bayes is often used for text classification, such as classifying news articles into
topics like sports, politics, or technology.
Logistic Regression:
How It Works: Despite its name, logistic regression is a classification algorithm used to predict the
probability of a binary outcome (yes/no, 0/1).
Example: Predicting whether a customer will buy a product (1) or not buy (0) based on features
like age, income, and browsing history.
K-Nearest Neighbors (KNN)
• The K-Nearest Neighbors (KNN) algorithm is a supervised learning method that classifies
data points based on the similarity of nearby data points. It is used for classification and
regression tasks.
• The KNN algorithm works by finding the k nearest neighbors to a new data point and
making predictions based on the majority class or average value of those neighbors.
Image Source: [Link]
Distance Measure
Pros and Cons of KNN Algorithm:
Pros:
[Link] and Easy to Understand: KNN is a simple algorithm that doesn't require much training time.
[Link] Assumptions: It makes no assumptions about the underlying data distribution.
[Link] for Both Classification and Regression: It can be applied to both types of tasks.
[Link] for Multi-class Problems: It works well when there are more than two classes.
Cons:
[Link] Prediction Time: For large datasets, predicting for new data points can be slow since the algorithm needs to
compare it to all the training data.
[Link] to Data Scaling: If the data features are on different scales, the algorithm may be biased. Data needs to be
standardized.
[Link] Intensive: KNN stores all the training data, requiring large memory usage.
[Link] Sensitivity: KNN can be affected by noisy data and outliers.
[Link] Ideal for High-Dimensional Data: KNN may not perform well when the number of features (dimensions) increases
significantly.
Decision Trees
A Decision Tree is a supervised learning algorithm used for classification and
regression tasks. It works by creating a tree-like structure that splits data based on
different features to make predictions.
Advantages of Decision Trees:
[Link] to Understand and Implement: Visual and interpretable model.
[Link] Need for Data Scaling: It works with both numerical and categorical data.
[Link] Missing Data: Some decision tree algorithms can deal with missing data.
[Link]-Linear Relationships: Can model complex, non-linear relationships.

Disadvantages of Decision Trees:

[Link]: Decision trees can easily overfit the data, especially if the tree
is too deep. Solution: Pruning is used to trim the tree and prevent
overfitting.
2. Instability: A small change in the data can lead to a completely different tree
structure.
[Link] Toward Features with More Levels: If a feature has many possible values, it can dominate the
splitting process. Solution: Random Forests (ensemble method) can be used to handle this issue.

Real-World Applications of Decision Trees:

[Link] Diagnosis: Predicting whether a tumor is cancerous or benign.
[Link]: Predicting whether a loan will be approved or denied.
[Link]: Predicting if a customer will buy a product based on demographics.
Random Forests
Random Forests are an ensemble learning method used for both classification and regression
tasks. They combine multiple decision trees to improve the overall performance by reducing the
overfitting issue seen in individual decision trees.

Steps in Random Forest:

[Link] 1: Randomly sample K cases from the dataset for training each decision tree.
[Link] 2: Select m features from p available features at each node.
[Link] 3: Grow each tree as deep as possible, without pruning.
[Link] 4: For each new data point, each tree gives a prediction.
[Link] 5: Voting or Averaging to decide the final prediction.
Image Source: [Link]
Random Forests
How Random Forest Works:
[Link] Sampling:
From the dataset, random samples (with replacement) are taken to train multiple decision trees. This is similar to
bagging (Bootstrap Aggregating), but random forests introduce another layer of randomness.
[Link] Feature Selection:
For each decision tree, a subset of features (m predictors) is randomly selected at each node. The best split is found
from these m features. This prevents trees from being overly correlated with each other.
Difference from Bagging: In bagging, all features are considered at every split, but in random forests, only a random
subset is used.
[Link] Growth:
Each decision tree is grown to its full depth without pruning, ensuring a more complex model with more diversity in
trees.
[Link] Prediction:
For classification, each tree in the forest votes on the class, and the class with the most votes is chosen as the final
prediction.
For regression, the predictions of all trees are averaged to get the final prediction.
Naïve Bayes Classification
Naïve Bayes is a probabilistic machine learning algorithm based on Bayes' Theorem. It is
simple, fast, and widely used for classification tasks, such as spam detection, sentiment
analysis, and text classification.

Bayes' Theorem:
Bayes' Theorem helps calculate the probability of an event occurring given prior knowledge about related
events.

It is the foundation of Naïve Bayes classification.

P(A|B): The probability of A occurring given that B is true (posterior probability).

P(B|A): The probability of B occurring given that A is true (likelihood).
P(A): The probability of A occurring (prior probability).
P(B): The probability of B occurring (evidence).
Applications of Naïve Bayes:
[Link] Classification: Classifying whether a document is spam or not.
[Link] Analysis: Analyzing tweets or reviews to classify them as positive, negative, or neutral.
[Link] Systems: Predicting whether a user will like a product based on their past
behaviors.
[Link] Diagnosis: Classifying whether a patient has a certain disease based on medical features.
Deep Learning and Neural Networks
Deep Learning and Neural Networks
Deep Learning and Neural Networks
Introduction to Neural Networks:
Neural Networks (NN) are inspired by the human brain and consist of layers of interconnected
neurons.
Key Concept: Neurons process input data and learn to make predictions or classifications based on
the patterns they detect.
Training: The neural network adjusts its internal structure (called weights) to minimize errors
during training.
Example: Think of recognizing a dog in an image; the neural network learns to identify features
like shapes, edges, and colors by processing the image through layers of neurons.
Deep Learning and Neural Networks

Steps in Training a Neural Network:

[Link] Layer: Input data is fed to the first layer of
neurons.
[Link] in Hidden Layers: Each layer processes the
data using its weights and applies the activation
function.
[Link] Layer: The final result is output after all layers
have processed the data.
[Link]: Errors from the output layer are
passed backward through the network, adjusting the
weights for better future predictions.
Applications of Neural Networks and Deep Learning:
[Link] Recognition: Used in facial recognition, object detection, and handwriting
recognition.
[Link] Detection: Identifying fraudulent activities or rare events in data.
[Link]-Series Prediction: Predicting stock prices, weather forecasting.
[Link] Language Processing: Applications like sentiment analysis, machine
translation, and speech recognition.
[Link] Systems: Suggesting products or content based on user
preferences.
[Link] Diagnosis: Identifying diseases in medical images or predicting patient
conditions.
Support Vector Machine (SVM)

• A Support Vector Machine (SVM) is a popular supervised learning algorithm

primarily used for classification tasks.
• Its main goal is to classify data points into one of two categories based on features in
the dataset.
• An SVM is a classification algorithm that works by finding the best hyperplane that
separates two classes of data points in an n-dimensional space (where n is the number
of features).
• The hyperplane is the decision boundary that helps classify data points into their
respective classes.
Hyperplane:
• A hyperplane is simply a line (in 2D), a plane (in 3D), or a general decision boundary in
higher dimensions that separates the data into two classes.
• Example: In a 2D space, the hyperplane would be a line that divides the data into two
parts (one part for Class 1 and another for Class 2).
Support Vectors:
• Support vectors are the data points that are closest to the hyperplane. These points are
critical in determining the optimal hyperplane.
• The margin is the distance between the hyperplane and the closest support vectors.
The larger the margin, the better the classifier.
Margin:
• The margin is the distance between the hyperplane and the closest points from each
class. The SVM tries to maximize this margin, as a larger margin leads to a better
separation between the classes and a more accurate model.
Example: We have the following data points that we need to classify as Batsmen or Bowlers based on
their Runs and Wickets:
Batsmen:
(250, 10)
(300, 12)
(400, 8)
Bowlers:
(50, 30)
(60, 40)
(80, 20)
We want to classify a new player with the following data:
New player: (350, 20) (Runs = 350, Wickets = 20)
Applications of SVM:
[Link] Recognition: SVMs are used to recognize patterns and objects in images.
[Link] Classification: SVMs are used for spam email classification or document categorization.
[Link] Detection: SVMs are used to detect faces in images by classifying pixel data as face or non-
face.
[Link]: SVMs are used for classifying protein sequences and gene expression data.