Unit –4

Introduction to Machine Learning

Machine Learning (ML) is a branch of Artificial Intelligence that enables a

computer system to learn from data and improve its performance without
being explicitly programmed. Instead of writing fixed rules, data is given
to the machine and it automatically finds useful patterns and relationships
from it. Using these patterns, the system can make predictions and
decisions. Machine Learning is widely used in real life, such as in email
spam filters, YouTube and Netflix recommendations, and face recognition
systems.

Machine Learning (ML) is a branch of Artificial Intelligence (AI).

It allows computers to learn from data and improve automatically
without being programmed again and again.

In traditional programming, we give rules + data to get output.

In machine learning, we give data + output, and the computer learns the
rules.

Example:
Email spam filter learns which mails are spam by studying many
examples.

Supervised Learning

Supervised Learning is a type of Machine Learning in which data and

correct answers are given to the machine.
This means that for every input, the machine already knows the right
output.

The machine learns by comparing its answer with the correct

answer and improves again and again.

Why it is called Supervised

It is called supervised because a teacher is present.

The teacher is the correct data that tells the machine what is right and
what is wrong.

How Supervised Learning Works

1. First, a dataset is given to the machine.

This dataset contains input data and correct output.

2. The machine studies this data and finds patterns.

3. A model is created based on this learning.

 4. When new input is given, the model predicts the correct output.

Naïve Bayes Algorithm

Naïve Bayes is a probability-based supervised learning algorithm.

It is based on Bayes Theorem, to decide what likely is going to happen

It is called Naïve because it thinks that all things (like words or features)
are separate and not connected.

Even though this is not always true, this method still works very well in
real life, especially for things like spam emails and messages.

Working of Naïve Bayes using email example

1. Show old emails

We show the computer many old emails.
Some are SPAM, some are NOT SPAM.

2. It sees which is more

It sees:
“How many are spam?”
“How many are not spam?”

3. It sees words
It notices:

 The word “free” comes many times in spam

 The word “meeting” comes many times in not spam

4. A new email comes

New email has:
“Free gift for you”

The computer looks at these words.

5. It decides
Because “free” is mostly in spam,
the computer says:
👉 This is SPAM

Decision Tree

A Decision Tree is a machine learning method used to make decisions.

It looks like a tree, where we start from the top and go down to get an
answer.

It is used in supervised learning, because data already has correct

answers.
Parts of a Decision Tree

A decision tree has three main parts:

1. Root Node

This is the starting point of the tree.

All data begins from here.

2. Branches

Branches are the conditions or questions.

They decide which path to follow.

Example:
Age > 18?
Yes → go right
No → go left

3. Leaf Nodes

These are the final answers.

No more questions are asked here.

Example:
Eligible
Not Eligible

How Decision Tree Works

1. First, all data is placed at the root node.

2. The tree asks a question about the data.

3. Based on the answer, data moves to a branch.

4. More questions are asked.

5. This continues until a final answer is reached.

6. The last answer is the leaf node.

So, it works like:

Question → Check → Go → Final Answer

Unsupervised learning

Unsupervised Learning is a type of Machine Learning in which only input

data is given to the machine and no correct output is provided.
The machine learns by finding patterns and groups in the data by
itself.
In this type of learning, the machine is not guided by a teacher.
It has to understand the data on its own.

The main goal of unsupervised learning is to discover hidden

information from the data.
It tries to find which data items are similar and which are different.

K-Means Algorithm

K-Means is a clustering algorithm used in Unsupervised Learning.

It is used to divide data into groups.

Here, K means the number of groups we want.

Example:
If K = 3, then the data will be divided into 3 groups.

Simple Example

Suppose we have data of customers.

Some buy expensive items, some buy cheap items.

If K = 2,
K-Means will make:

 One group of high-spending customers

 One group of low-spending customers

No one tells the machine this.

It finds the groups by itself.

K-Means Easy Steps (Memory Trick: “Choose – Pick – Put – Find –

Move – Stop”)

1. Choose
Choose the number of groups (K).

2. Pick
Pick K random points as group centers.

3. Put
Put each data item into the nearest group.

4. Find
Find the new center of each group.

5. Move
Move data again to the nearest center.
 6. Stop
Stop when groups do not change.

Clustering

Clustering is a technique used in Unsupervised Learning.

It means dividing data into different groups.

In clustering, similar data items are placed in the same group, and
different data items are placed in different groups.

The machine does not know the correct groups in advance.

It finds the groups by studying the data.

Simple Example

In a shop:

 Customers who buy expensive items form one group

 Customers who buy cheap items form another group

The machine creates these groups automatically.

Application Areas of Machine Learning

1. Healthcare – Disease Prediction

Machine learning helps doctors to find diseases early.

It studies patient data and predicts if a person is sick or not.

2. Banking – Fraud Detection

Banks use machine learning to detect fake transactions.

If something looks unusual, the system gives an alert.

3. E-commerce – Product Recommendation

Online shopping sites suggest products you may like.

This is done using machine learning by studying what you search and buy.

4. Email – Spam Filtering

Machine learning checks emails and puts spam emails in the spam
folder.
It looks at words and patterns in emails.

5. Self-Driving Cars

Self-driving cars use machine learning to understand roads, signals,

and traffic.
They learn when to stop, turn, or go.

6. Speech Recognition
Machine learning helps computers understand human voice.
Examples: Google Assistant, Siri.

7. Face Recognition

It is used to identify a person from their face.

Used in mobile phones and security systems.

8. Weather Forecasting

Machine learning studies past weather data to predict future weather.

Statistical Learning Methods

Statistical learning methods are a part of machine learning that use

mathematics and probability to learn from data.
These methods study the data and try to find a relationship between
input and output.

The main aim of statistical learning is to predict future values and help
in making decisions.

For example, it can predict marks from study hours or decide whether an
email is spam or not.

Examples of Statistical Learning Methods

1. Linear Regression

It is used to predict numbers like marks, salary, or sales.

2. Logistic Regression

It is used to classify data, such as yes/no, pass/fail, spam/not spam.

3. Bayesian Methods

These methods use probability to make decisions and predictions.

Learning with Complete Data

Learning with Complete Data means that all required information is

available to the machine.
The machine knows:

 The input data

 The correct output

 The class labels

So nothing is missing.

Simple Example
We have data like:

Mark Resul
s t

60 Pass

30 Fail

Here, the machine knows both marks and result.

So this is complete data.

How Machine Learns

Since everything is known, the machine can easily learn the

relationship between input and output.
It can then predict results for new students.

Learning with Hidden Data

Learning with Hidden Data means that some information is missing in

the data.
The machine does not have full knowledge.

In this case, we do not know:

 The true labels

 The exact values

So the machine has to guess and learn.

Simple Example

Suppose we have customer data, but:

 We do not know what product the customer likes

 Some records are missing

So the machine must predict the missing information.

Why Hidden Data is Difficult

Because the machine cannot see the full picture.

It has to estimate the missing parts before learning.

EM Algorithm (Expectation–Maximization)

EM Algorithm is used when the data is incomplete or hidden.

It helps the machine to find missing information and then learn from it.

It is mainly used in Learning with Hidden Data.

Why EM Algorithm is Needed

Sometimes in data:

 Some values are missing

 Some class labels are unknown

So the machine cannot learn properly.

EM algorithm helps to estimate the missing parts.

Two Steps of EM Algorithm

1. E-Step (Expectation)

In this step, the algorithm guesses the missing data using the current
information.
It calculates what the missing values might be.

2. M-Step (Maximization)

In this step, the algorithm updates the model to improve accuracy.

It uses the estimated data from the E-step to make the model better.

Uses of EM Algorithm

EM algorithm is used in:

 Clustering

 Image processing

 Speech recognition

EM Algorithm – Easy Steps

1. Guess
The machine makes a first guess.

2. Estimate (E-step)
It guesses the missing data.

3. Improve (M-step)
It improves the model using the guess.

4. Repeat
It does E-step and M-step again and again.

5. Stop
It stops when the result does not change.

Reinforcement Learning
Reinforcement Learning is a type of Machine Learning in which the
machine learns by doing actions and getting feedback.
It learns from its mistakes and successes.

The machine tries something, sees the result, and then tries to do better
next time.

Main Parts of Reinforcement Learning

1. Agent

The agent is the learner.

It is the machine or program that is trying to learn.

Example: A robot, a game player, a car.

2. Actions

Actions are the steps taken by the agent.

Example:
Move left, move right, stop, jump.

3. Reward

Reward is the feedback given to the agent.

 Good action → Positive reward

 Bad action → Negative reward

The agent wants to get more reward.

How Reinforcement Learning Works

1. The agent takes an action.

2. It gets a reward or punishment.

3. It remembers what happened.

4. It tries again.

5. Slowly, it learns which action is best.

So it learns by trial and error.

Example

A robot tries to walk.

It falls → bad reward
It stands → good reward

After many tries, it learns how to walk properly.

Uses

Reinforcement learning is used in:

 Games (chess, video games)

 Robotics

 Self-driving cars

Unit 5

Pattern recognition

Pattern recognition is the process of the identifying the pattern in the

data/ information . It means teaching a computer how to recognize things
from the information given to it. Computer or the machine studies many
examples and and learn to identify similar things in new data

Computer helps in:

Face recognition

Speech recognition

Handwriting recognition

For example, when a computer sees many human faces, it learns what a
face looks like. After that, it can recognize a new face. In the same way,
when it reads many handwritten words, it learns how letters are written
and can read new handwriting. This is called handwriting recognition.
When a computer listens to many voices, it learns how speech sounds and
can understand spoken words. This is called speech recognition.

Design Principles of Pattern Recognition & component

These are the basic steps used to recognize patterns.

1. Data Collection
First, the system collects data like images, sound, or text.

2. Pre-processing
The data is cleaned to remove noise and errors so it becomes clear.

3. Feature Extraction
The system finds the important parts of the data.

4. Classification
The data is put into a group or class.
5. Decision Making
Finally, the system gives the result.

Statistical Pattern Recognition

Statistical pattern recognition is a method where the computer uses math

and probability to recognize patterns.
It studies the data and checks how often things happen to make
decisions.

The computer looks at numbers and chances to decide what the pattern
is.

For example, it checks how many times a face looks similar to other faces,
or how a voice matches stored voices.

This method is used in:

 Face detection

 Voice recognition

 Medical diagnosis

Parameter Estimation Methods

Parameter estimation means finding the best values that describe the
data.
It helps the computer to understand and use data in a better way.

Two important parameter estimation methods are PCA and LDA.

(a) Principal Component Analysis (PCA)

PCA is a method used to reduce the size of data.

When data has too many details, PCA keeps only the most important
information and removes the useless parts.

It does not change the meaning of data; it just makes it simpler and
smaller.

Why PCA is used

 To remove unwanted data

 To make processing faster

 To store less data

 To make analysis easier

PCA – Very Simple Steps

1. Take big data

 2. See what is important

3. Keep important part

4. Remove rest

5. Get small data

(b) Linear Discriminant Analysis (LDA)

LDA is used to separate data into different groups.

It finds a line or boundary that best divides the data into classes.

It tries to make the distance between different groups as large as

possible so they do not mix.

Why LDA is used

 To clearly separate classes

 To improve classification

 To reduce mistakes

LDA – Very Simple

1. Take data

2. Find a line

3. Put data on line

4. Groups become separate

Classification Techniques

(a) Nearest Neighbor Rule

This method classifies data based on closest data.

If most nearby points belong to one class, the new data is also placed in
that class.

Example:
If a new student is like passed students, he is marked as pass.

Nearest Neighbor Rule – Easy Steps

1. Look at all old examples.

2. A new example comes.

3. Find the one that is most similar.

4. See what group it belongs to.

 5. Put the new one in the same group.

b. Bayes Classifier

Bayes Classifier is a method used to decide the class of data using chance
(probability).
It checks which class is more likely for the given data.

The class that has the highest chance is selected.

Simple Example

If an email has words that mostly appear in spam emails,

then the classifier says it is spam.

If the words are common in normal emails,

it says not spam.

Uses of Bayes Classifier

 Spam detection

 Disease prediction

Bayes Classifier – Super Simple steps

1. Look at old data

2. See common groups

3. Look at new data

4. Pick the best group

K-Means Clustering (same as unit 4)

K-Means is a clustering method.

It divides data into K groups based on similarity.

Used in:

 Customer grouping

 Market analysis

Support Vector Machine (SVM)

Support Vector Machine (SVM) is a supervised machine learning

algorithm used mainly for classification.
It helps the computer to separate data into different groups.
The main idea of SVM is to draw a best line (or boundary) between two
groups of data so that the groups are clearly separated.

Why SVM is Powerful

 It gives high accuracy

 It works well even with small data

 It creates a clear boundary between classes

Uses of SVM

 Face recognition

 Text classification

 Medical diagnosis

 Image classification

SVM – Simple Steps

1. Take data with two groups.

2. Find a line that can divide them.

3. Check which line separates best.

4. Choose the best line.

5. Use that line to classify new data.