MACHINE LEARNING

As per the New NEP Syllabus for BCA 6“*Semester Course of

Bengaluru City University and Bangalore University

Authored by

Srikanth S

Lalitha Y Divya S R
[Link], MCA [Link]., [Link]
Assistant Professor Assistant Professor
Department of Computer Science DeparU uent of Computer Science
Vijaya College, Jayanagar V ijaya College,R V Road
Bengaluru Bengaluru

Roopa H R Dr. Kadli Nanjundeshwara

MCA (PhD)
MCA(PhD)
Assistant Professor
Principal
Department of Computer Science, KLE
GS College o f Management, Yelahanka,
Society's S Nijalingappa College, Rajajinagar
Bengaluru
Bengaluru

Skyward Publishers
# 157, 7th Cross, 3rd Main Road, Chamarajpet,
Bengaluru-18. Phone ; 080-43706620 / 080-26603535
Mob: 9611185999
E-mail: [Link]@[Link]
Website: [Link]
A Text Book of “Machine Learning” - by Srikanth S, Lalitha Y, Divya S R, Mrs. Roopa H R, &
Dr. Kadli Nanjundeshwara as per the New NEP Syllabus for VI Semester BCA, Bengaluru City University &
Bangalore University.

© Authors

Copy Right;No part o f this book may be reproduced, stored in a retrieval system, or transmitted, in any
form or by any means, without the previous permission o f the copyright holders. Every effort has been
made to avoid errors or omissions in this publication. In spite of this, some errors might have crept in.
Any mistake, error or discrepancy noted may be brought to our notice which shall be taken care in the
next edition. The publisher shall not verify the originality, authenticity, ownership, non-infringement
o f the data, content, and information. The Authors are the sole owners o f the copyrights o f the Work It
shall be Authors sole responsibility to ensure the lawfulness o f the content and publisher is not responsi­
ble for any copyright issues. It is notified that publisher will not be responsiblefo r any damage or loss o f
action anyone, o f any kind, in any manner, there from all disputes are subject to Bengaluru jurisdiction
only.

Disclaimer: S l^ a r d Publishers has exercised due care and caution in collecting all the data before
publishing the book In spite o f this, if any omission, inaccuracy or printing error occurs with regards
to the data contained in this book, Styward Publishers will not be held responsible or liable. S i^ ard
Publishers will be grateful fo r your suggestions which will be o f great help for other readers.

First Edition: 2024

ISBN: 978-93-95085-78-6

P r i c e 275/ -

Published by:
Skyward Publishers
#157,7th Cross, 3rd Main Road, Chamarajpet
Bangalore-18. Phone: 080-26603535/43706620,
Mob: 9611185999
E-mail: [Link]@[Link]
Website: [Link]

DTP By
Nirmala & Mary, Skyward Team
PREFACE
Welcome to the world of Machine Learning! This book has been meticulously crafted to serve
as a comprehensive guide for students pursuing the 6 * Semester BCA course at Bengaluru City
University and Bangalore University, in alignm ent with the latest National Education Policy (NEP)
syllabus. Machine Learning has emerged a s a transformative field th at empowers computers
to learn from data and make intelligent decisions without being explicitly programmed.
Understanding the fundamentals of M achine Learning is essential in today's data-driven world,
and this book aims to equip you with the knowledge and skills necessary to excel in this domain.

Key Features of this Book:

1. Simple to Understand; Complex Machine Learning concepts are presented in a dear and accessible
manner, making it easy for students to grasp the fundamental principles.
2. Examples with Explanation: Each concept is accompanied by illustrative examples and detailed
explanations to enhance understanding and facilitate practical application.
3. Python Code for Practical Approach: Practical implementation is crucial in Machine Learning.
P3Tthon code snippets are provided throughout the book to demonstrate practical applications and
hands-on learning.
3. Real-Life Examples: Real-world scenarios and case studies are integrated into the content to
showcase the relevance and impact of Machine Learning in various industries and domains.
4. Lab Programs: Practical exercises and lab programs are included to provide students with hands-on
experience in implementing Machine Learning algorithms and techniques.
5. Model Question Papers: To aid in exam preparation, model question papers are included at the end
of the book to help students assess their understanding and readiness for examinations.

As you em bark on this educational jo u rn e y through the realms o f Machine Learning, we

encourage you to engage actively with th e content, experiment with the provided Python code,
and explore the practical applications o f th e concepts discussed. By delving into the world of
Machine Learning, you are not only expanding your knowledge but also preparing yourself for the
future of technology and innovation.
We welcome feedback, suggestions, and inquiries from readers and educators. Please feel free to
reach out to us at skyw [Link] bIish ers@ gm aiLcom with any com m ents or queries. Your input is
invaluable in our continuous efforts to en h an ce the learning experience for students.

Wishing you a fulfilling and enlightening exploration of Machine Learning!

- Authors

IV
SYLLABUS

UNIT - 1 : Fundamentals of Machine Learning [12 Hours]

Introduction to M achine Learning: W hat is Machine Learning? Why Use Machine Learning?
, Types of Machine Learning Systems, Main Challenges of Machine Learning, Applications of
Machine Learning. Why Python, scikit-learn. Essential Libraries and Tools.

UNIT - II: Data Preparation [12 Hours]

Working with Real Data, look at the Big Picture, Get the Data, Discover and Visualize the Data to Gain
Insights, Prepare the Data for Machine Learning Algorithms, Select and Train a Model.

UNIT - III: Supervised Learning [12 Hours]

Classification and Regression, Some Sample Datasets, k-Nearest Neighbours, Linear Models,
Naive Bayes Classifiers, Decision Trees.

UNIT-IV : Unsupervised Learning [12 Hours]

Clustering, K-Means, Limits of K-Means, using clustering for image segm entation. Using Clustering
for Preprocessing, Using Clustering for Semi-Supervised Learning, DBSCAN, Other Clustering
Algorithms.

Machine Learning Lab

1. Install and set up Python and essential libraries like NumPy and pandas.
2. Introduce scikit-learn as a machine learning library
3. Install and set up scikit-learn and o th er necessary tools.
4. Write a program to Load and explore the dataset of .CVS and excel files using pandas.
5. Write a program to Visualize the d ataset to gain insights using Matplotlib or Seaborn by
plotting scatter plots, bar charts.
6. Write a program to Handle missing data, encode categorical variables, and perform feature
scaling.
7. Write a program to implement a k-N earest Neighbours (k-NN) classifier using scikitlearn
and Train the classifier on the dataset and evaluate its performance.
8. Write a program to implement a linear regression model for regression tasks and Train the
model on a dataset with continuous targ et variables.
9. Write a program to implement a decision tree classifier using scikit-learn and visualize the
decision tree and understand its splits.
10. Write a program to Implement K-Means clustering and Visualize clusters. - ^<
 CONTENTS
tmiliimeiiliilsellB(MiclMiiW_________ ________________________________ 1.1-1.58 §
12
1.1 Introduction to Machine Learning

1.1.1 W hat is Learning?

1.1.2 Understanding Human Learning

1.1.3 Understanding Machine Learning

1.1.4 Goals of Machine Learning

1.2 History of Machine Learning 1.4

1.3 What is Machine Learning?

1.9
1.4 Features of Machine Learning
1.5 Traditional Programming Approach Vs Machine Learning Approach 1-10
1.12
1.6 Why Use Machine Learning?
1.12
1.7 Machine Learning Problem
1.8 Understanding the Operational Mechanisms of Machine Learning 1-13
1.15
1.9 How Machine Learning is being used?
1.16
1.10 Tjq)es o f Machine Learning
1.17
1.10.1 Supervised Learning
1.22
1.10.2 Unsupervised Learning

1.10.3 Semi-Supervised Machine Learning 1.26

1.10.4 Reinforcement Learning

1.11 Difference between Supervised and Unsupervised Learning 1-33

1.12 Applications of Machine Learning

1.13 Machine learning Life Cycle

1.39
1.14 Main Challenges of Machine Learning
1.41
1.15 Why Python?
1.42
1.16 Scikit-learn

1.16.1 Features of Scikit-learn

1.16.2 Installing Scikit-learn

1.17 Essential Libraries and Tools

vi
 1.17.1 Jupyter Notebook 1-45

1.17.2 NumPy 1-47

1.17.3 SciPy 1-49

1.17.4 Pandas 1-52

1.17.5 M atplotlib 1-54

1.18 Review Questions 1-56

Bala preparadon ___________ 2A-2^

2.1 Introduction 2.2

2.1.1 Meaning of Data in Machine Learning 2.2

2.1.2 Categories o f Data in Machine Learning 2.2

2.2 Data Preparation in Machine Learning 2.4

2.2.1 Im portance and Benefits of Data Preparation 2.5

2.2.2 Data Preparation Issues in Machine Learning 2.5

2.2.3 Steps in Data Preparation Process 2.5

2.3 Working with Real Data 2.9

2.4 Look at the Big Picture * 2.11

2.4.1 Structured Approach in Implementing Machine Learning 2.12

2.4.2 E xam p le: Implementing ML in Real E state for House Price Prediction 2.12

2.5 Get the Data 2.14

2.5.1 Load the Data and Explore the Data 2.16

2.5.2 Create a Test Set 2.19

2.6 Discover and Visualize the Data to Gain Insights 2.22

2.6.1 Why Visualizing the Data is Needed During Data Preparation? 2.22

2.6.2 Data Visualization Techniques 2.23

2.7 Prepare the Data for Machine Learning Algorithms 2.30

2.7.1 Data Cleaning 2.30

2.7.2 Data Transformation 235

2.7.3 Data Reduction 2.40

2.7.4 Feature Engineering 2.44

VII
 2.7.5 Data Spitting 2.46

2.8 Select and Train a Model 2.49

2.9 Review Questions 2.56

Supemsefl learning 3.1-3.76 %

3.1 Introduction 3.2

3.2 Types of Supervised M achine Learning 3.2

3.2.1 Classification 3.2

3.2.2 Regression 3.8

3.2.3 Difference betw een Regression and Classification 3.14

3.3 Some Sample Datasets 3.15

3 .4 K-Nearest Neighbors(K-NN) Algorithm 3.17

3.4.1 Characteristics o f K-Nearest Neighbors (K-NN) Algorithm: 3.18

3.4.2 How K-Nearest Neighbors (K-NN) Works ? 3.19

3.4.3 K-NN Algorithm 3.20

3.4.4 How to select the value of K in the K-NN Algorithm ? 3.20

3.4.5 How to Calculate Euclidean Distance ? 3.21

3.4.6 Applications of KNN Algorithm 3.26

3.4.7 Advantages and Disadvantages of KNN A lgorithm 3.27

3.5 Linear Models 3.27

3.5.1 Classification and Regression Tasks with L in ear Models 3.28

3.5.2 Characteristics o f Linear Models 3.29

3.5.3 Linear Regression 3.30

3.5.4 Logistic Regression 3.34

3.5.5 Applications of Linear Models 3.37

3.5.6 Advantages and Disadvantages of Linear M odels 3.38

3.6 Naive Bayes Classifiers 3.39

3.6.1 Bayes' Theorem 3.39

3.6.2 Naive Bayes Classifier 3.40

3.6.3 Types of Naive Bayes Classifiers 3.46

VIII
 3 .6 .4 Applications of Naive Bayes Classifiers 3.47

3.6.5 Advantages and Disadvantages of Naive Bayes Classifiers 3-48

3.7 Decision Trees

3.7.1 The Decision Tree Algorithm 3.49

3.7.2 Attribute Selection Measure (ASM) 3.49

3.7.3 1D3 Algorithm 3.59

3 .7 .4 C4.5 Algorithm 3.61

3.7.5 CART( Classification and Regression Tree) 3.67

3.7 .6 Applications of Decision Based Algorithms in ML 3.72

3.7.7 Advantages and Disadvantages of Decision T ree Based Algorithms 3.73

3.8 Review Questions 3.74

Dnsupervlsed Learning _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _________

4.1 Introduction

4.2 Types of Unsupervised Learning 4.3

4.3 Clustering

4 .3.1 Real World Examples o f Clustering *4.5

4.3.2 Importance of Clustering in Unsupervised Learning 4.6

4 .3.3 Applications of Clustering 4.7

4 .3 .4 Clustering Attributes 4.8

4.3.5 Types of Clustering Methods 4.9

4 .3 .6 Similarity and Distance Measures 4.13

4 .4 K-Means Clustering Algorithm 4.13

4.4.1 How K-Means Clustering Works? 4.14

4 .4.2 Applications of K-Means Clustering 4.18

4.4.3 Advantages or Benefits of K-Means Clustering Algorithm 4.18

4 .4 .4 Limits of K-Means Clustering Algorithm 4.19

4.5 Using Clustering for Image Segmentation 4.19

4.5.1 What is Image Segmentation ? 4.19

4.5.2 How Image Segm entation Works? 4.20

IX
 4.5.3 K-Means Clustering for Image Segm entation 4.20

4.6 Using Clustering for Preprocessing 4.22

4.7 Using Clustering for Semi-Supervised Learning: 4.27

4.8 DBSCAN 4.32

4.8.1 Importance of DBSCAN 4.32

4.8.2 How DBSCAN Works? 4.32

4.8.3 Applications o f DBSCAN 4.37

4.8.4 Advantages and Disadvantages of DBSCAN 4.38

4.9 Other Clustering Algorithms. 4.38

4.10 Review questions. 4.40

Lafi Programs _________________ U - L i i

Moflci gnesdon Papen B.l •B.4 j

Model Question Paper - 1 B-1

Model Question Paper - 2 B.2

Model Question Paper - 3 B.3

Model Question Paper - 4 B.4

 Xi

FUNDAMENTALS OF
':fSj'^ ?VV

: ‘It

r
Contents

-■ Introduction to Machine Learning

-- History of Machine Learning
•■ What is Machine Learning?
Features of Machine Learning
-■ Traditional Programming Approach Vs Machine Learning Approach
-- Why Use Machine Learning?
-■ Machine Learning Problem
Understanding the Operational Mechanisms o f Machine Learning
-- How Machine Learning is being used? ;
Types of Machine Learning
Difference betw een Supervised and Unsupervised Learning
Applications of M achine learning
-- Machine learning Life Cycle
-- Main Challenges o f Machine Learning
-■ Why Python?
Scikit-learn
Essential Libraries and Tools
•• Review Questions
 Machim Learning

■ ■ ■ in tr o d u c tio n t o M a c h i n e L e a rn in g
The rise of "big data" has changed how we use technology. With more personal com puters and
w ireless devices around, we now create and use lots of data. Every time we do som ething online, like
shopping or surfing the web, we create important data that can be used to customize products and
services based on what we like.
Think about a big superm arket th a t tracks a lot of sales data every day By looking at this data, the store
tries to guess what custom ers like to buy. Customers also w ant products that suit them . Customers
shopping habits aren't com pletely random; they change over time and place, but understanding
patterns can predict what custom ers might want.
Algorithms play a pivotal role in addressing computational challenges. While algorithm s are readily
available for tasks like sorting and seraching complexities arise when no algorithm exists for tasks
such as predicting custom er behavior or detecting spam emails. That's where m achine learning
com es in by teaching com puters o r machines to find patterns in data and make predictions without
specific instructions.
Machine Learning is like teaching a computer to learn and make decisions on its own by showing it
examples and patterns, ju st like how we learn from our experiences Machine learning likes to look at
big data to find useful inform ation or make predictions. Industries such as banking, manufacturing,
healthcare, and telecom use m achine learning to catch fraud, improve processes, diagnose illnesses,
and manage networks.
Machine Learning is primarily a concept and a field of study
within the broader domain of artificial in te llig e n c e . Machine
learning is about extracting knowledge from data. It is a research
field at the intersection of stadstics, artificial intelligence, and
computer science and is also known as p re d ictiv e a n a ljtic s
or statistical le arn in g . Machine Learning involves developing
algorithms and models that enable computers to learn from
data and make predictions or decisions without bein g explicitly
programmed for each task.

Machine learning methods are now widely used in various aspects of daily life. They are utilized in
suggesting movies, recom m ending food or products, identifying individuals in photos for security or
tagging purposes, optimizing search engine results for relevance, personalizing social media feeds,
predicting user behavior for targeted advertising, enhancing virtual assistants like Siri or Alexa,
improving healthcare diagnostics through image analysis, and automating fraud detection in financial
transactions on modern w ebsites and [Link] such as Facebook, Amazon, and Netfiix uses
multiple machine learning m odels across various sections o f their websites.

m m y What is Learning?
Learning is the process of acquiring knowledge or skills through study, experience, o r teaching. It
involves the ability to understand, retain, and apply new information or behaviors. It is a fundamental
human activity, essential for personal growth and development, adaptation to changes, and problem­
solving in daily life.
 Fundamentals o f INachine Learning ^ 1.3

The term "Intelligence" is defined b y the words knowledge, skill, and memory. Human begin learning
by memorizing. After some tim e, h e realizes that the sim ple ability to remember som ething is not
intelligence. Then he practices converting the data stored in his memory into knowledge and applies
it to develop problem-solving abilities in the real world.

Various Definitions o f Learning

Behavioral Definition: Learning is defined as a change in behavior due to experience or practice.

For example, a dog learning to sit on command after being consistently rewarded for the behavior.
Cognitive Definition: Learning is viewed as the acquisition of knowledge, understanding, or
mental skills. This definition focuses on tiie internal processes involved in gaining new information
and insights.
Constructivist Definition: Learning is seen as a process of constructing knowledge and meaning
through active engagement with experiences and reflection. This perspective emphasizes the role
of prior knowledge and social interactions in learning.
Social Definition: Learning is considered a social activity that occurs through interactions with
others, such as teachers, peers, or mentors. This definition emphasizes the importance of social
context and collaboration in the learning process. ______________

Examples Learning
Academic Learning: A student learns about the laws of physics in a classroom setting through lectures,
textbooks, and experiments.
Skill Acquisition: An individual learns to play the guitar through practice, observing others, and
perhaps taking music lessons.
Social Learning: A child learns social norms and behaviors by observing and imitating their parents
or peers.
Professional Development: An employee learns new software or a new system at work through
training sessions and by using the software in practical tasks.
Adaptive Learning: An individual learns to adapt to a new culture and language by living in a foreign
country, interacting with locals, and experiencing the culture firsthand. ______________________

U n d e r s ta n d in g H u m a n L e a r n in g
Human learning is something w e do naturally as we go through life. Human le a r n in g m eans
g ain in g new knowledge, s k ills , an d behaviors by e x p e rie n cin g things, w atching o th e rs , and
b e in g taught. Unlike machines, humans can think, reason, and adjust what they know to different
situations. Our brains are am azing organs that help us take in information through our senses, make
links between things, and rem em ber them. Human learning isn't just about one area bu t includes
many mental skills like solving problems, thinking creatively, and understanding em otions.

U n d e r s ta n d in g M a c h in e L e a r n in g
Machine learning is the branch o f Artificial Intelligence th at focuses on developing models and
algorithms that let com puters learn from data and improve from previous experience without
being explicitly programmed fo r every task. In simple words, ML teaches the system s to think and
understand like humans by learning fi'om the data.
Unlike traditional programming, w here instructions are explicitly given to the computer, machine
learning algorithms have the ability to learn and improve from pattern s in data. These algorithm s are
trained using labelled datasets, w here the desired output is know n, and then applied to new, unseen
data to make predictions or decisions.
One o f the key strengths of m achine learning lies in its ability to process and analyse vast amounts
of data quickly and accurately This has led to ground breaking advancements in various fields, such
as image and speech recognition, natural language processing, and autonomous vehicles. Machine
learning algorithms can detect com plex patterns that may n o t be obvious to humans, leading to
precise predictions and improved decision-making processes.

I l l j l l l l G o a ls o f M a c h in e L e a r n i i ^

The m ain goals of Machine Learning a r e :

• Enhance computer intelligence and capabilities through advanced algorithms.
• Utilize developed models to categorize and organize data effectively
• Predict future outcomes by leveraging established m odels and patterns.
• Conduct computer sim ulations and model human learning processes for enhanced
understanding.
• Optimize decision-making processes through data-driven insights.
• Improve efficiency and accuracy in various tasks through automated learning.
• Enhance problem-solving abilities by leveraging m achine learning algorithms.
• Enable machines to adapt and evolve based on new inform ation and experiences.
• Drive innovation and advancem ents in various fields through intelligent systems.
• Facilitate personalized experiences and recommendations based on user behavior and
preferences.

■ K a H is to ry o f M a c h in e L e a r n i n g

Before som e years (about 4 0 -5 0 years), machine learning was scien ce fiction, but today it is the part of
our daily life. Machine learning is making our day to day life easy from self-driving c a rs to Amazon
v irtu a l a ssista n t "Alexa". However, the idea behind machine learning is so old and has a long history.
Machine learning has a rich history that dates back to the m id -20th century Let us discuss the key
m ilestones and developments in the history of machine learning:
1. 1 9 5 0 s - 19 6 0 s: The foundation of machine learning was laid during this period with the work
o f pioneers like Alan Turing, who proposed the concept o f a "learning machine" in his paper
"Computing Machinery and Intelligence" in 1950.
• Arthur Samuel, a pioneer in the field of machine learning, created a program in 1952
that enabled an IBM com puter to play checkers at a high level of performance.
• A rthur Sam uel coined the term "Machine L e a rn in g " in 1959.
2 . 1 9 7 0 s - 1 9 8 0 s: This era saw the emergence o f sym bolic A1 and expert system s, where
knowledge was represented explicitly in the form o f rules. However, limitations in handling
uncertainty and complexity led to a shift towards m ore data-driven approaches.
 f undamenfals of Machine Learning

3 . 1 9 9 0 s : The 1990s marked the resurgence of interest in neural networks with the development
o f backpropagation algorithms for training deep neural networks. Support vector m achines
(SVMs) also gained popularity as powerful tools for classification tasks.
4 . 2 0 0 0 s : Grouping methods like random forests and gradient boosting became popular for
com bining multiple models to improve predictions. M achine learning started being used in
various real-world applications.T his period also saw the increasing use of data mining and
m achine learning in various applications, including recommendation systems and natural
language processing.
5. 2 0 1 0 s - P resent: The past decade has been characterized by the widespread adoption of deep
learning due to advancement o f computational power and the availability of large datasets.
Deep learning models, particularly convolutional neural networks (CNNs) and recurrent
neural networks (RNNs), have achieved remarkable success in tasks such as image recognition,
sp eech recognition, and language translation.
6. F u tu re Trends: The future o f machine learning is likely to be shaped by developments in areas
such as reinforcement learning, generative adversarial netw orks (GANs), and explainable AI.
T h ere will also be a focus on ethics, fairness, and making Al systems easier to understand and
tru st.

W h a t is M ach in e L e a r n i n g ?

Machine learning is a subfield of artificial intelligence, which is broadly defined as the capability o f a
m achine to imitate intelligent human behaviour. Artificial intelligence systems are used to perform
com plex tasks in a way that is sim ilar to how humans solve problem s.
In the real world, we are surrounded by humans who can leam everything from their experiences
with th e ir learning capability, and we have computers or m achines which work on our instructions.
But can a m achine also learn fi-om experiences or past data like a human does? So here com es the
role o f M ach in e Learning.

lcanlearneverytWngV
Human
C automatkafly from j
eqperiences.
Canuleam?
j Machine
M achine learning is programming computers to optimize a perform ance criterion using exam ple data
or past experience. We have a model defined up to some param eters, and learning is the execution
of a computer program to optim ize the parameters of the m odel using the training d ata or past
experience. The model may be predictive to make predictions in the future, or descriptive to gain
knowledge from data, or both.

What is Machine Learning ?

Machine learning (ML) is a branch of artificial intelligence (AI) that enables machines to
automatically learn from data and previous experiences in order to identify patterns and make
predictions with minimal human intervention.
The goal of machine learning is to develop systems that can automatically learn and adapt to new
information and tasks, ultimately improving their performance over time._____________

Machine learning algorithms m ake a model using past data to help predict or decide things without
direct instructions. By using historical data, these algorithms com bine statistics and com puter science
to create predictive models. The more data we give them, the better they perform. If a m achine gets
more data, it can learn and improve its predictions.

Various Definitions o f Machine Learning

L Arthur Samuel defined machine learning as the field of study that gives computers the ability to
learn without being explicitly programmed.
Ethem Alpaydin defined machine learning as programming computers to optimize a performance
criterion using example data or past experience.
Kevin Murphy defined machine learning as the science of getting computers to act without being
explicitly programmed.
Pedro Domingos defined machine learning as the process of learning to make accurate predictions
based on past observations.
Tom Mitchell defined Machine Learning as the process in which a computer program learns from
experience E in tasks T, improving its performance measure P as a result of that experience_____

□ Examples Popular Examples of Machine Learning

cr-Q
1. Virtual Personal Assistants: Virtual personal assistants like Siri, Alexa, and Google Assistant use
machine learning to understand and respond to voice commands. They learn from interactions to
provide more accurate and personalized responses over time.
2. Movies or Music Recommendation Systems: Platforms like Netflix, Amazon, and Spotify use machine
learning to recommend movies and music based on past preferences and behavior. These systems
learn ft-om choices to suggest content that is likely to be enjoyed.
3. Image Recognition: Machine learning is used in image recognition applications like Facebooks
automatic tagging feature. It can identify and tag people in photos by learning from previous tags and
facial features. __ ____________________________
 J=undamenfais of ^ ch in e Learning ^ \y

4. Self-Driving Cars: Self-driving cars use machine learning algorithms to analyze data from sensors and
cameras to navigate roads, detect obstacles, and make driving decisions. The system learns from real-
world driving experiences to improve safety and efficiency.
5. Fraud Detection: Banks and credit card companies use machine learning to detect fraudulent
transactions by analyzing patterns in spending behavior. The system learns to identify unusual
activities and flag them for further investigation.
6. Personalized Product Recommendations: E-commerce platforms like Amazon, flipkart and eBay
utilize machine learning to suggest products based on browsing history and purchase behavior. These
systems learn from user interactions to recommend items that align with user preferences,
7. Dsmamic Pricing Strategies: Online marketplaces like eBay, amazon and Airbnb use machine learning
to adjust prices in real-time based on factors such as demand, competition, and customer behavior. By
analyzing vast amounts of data, these systems can set optimal prices to maximize revenue and attract
customers, leading to a more competitive pricing strategy in the e-commerce industry.

Case S tu d y U nderstanding M ach in e Learning in th e C o iite x t o f E-Com merce (E xam p le:

1.1
- . V ' %Amazon, F lip k art, e b a y etc.,)

In the competitive landscape of e-commerce, personalized recommendations play a crucial role in enhancing
user experience and driving customer engagement. Companies like Amazon leverage machine learning
algorithms to analyze user behavior, understand preferences, and deliver tailored product suggestions. Let's
explore how Amazon utilizes machine learning in the context of a user browsing Dell laptops on their platform.
Step 1: U ser Browsing Dell Laptop on Amazon
A user visits the Amazon website and explores a variety of Dell laptops, comparing models,
specifications, and prices.
Step 2: Tracking User Behavior
Amazon's system tracks the user's browsing activity, capturing details of the Dell laptop models
viewed, time spent on each product page, and interactions like adding items to the cart or wish list.
Step 3: Data Collection and Analysis
Machine learning algorithms analyze the collected data, processing the user's interactions with Dell
laptops to identify preferences and patterns.
Step 4: Building User Profile Based on the analyzed data
Amazon creates a user profile that includes preferences for Dell laptops, budget constraints, desired
features (e.g., screen size, processor speed), and relevant past purchase history.
Step 5: Recommendation Generation
Utilizing the user profile and machine learning models, Amazon's recommendation system generates
personalized suggestions, recommending other Dell laptops or related accessories that align with the
user's preferences.
Step 6: Dlsplajing Recommendations
Upon the user's return to the Amazon platform, personalized recommendations are showcased on the
______ homepage or product pages, displaying relevant Dell laptops based on the user's previous interactions.
 Machine Learning
G X Machii

Step 7: User Engagement and Feedback Loop

As the user continues to engage with the recommended products, the system collects feedback on user
interactions, such as clicks, views, and purchases. This feedback loop refines the machine learning
models for more accurate and tailored recommendations.
Step 8: Continuous Learning and Improvement
Amazon's machine learning algorithms evolve with user behavior, adapting to changing preferences and
trends. By analyzing vast amounts of data and user interactions, the system enhances recommendation
accuracy over time, elevating the shopping experience for customers.
In this case study, we see how Amazon leverages machine learning to understand user preferences, analyze
browsing behavior, and provide personalized recommendations for Dell laptops. By utilizing data-dnven
insights and predictive algorisms, Amazon enhances user engagement and satisfaction, showcasing the
power of machine learning in the e-commerce industry.

U nderstanding M ach in e Learning in th e C o n te x t o f Online S tream in g

P latform s (E xam p le: Netflix, A m azon P rim e Vidoe etc,.)

The online streaming platforms like Netflix or Amazon Prime Vidoe, the utilization of machine learning
algorithms has revolutionized the way users discover and engage with content By analyzing user behavior and
preferences, platforms can offer personalized recommendations, enhancing the overall viewing experience.
Let's delve into a case study that illustrates how Netflix leverages machine learning to understand user
preferences and optimize content recommendations.
Step 1: User Watching Movies on Netflix
A user accesses their Netflbc account and begins watching movies across various genres, including
action, comedy, and drama.
Step 2: Tracking User Viewing Behavior
Netflix's system tracks the user's viewing patterns, capturing details such as the genres of movies
watched, viewing session durations, and interactions like adding movies to the watchlist or providing
ratings.
Step 3: Data Collection and Analysis
Machine learning algorithms analyze the collected data on user viewing habits to discern preferences
and viewing trends.
Step 4: Building User Profile Based on the analyzed data
Netflix constructs a user profile that encompasses preferred genres, favorite actors or directors,
viewing habits (e.g., binge-watching), and movie ratings provided by the user.
Step 5: Recommendation Generation
Leveraging the user profile and machine learning models, Netflix's recommendation system generates
personalized suggestions, recommending movies or TV shows in similar genres or featuring preferred
actors to align with the user's tastes.
Step 6: Displaying Recommendations
When the user explores the Netflix library, personalized recommendations are showcased on the
homepage or category pages, presenting relevant content based on the user's viewing history and
preferences.________ _________ _____ ___________ ______________ _______ _______________ _____
 ;T5i ' , ' , , , --'‘.y i^ ^ fF u n d a m e n ta ls o f Maehin^Uarning

Step 7: User Engagement and Feedback Loop

Continuing to watch content and interact with recommended titles, the system collects feedback on
user interactions, such as which recommendations were watched, liked, or skipped. This feedback loop
refines the machine learning models for more precise and taUored recommendations.
Step 8: Continuous Learning and Improvement
Netflix's machine learning algorithms evolve with user behavior, adapting to changing preferences and
interests. By analyzing extensive data and user interactions, the system enhances recommendation
accuracy over time, delivering a personalized and immersive viewing experience for subscribers.
This case study explains how Netflbc or Amazon Prime Video utilizes machine learning to analyze user behavior,
create personalized recommendations, and highlight content discovery. Through data-driven insights and
predictive algorithms, Netflix enhances user satisfaction and engagement

F e a t u r e s o f M a c h in e L e a r n i n g

Machine learning (ML) is a subset of artificial intelligence that focuses on building systems that
learti from data, identify patterns, and make decisions with minimal hum an intervention. Some key
features of m achine learning along with exam ples are listed below:
1. A d a p ta b ility : Machine learning m odels can adapt to new data and changing environments,
making th e m versatile for various applications.
E x a m p le : In autonomous vehicles, m achine learning algorithms continuously learn from real­
time sen so r d ata to adapt to different driving conditions and improve decision-making.
2. A u to m a tio n : Machine learning enables automation of tasks by allow ing systems to learn from
data and m ak e decisions without explicit programming instructions.
Exam ple : In email spam detection, machine learning algorithm s can automatically classify
incoming em ails as spam or non-spam based on patterns in the content.
3. S c a la b ility : Machine learning algorithm s are designed to handle large volumes of data and
scale w ith it. As more data becom es available, machine learning models can update their
predictions and decisions, often improving in accuracy.
Exam ple : In social media platforms, machine learning algorithm s analyze vast amounts of
user-gen erated content to personalize feeds and recommendations for millions of users.
4. P e rso n a liz a tio n : Machine learning enables personalized experiences by tailoring
recom m endations and content based on individual preferences.
Exam ple: In streaming services like Spotify, machine learning algorithm s analyze user listening
habits to c re a te personalized playlists and recommendations.
5. P re d ictiv e A n a l)^ c s : Machine learning excels in making predictions and forecasts based on
historical d ata.
Exam p le : Financial institutions use m achine learning to predict stock market trends, assess
loan risks, an d detect fraudulent transactions based on historical data. In weather forecasting,
machine learn in g models analyze past weather patterns and cu rren t atmospheric conditions
to pred ict fu tu re weather outcomes with improved accuracy.
6. C o n tin u o u s Im p ro v em en t: As m achine learning algorithms are exposed to new data, they
are able to independently improve th eir performance.
 ......
/■;*^’■V ' ^ ' ? - V % ' ‘ ‘ . . : ;I'3k-^V

B ia m p le ; The voice assistan t technologies like Siri and Alexa, which becom e m o re accurate in
understanding and processing user requests over time.
7. Decision M a k in g : ML can assist in making decisions with minimal human intervention.
Example : In h ealthcare, machine learning models help in diagnosing diseases and
recommending tre atm e n t plans based on patient data and trends found in h isto rical data.
8. Pattern R e c o g n itio n : ML excels at recognizing patterns and regularities in d ata.
Example : Facial recognition technology uses machine learning to id en tify and venfy
individuals based on digital images of their faces.
9. Efficiency: Machine learning can enhance efficiency by automating rep etitiv e tasks and
optimizing processes.
Example : In manufacturing, machine learning algorithms analyze production data to predict
equipment failures and schedule maintenance proactively, reducing dow ntim e and improving
productivity. _______________________

m y T ra d itio n a l P r o g r a m m i n g A p p ro a c h V s M a c h in e L e a r n in g A p p r o a c h

In software development, th ere are two main approaches to solving tasks re la te d to pattern
recognition or decision-m aking: the traditional programming approach and the m achine learning
(ML) approach.
The traditional program m ing is suitable for tasks w ith clear rules and stru ctu res and machine
learning (ML) approach o ffers a more dynamic and adaptive approach for tasks mvolving pattern
recognition or decision-m aking in complex and evolving environments.
1. Traditional P ro g ra m m in g Approach:
. M ethod : In th e traditional programming approach, developers m anually write explicit
instructions (algorithm s) to solve a specific task or problem.
. P rocess : D evelopers analyze the problem, define rules and conditions, and create a set
of instructions (cod e) that the computer follows to produce the d esired output.
. P attern R e c o g n itio n : In tasks involving pattern recognition o r decision-making,
developers n eed to anticipate all possible scenarios and explicitly program rules to
handle each case.
. L im itatio ns : T h is approach is effective for tasks with clear and w ell-defined rules but
can be challenging for complex problems w ith ambiguous patterns o r evolving data.

2. M achine L ea rn in g A pproach:
• M ethod : In contrast, the machine learning approach involves train in g algorithms on
data to learn patterns and make decisions without explicit programm ing.
. P ro cess : Instead of manually coding rules, machine learning algorithm s analyze large
datasets, identify patterns, and adjust th eir models based on feed b ack to improve
accuracy.
. P attern R e c o g n itio n : Machine learning algorithms excel at recognizing complex
patterns and adapting to new information without the need for pred efin ed rules.
 F u n d a m e n ia is^ M o ch in e L e a rn in g A | .n

Advantages : This approach is highly effective for tasks where patterns are difficult to
define explicitiy or v^^hen the data is constantly changing or evolving.

Flexibility : Traditional programming requires developers to anticipate all scen ario s and
write explicit rules, while m achine learning adapts to new patterns and data automatically.
Scalability : Machine learning can handle large and complex datasets more efficiently than
traditional programming, making it suitable for a wide range of applications.
A daptability : Machine learning models can evolve and improve over time as they learn
from more data, whereas traditional programs may require manual updates to accomm odate
changes.

Example Spam Email F ilte r Example

Spam emails are unwanted messages that flood our email inboxes with advertisements, scams, or malicious
content. A spam filter acts as a shield, blocking these unwanted emails from reaching us. Without a filter,
our inboxes would be overwhelmed with junk, making it harder to find important emails and exposing us to
potential security threats. The filter helps keep our email safe, organized, and free from unwanted distractions.
1. Traditional Approach:
• Method: Manually defining rules based on common spam patterns to identify and block
unwanted emails.
- For example, identify common patterns in spam emails, such as specific words or phrases
like “4U," "credit card," "free," "exciting" and "amq^ing," in elements like the subject line,
sender's name, and email body.
- Develop detection algorithms for each identified pattern to flag emails as spam if multiple
patterns are detected.
- Test and refine the program iteratively until it reaches a satisfactory level for deployment
If spammers realize their emails with "4U" are blocked, they may switch to using "For U" to evade
detection. Continuous updates to the spam filter are necessary as spanuners adapt their tactics,
leading to an ongoing cycle of rule creation.
• Drawbacks:
- Time-consuming and ineffective to maintain and update rules against evolving spam
tactics.
- Lacks scalability and struggles with the complexity and variability of spam content.
2. Machine Learning Approach:
• Method: Training algorithms on large datasets of spam and non-spam emails to automatically
learn patterns and features for filtering. For example, a spam filter based on Machine Learning
techniques automatically notices that "For U" has become unusually frequent in spam flagged by
users, and it starts flagging them without our intervention.
• Advantages:
- Dynamic and adaptive solution that evolves with new spam tactics.
- Improved accuracy and reduced false positives.
- Efficient handling of complex and evolving spam content ______
 W h y U s e M a c h in e L e a rn in g ?

Why Use Machine Learning ?

reducing manual effort and enhancing efficiency.

2. AdapUbillty : Machma learning models can adjust» new Wonnation and [Link] condmons,
ensuring effectiveness in dynamic environments.
3. scalabllln- = Machine leading techniques can handle large a-d complex datasets and
suitable for various applications.
4. Enhanced Decision-Making: Machine learning enables infomed decisions and pred«ons by
Ipvpraffine data-driven insights,
5. l„„[Link]„andE«iclency:Machlneleamlngdrlvesinno»[Link]

that optimize processes and improve user experiences.

6. Streamlined and Accu^te [Link] : ML algorithms pn,vide concise and accurate results.
particularly in tasks like spam detection. rlpar
7. complex Problem Resolution: Machine learning «<cels in solving complex tasks «nd,ou

algorithmic solutions such as natural language processing.

8. Progressive Learning Capability: ML systems condnuousl, evolve and improve accuracy as [Link]
learn from more data.
9. InsightfulAnalysU: Machineleaminguncovershlddenpattemsmvastdatasetsbyundersta

of complex phenomena through data mining.

10. Decision Process A«[Link] : ML algorithms revolutionize decision-making by automat, g
processes based on data-driven insights.
11 Flexibility: Traditional programming requires developers to anticipate all scenanos and wn
eStr^ es.„hllem achinelearnlngadaptsto„.w [Link]>,nancaU y.

12. Adaptation to Fluctuadng DaU. Environments : ML's Beribility makes it ideal for [Link]
requiring real-time adaptability. ^
13. Discovery and Uaming : ML algorithms reveal new correlations and trends, contnbufng
human learning and knowledge expansion^

KM Machine Learning Problem

A m achinelearnlngproblem refers t o a s p e d f l c t a s k o ^ i ^ t o c a ^ b e a d ^ ^ ^ ^ ^
a model

programmed.
 Note
A computer program which learns from experience is called a machine learning program or simply a learning
program. Sudi a program is sometimes also referred to as a learner._________ ______________ ___________
Machine learning problem s typically involve:
1. Task o r O b jectiv e ( T ) : Defining what needs to be achieved, such as classification, regression,
clustering, or p attern recognition.
2. Perform ance M e tric (P ): Determining how the performance of the m achine learning model
will be evaluated su ch as accuracy, precision, recall, or F I score.
3. Training E x p e rie n c e fE): Providing the model with a dataset of input features and
corresponding lab els to leam from during the training process.
By formulating a m achine learning problem with a clear task, performance m etric, and training
experience, developers and data scientists can design and implement effective machine learning
solutions to address a w ide range of real-world challenges and applications.

ExaWie 1 I Handwriting Recognition Learning Problem

Handwriting recognition learning problem

• Task (T): Recognising and classifying handwritten words within images
. Perfonnance M etric (P): P is measured by the percentage of words correctly classified.
. Training Experience (E): Training experience E consists of a dataset of handwritten words with their
corresponding classifications._________ _________ _______________________________ _________

Exam ^^ ■jA Robot Driving Learning Problem

Task(T): Driving on highways using vision sensors
Performance M etric (P): Average distance travelled before an error
Training Experience (E) : A sequence of images and steering commands recorded while observing a
human d r i v e r . _________________________________________________

Example 3 A Chess Learning Problem

• Task (T): Playing chess

. Perfonnance M etric (P): Percent of games won against opponents.
. Training Experience (E): Playing practice games against itself ________________________ ____
The function of a m achine learning system can be d escrip tiv e, meaning that the system uses the data
to explain what happened; predictive, meaning the system uses the data to pred ict w hat will happen;
or prescriptive, meaning th e system will use the data to make suggestions about w hat action to take.

iTTnderstanding the Operational Mechanisms of Machine Learning

Machine learning (ML) system s are designed to create predictive models by learning from past data.
These models are trained to make predictions or decisions without being explicitly programmed for
each possible scenario.
, 14 ^ M aeh in c Learning ■■■S&S’.V

1. P red iction Models: ML uses statlstica: techniques to build models that can predict future
outcom es based on historical d ata.
2 . L earn in g from Data: ML system s learn ftom pr«rtous d atasets The more data they have,
b e tte r they can make predictions by recognizing patterns m the data.

actual results. ,
4 Feedback and Iteration: M achine learning is a continuous process. The
are compared to real outcom es, and any differences are used to improve the model. T h .s may
involve retraining the model w ith new data or adjusting its settings.
• f^n^[Link] diapram :

— -- ------------- r '
r
Building w Output
Training Machine Learning ---- ^ Logical Models
W
Input Past Data- Algorithm
^ J
!
New Data
Learn from Data

1 in p u t Past DaU: Historical d ata is fed into the ML algorithm for training purposes.
E xam p le : The system is fed w ith historical data on custom er purchases, mcludmg items
bought, purchase frequency, an d spending patterns.
2 Training: The algorithm u n d erg oes a tiaining phase w here it learns fro m the data prodded.
E xam p le • During the train in g phase, the algorithm learns from the past purchase data to
identify trends such as popu lar products, seasonal buying patterns, and customer p reference .
3 M achine Learning A lgorith m : This refers to the core s e t of rules and statistical processes
“ let^ealgorithm to learn from d,[Link] tbatproces

information and makes d ecisions.

E x a m p le : This includes th e statistical processes and rules that enable fl.e ^ s p to analyze
th ecu Lm erp urch asedataan dm akep red lctionsab ou tfu turebu y,ngbeh av,or.

4 Bnildlns Logical M odels: A fter th e training phase, the system constructs logical models based
r !h ™ d patterns and relationships in the data. These models represent the insights
gained from the training data. , , . .
E xam p le : Based on the train in g data, the system creates logical models that ^how how
different factors like prod uct preferences and buying frequency influence custom er behavior.
5 O utput When new data is introduced to the system, the logical models created durm g training
° e t e d r p r e d i c t outcom es. The algorithm applies the learned patterns to the new data to

retail store make personalized recommendations.

 • Fundamenials of Machine Learning

H o w M a c h in e L e a rn in g is b e i n g u se d ?
Machine learning is a modern innovation th at has enhanced many industrial and professional
processes as well as our daily lives. It’s a su b set o f artificial intelligence (Al), which focuses on using
statistical techniques to build intelligent com puter systems to learn from available databases.
With m achine learning, computer system s can take all the custom er data and utilise it. It operates
on what’s been programmed while also adjusting to new conditions or changes. Algorithms adapt to
data, developing behaviours that were not programmed in advance.

Examples

1. Image Recognition
Image recognition is a well-known and widespread example of machine learning in the real world. It
can identify an object as a digital image, based on the intensity of the pixels in black and white images
or colour images.

Real-world examples of image Recognition:

• Label an x-ray as cancerous or not
• Assign a name to a photographed face (aka "tagging" on social media)
Recognise handwriting by segmenting a single letter into smaller images
Machine learning is also frequently used for facial recognition within an image. Using a database of
people, the system can identify commonalities and match them to faces. This is often used in law
enforcement.
2. Speech Recognition
Machine learning can translate speech into text Certain software applications can convert live voice
and recorded speech into a text file. The speech can be segmented by intensities on time-frequency
bands as well.
Real-world examples of Speech Recognition:
• Voice search
• Voice dialling
Appliance control
Some of the most common uses of speech recognition software are devices like Google Home or
Amazon Alexa.
3. Medical Diagnosis
Machine learning can help with the diagnosis of diseases. Many physicians use chatbots with speech
recognition capabilities to discern patterns in symptoms.

Real-world examples for Medical Diagnosis:

• Assisting in formulating a diagnosis or recommending a treatment option
• Oncology and pathology use machine learning to recognise cancerous tissue
Analyse bodily fluids
In the case of rare diseases, the joint use of facial recognition software and machine learning helps scan
patient photos and identily phenotypes that correlate with rare genetic diseases.__________________
 T h « i

4. Statistical Arbitrage

r ™ “ := ^ r ir " .x - 'iT = c s .r r i ...

correlations.
Real-world examples o f Statistical Arbitrage:
. Algorithmic trading which analyses a market microstructure
. Analyse large data sets
• Identify real-time arbitrage opportunities______ _______—
Machine learning optimises the arbitrage strategy to enhance results.

5. Predictive Analytics
Machinelearningcan classify [Link],whicharethe„defin^by

i/Vhentheclassificati(
Real-world examples o f Predictive Analytics:
. Predicting whether a transaction is fraudulent or legitimate
Improve prediction systems to calculate the possibility of f a ^
Predictive analytics is one of the most promising examples of machine leammg. It s applicable for
everything: from product development to real estate pricing.

annotating datasets for predictive analytics tools.

Real-world examples o f extraction:
. Generate a model to predict vocal cord disorders
. Develop methods to prevent, diagnose, and treat the disorders
• Help physicians diagnose and treat problems quickly
TVpically, these processes are tedious. But machine learning can track and extract informatron to

obtain billions of data samples.

K i l l T y p e s o f M a c h i n e L ie a r n in g

!• several types o f m achine learning, each with special characteristics and applications. Som
o f the main types of m achine learning algonthms are as follows.

1. Supervised Machine Learning

2. Unsupervised M achine Learning
3. Semi-Supervised M achine Learning
4. Reinforcement Learning.
 S u p e r v is e d L e a r n i n g
The word supervised implies "to o b serv e and con trol tlie p erform ance o f a ta s k . In supervised
m achine learning, supervised m eans working in the presence of supervision. The machine is
first trained by providing labelled data, and then the m achine is allowed to predict the outcomes.
Supervised machine learning algorithm s are intended to learn by example.
In supervised learning, sample labelled data are provided to the machine learning system for training,
and the system then predicts the output based on the training data. The system uses labelled data
to build a model that understands the datasets and learns about each one. After the training and
processing are done, we test the m odel with sample data to see if it can accurately predict the output

f( What is Supervised Machine Learning ?

Supervised learning is a tj^ e of machine learning where the algorithm learns to map input data to
the correct output by being trained on labeled examples. In supervised learning, the training data
consists of input-output pairs, where the input data is accompanied by the corresponding correct
output or label. The goal of supervised learning is for the algorithm to learn a mapping function
from the input to the output so that it can make accurate predictions on new, unseen data^_______

Key Components of Supervised Machine Learning

1. Labeled Data: In supervised learning, the training data is labeled, meaning th a t each input
data point is associated w ith the correct output or targ et label. For example, in a spam email
detection system, each em ail is labeled as either spam or not spam.
2. Training Phase: During th e training phase, the algorithm uses the labeled data to learn the
relationship between the input features and the corresponding output labels. The algorithm
adjusts its internal param eters based on the training data to minimize the erro r betw een its
predictions and the true labels.
3 . Testing Phase: Once the model is trained, it is tested on a separate test d ataset th a t contains
new data that it has not seen before.
4 . Prediction: Once the m odel is trained, it can be used to make predictions on new, unseen
data. The model takes the input data and uses the learned mapping function to predict the
corresponding output label.
 Machine JLcwrning ■■'d'h

5 . Evaluation: The perform ance of a supervised learning model is evaluated by comparing its
predictions on a separate test dataset with the true labels. Common evaluation m etrics include
accuracy, precision, recall, and FI score, depending on the nature of the problem.

Example How Supervised Learning Works?

In supervised learning, models are trained using labelled dataset, where the model learns about each type
of data. Once the training process is completed, the model is tested on the basis of test data (a subset of the
training set], and then it predicts the output.

The working of Supervised learning can be easily understood by the below example and diagram:
Labeled Data

Prediction
Square
1

Model Training
A Triangle

Laboles

O " □
Hexagon y V r Square
Test Data

Triangle

Suppose we have a dataset of different types of shapes which includes square, rectangle, triangle, and Polygon.
Now the first step is that we need to train the model for each shape.
• Labeled Data:

If the given shape has four sides, and all the sides are equal, then it will be labelled as a Square.
- If the given shape has three sides, then it will be labelled as a triangle.
- If the given shape has six equal sides dien it will be labelled as hexagon.
• Training Phase : The machine learning model is trained on this labeled dataset to learnthe patterns
and relationships between the input features (number of sides) and the corresponding class labels.
• Testing Phase: Once the model is trained, it is tested on a separate test dataset that contains new
instances of shapes that it has not seen before. The model takes the features of each shape in the test
set [e.g., number of sides) and predicts the class label based on the learned mapping function from the
training phase.
• Prediction: For each new shape in the test set, the model predicts the class label based on the learned
criteria (e.g., number of sides). If the shape has four equal sides, the model would predict it as a square.
If it has three sides, it would predict it as a triangle, and so on based on the defined rules.
•_Evaluation: The performance of the model is evaluated based on how accurately it predicts the class
______labels for the shapes in the test set

Types or Categories of Sujiervised Machine Learning Algorithms

upervised machine learning algorithms can be categorized into two main types: Regression and
lassification.
^ —----— ————— ——— — — — — -Fundamentals o f M achine'
.................................

1. C lassification A lgorithm s: Classification algorithm s are used when the output variable is
categorical, m eaning it falls into distinct cla sse s or categories. The goal is to predict the class
label of new data points based on the p atte rn s learned from the training data. For example,
classifying em ails as spam or not spam, o r predicting whether a patient has a high risk of
heart disease o r not. Classification algorithm s learn to map the input features to one of the
predefined classes.
A list of co m m o n classification a lg o rith m s used in supervised learn in g:
o Logistic Regression ° Decision Trees
o Random Forest ° Support Vector Machines (SVM)
o Naive Bayes ° K-Nearest Neighbors (KNN)

Examples Classification Algorithm s fo r Supervised Machine L earn in g

1. Spam Filtering: In email filtering, a classification algorithm can be used to classify incoming
emails as either spam or non-spam based on the content and features of the email.
Algorithm: A common algorithm used for this task is Naive Bayes, which calculates the
probability of an email being spam or non-spam based on the presence of certain keywords or
features in the email content
2. Customer Chum Prediction: Predicting whether a customer is likely to churn (cancel their
subscription or service) based on historical data and customer behavior.
Algorithm: Random Forest is a popular algorithm for this task, as it can handle complex
relationships in the data and provide insights into the factors influencing customer chum.
3. Sentim ent Analysis: Analyzing text data to determine the sentiment (positive, negative,
neutral) expressed in reviews, social media posts, or customer feedback.
Algorithm: Support Vector Machines (SVM) are commonly used for sentiment analysis tasks
due to their ability to handle high-dimensional data and find optimal decision boundaries
between classes.
4. Image Classification: Classifying images into predefined categories such as animals, objects,
or scenes based on their visual features.
Algorithm: Convolutional Neural Networks (CNNs) are widely used for image classification
tasks due to their ability to learn hierarchical features from images and achieve state-of-die-art
performance.
5. Medical Diagnosis: Predicting the presence or absence of a disease based on patient symptoms,
medical history, and test results.
Algorithm: Decision Trees can be used for medical diagnosis tasks to create interpretable rules
for identifying patterns in patient data and making diagnostic decisions.
6. Fraud Detection: Identifying fraudulent transactions or activities in financial systems to
prevent financial losses.
Algorithm: Logistic Regression is commonly used for fraud detection due to its ability to model
binary outcomes and provide probabilities of fraudulent behavior based on transaction data.
2. R egression A lg o rith m s: Regression algorithm s are used when the relationship between the
input variables and the continuous output variable needs to be predicted. The goal is to estimate
a continuous value based on input featu res For example, predicting the price of a house based
 ^ ch in e Learning

on its size, location, and amenities, or forecasting the sales of a prod u ct Regression algorithms
leam to map the input features to a continuous numerical value.
A lis t o f com m o n reg ressio n alg o rith m s used in supervised le a rn in g :
o Linear Regression o Polynomial Regression
o Ridge Regression o Lasso Regression
o Decision tree Regression o Random Forest Regression
o Support Vector Regression (SVR) o Gradient Boosting Regression

Examples Regression Algorithms for Supervised Machine L earning

1. House Price Prediction: Predicting the selling price of a house based on features like area,
number of bedrooms, location, etc.
Algorithm: Linear Regression is commonly used for house price prediction as it models the
relationship between the input features and the house price with a linear equation.
2. Stock Price Forecasting: Forecasting the future price of a stock based on historical stock data,
market trends, and other relevant factors.
Algorithm: Time Series Forecasting techniques like ARIMA (AutoRegressive Integrated Moving
Average) or Prophet can be used for stock price prediction to analyze and predict stock price
movements over time.
3. Demand Forecasting: Predicting the demand for a product or service in the future based on
historical sales data, market trends, and external factors.
Algorithm: Random Forest Regression can be employed for demand forecasting tasks to
capture complex relationships in the data and predict future demand levels accurately.
4. Temperature Prediction: Forecasting temperature values for weather forecasting applications
based on historical weather data and meteorological factors.
Algorithm: Support Vector Regression (SVR) is a regression algorithm that can be used
for temperature prediction tasks by finding the optimal hjfperplane to predict continuous
temperature values.
5. Sales Revenue Prediction: Estimating future sales revenue for a company based on historical
sales data, marketing campaigns, and economic indicators.
Algorithm: Gradient Boosting Regression algorithms like XGBoost or LightGBM are effective
for sales revenue prediction tasks as they can handle large datasets and capture non-linear
relationships in the data.

Applications of Supervised Learning

Applications o f Supervised Learning include:

• Im age C lassification: Used to recognize objects, faces, and features within images.
• Natural Language Processing: Involves extracting insights from text, including sentim ent
analysis, entity identification, and relationship extraction.
• Speech R ecognition : Converts spoken language into text for transcription purposes.
• R ecom m en d ation Systems: Offer personalized suggestions to users based on their
preferences.
 Fundamenlafs of JV ta ^ ^ S w rn in g

• P re d ictiv e Analytics: Utilized for forecasting outcomes such as sales figures, custom er
retention rates, and stock m arket trends.
• M ed ical Diagnosis: Helps in identifying diseases and medical conditions from patient data.
• Fraud D etection: Identifies and flags potentially fi^udulent transactions.
• A utonom ou s Vehicles: Enables vehicles to detectand respond to objects in dieir surroundings.
• E m ail S p am Detection; Classifies incom ing emails as eitiier spam or legitimate.
• Q u ality Control in M anu factu rin g: Used to inspect products for defects and maintain quality
standards.
• C red it Scoring: Assesses the cred it risk associated with borrow ers to predict loan default
probabilities.
• G am ing: Analyzes player behavior, character recognition, and NPC creation in gaming
environments.
. C u sto m er Su p p o rt Automates task s in customer service for improved efficiency.
. W e a th e r Forecasting: Predicts m eteorological parameters like temperature and precipitation
for w eather forecasts.
. S p o rts Analytics; Analyzes player performance, predicts gam e outcomes, and optim izes
strategies for sports teams.

A dvantages of Supervised M achine L earn in g

O Predictive Accuracy: Supervised models can make accurate predictions by learning patterns from
labeled training data.
O Interpretable Results: Models provide insights into the relationship between input features and
target variables.
O Evaluation Metrics: Clear evaluation metrics such as accuracy, precision, recall, and FI score enable
easy model performance assessment.
O Feature Importance: Identify the most relevant features that contribute to the prediction outcome.
O Generalization: Supervised models can generalize well to unseen data, making them suitable for real-
world applications. ____________ ^
____________ ___________________ ______________________

D isadvantages of Supervised M achine Learning

C Data Dependency: Supervised learning requires labeled training data, which can be time-consuming
and expensive to acquire, especially for large datasets.
C Overfitting: Models trained on labeled data may overfit, capturing noise or irrelevant patterns in the
training set and leading to poor generalization on unseen data.
C Limited Flexibility: Supervised models are constrained by the features and labels provided in the
training data, limiting their ability to adapt to new or unseen patterns.
 A

Machine learning , . ^r.*;/,/'

U n s u p e r v is e d L e a r n i n g
The word Unsupervised implies " n o t being done o r a c tin g u n d e r supervision". Unsupervised
learning is a learning method in which a machine learns w ithou t any supervision. The training is
provided to the machine with the s e t of data that has not been labelled, classified, or categorized, and
the algorithm needs to act on that data without any supervision.
The prim ary goal of Unsupervised learning is often to d iscov er hidden patterns, sim ilarities, or
clu sters within the data, which can then be used for various purposes, such as data exploration,
visualization, dimensionality reduction, and more.

What is Unsupervised Machine Learning ?

Unsupervised learning is a type of machine learning where the algorithm learns to identify patterns
and relationships in data without being explicitly trained on labeled examples. Unlike supervised
learning, unsupervised learning algorithms work on unlabeled data, where the algorithm tries to
find hidden structures or patterns within the data. The goal of unsupervised learning is to explore
the data and extract meaningful insights without the need for predefinedjabels^^^^^^^^^^^^

Key Components of Unsupervised Machine Learning

1. U nlabeled Data: In unsupervised learning, the algorithm works with unlabeled data, meaning
that the input data points do not have corresponding output labels. The algorithm s task is to
discover the underlying structure or patterns in the d ata on its own.
2. Clustering: Clustering is a common technique in unsupervised learning where the algorithm
groups similar data points together based on th eir features or characteristics. Clustering
algorithms aim to partition the data into clusters such th a t data points within the sam e cluster
are more similar to each o th er than to those in other clusters.
3. D im ensionality R ed u ctio n : Dimensionality reduction techniques are used in unsupervised
learning to reduce the num ber of features in the data w hile preserving important information.
This helps in visualizing high-dimensional data and rem oving noise or redundant information.
4 . Anomaly D etection: Unsupervised learning algorithm s can also be used for anom aly detection,
where the algorithm identifies data points that deviate significantly from the norm o r expected
behavior. Anomalies are data points that are rare or unusual compared to the m ajority of the
data.
5. A ssociation Rule L earn in g : Association rule learning is another technique in unsupervised
learning that discovers interesting relationships or associations between variables in large
datasets. It is commonly used in market basket analysis to identify patterns in consumer
behavior.
 Fundamentals of Machine Learning ^ 1.23

Example How U nsupervised Machine Learning W ork s?

The below diagram illustrates a conceptual representation of an unsupervised learning model in machine
learning.

M odel

1. Input Data: This is the dataset given to the model. In unsupervised learning, the data has no labels. The
shapes (circles, triangles) represent different data points with their features.
2. Model: This is the core of the machine learning system. The model finds patterns or insights in the
input data without labeled guidance. In unsupervised learning, common tasks include clustering,
where the model groups similar data points together, and dimensionality reduction, where the model
simplifies inputs by removing redundant features to make data more understandable.
3. Output Data: This is what the model produces after processing the input In unsupervised learning, the
output is not predicted labels but insights from the data. The output data, shown as shapes, indicates
that the model may have categorized or clustered the data based on their characteristics.
The diagram gives a general idea of how unsupervised learning processes data to uncover its structure.

Unsupervised machine learning algorithms can be categorized majorly into three main types:
C lustering, D im ensionality R ed uction and A sso ciation .
1. Clustering : Clustering, algorithms are used to group similar data points together based on
their inherent characteristics or features. The goal is to discover natural groupings or clusters
within the data w ithout any predefined class lab els. Clustering algorithms learn to identify
patterns in the data and assign data points to clu sters. Common tasks include grouping
customers based on purchasing behavior or clu stering documents based on content similarity.
List of com mon c lu s te rin g algorithm s used in u n su p erv ised learning:
• K-Means Clustering
• Hierarchical Clustering
• DBSCAN (Density-Based Spatial Clustering o f Applications with Noise)
• Gaussian Mixture Models

C lustering Algorithms for U nsupervised Machine Learning

1. Customer Segmentation: Identifying different customer groups based on purchasing behavior

and preferences to tailor marketing strategies.
Algorithm: K-Means is widely used for customer segmentation due to its efficiency in grouping
data into distinct non-overlapping clusters. ____________________________________
1.24 ^*^e^rneUorn1ng'=

2. Image Segmentation: Sep aratin g an image into different regions for image analysis and object
recognition.
Algorithm: Hierarchical clustering can be utilized in image segmentation for grouping pixels
or features that exhibit similar characteristics.
3. A n o m a ly Detection: Identifying miusual patterns or outiiers that do not conform to expected
behavior, such as fraud or network intrusions.
Algorithm: DBSCAN is effective for anomaly detection as it can find outiiers in datasets witii
noise and varied densities.__________________
2. Dimensionality Reduction Algorithms: Dimensionality reduction algorithm s are used to
reduce the number of input features in a dataset w hile presening im portant information. The
goal is to simplify the data by eliminating redundant or irrelevant features, m aking it easier to
visualize and analyze. Dimensionality reduction techniques are beneficial for high-dimensiona
data visualization and feature selection.
List of common dimensionaUty reducdon algorithms used in unsupervised learning:
• Principal Component Analysis (PCA)
• t-Distributed Stochastic Neighbor Embedding (t-SNE)
• Singular Value Decomposition (SVD)
• Independent Component Analysis (ICA)
. M achine Learning
Examples
1. Feature Selection: Reducing the number of input variables to simplify models and eliminate
redundancy.
Algorithm: PCAisfrequentlyused to tt-ansformhigh-dimensionaldataintoalower-dimensional

form while retaining most of the original variance.

2. Data Visualization: Reducing high-dimensional data to two or three dimensions for
visualization and exploration.
Algorithm:t-SNEisatechniquefordimensionalityreductiontiiatisparticularlywellsuitedfor

the visualization of high-dimensional datasets.

3. A ssociationA lgorithm s:A ssodationalgorithm sareatypeofunsupervisedlearnm gm ethod

that discovers interesting relationships, frequently occurring patterns, correlations, or

associations among a set of items in large d atasets. They are particularly useful in the context
of large datasets where manual pattern recognition would be impractical.
These algorithms are primarily used for m arket basket analysis but have applications in
various fields like bioinformatics, text mining, and recommendation system s.
List of common association algorithms used in un^pervised learning:
• Apriori Algorithm •

• FP-growth Algorithm
 of Machine Learning
E )

"Examples Association Algorithms fo r Unsupervised Machine L earn in g

1. Market Basket Analysis: Identifies products often purchased together to optimize marketing
strategies.
Algorithm: The Apriori algorithm is widely used for this application. It operates by identifying
the frequent individual items in the database and extending tiiem to larger item sets as long as
those item sets appear sufficiently often in the database.
2. Fraud Detection: Discovers patterns in data that may indicate fraudulent behavior.
Algorithm: Sequential pattern discovery using equivalence classes (SPADE] algorithm can be
used for detecting suspicious sequences of transactions.
3. Recommendation Systems: Recommends items based on user's past behavior.
Algorithm: Collaborative filtering approaches, which may include matrix factorization or
neural network-based recommendation algorithms, utilize association patterns to predict user
preferences.
4. Cross-Marketing: Finds associations between product categories to drive cross-promotional
strategies.
Algorithm: Association Rule Learning (ARL) algorithms, such as Apriori or FP-Growth, help
retailers bundle products in promotions effectively
5. Catalog Design: Arranges items in a catalog to maximize the discoveiy of associated items.
Algorithm: The Eclat algorithm uses transaction id intersections to improve computational
speed, ______________________________ __________________________

★ Clustering: Group similar data points into clusters.

★ Anomaly D e te ctio n : Identify outliers or anom alies in data.
★ D im ensionality Reduction: Reduce the dim ensionality of data while preserving its essential
information.
★ R ecom m en d ation Systems: Suggest products, movies, or content to users based on their
historical behaviour or preferences.
^ Topic M odelling: Discover latent topics w ithin a collection of documents.
★ Density E stim a tio n : Estimate the probability density function of data.
★ Image and V id eo Compression: Reduce the amount of storage required for multimedia
content.
★ Data P re -p ro cessin g : Help with data pre-processing tasks such as data cleaning, imputation
of missing values, and data scaling.
★ M arket B a s k e t A nalysis: Discover associations between products.
★ Genomic D ata A nalysis: Identify patterns o r group genes with sim ilar expression profiles.
★ Image S e g m e n ta tio n : Segment images into meaningful regions.
★ Community D e te ctio n in Social N etw orks: Identify communities or groups of individuals
with similar in terests or connections.
 Machine Ua^nW

C u sto m er Behaviour A nalysis: Uncover patterns and in sig h ts for better marketing and
product recommendations.
^ C o n ten t Recom m endation: Classify and tag content to m ake it easier to recommend sim ilar
item s to users.
^ E x p lo ra to ry Data A nalysis (ED A ): Explore data and gain insights before defining specific
tasks.

Advantages and Disadvantages of Unsupervised Machine Learning

A dvantages of Unsupervised M achin c Learning

O Discover Hidden Patterns: Uncover hidden patterns and structures in data without the need for
labeled outcomes.
0 Data Exploration: Facilitate data exploration and visualization by reducing high-dimensional data to
lower dimensions.
O Anomaly Detection: Identify outliers or anomalies in datasets that deviate from normal patterns.
O Feature Extraction: Extract essential features from data to improve model performance and reduce
dimensionality.
O Scalability: Easily handle large volumes of data and adapt to new data without the need for manual
labeling._________ ____________ ____________________________ _— ----------------- —----------------- —

D isadvantages of Unsupervised Machine 1-earning

C Interpretability: Models may be harder to interpret compared to supervised learning models due to
the lack of explicit labels.
C Evaluation Metrics: Lack of clear evaluation metrics for unsupervised learning tasks can make model
performance assessment challenging.
C Domain Knowledge: Requires domain expertise to interpret and validate the discovered patterns
effectively.
C Computational Complexity: Some unsupervised algorithms can be computationally intensive and
time-consuming, especially for large datasets.

S e m i-S u p e rv is e d M a c h i n e L e a r n in g
Semi-Supervised learning is a type o f Machine Learning algorithm th a t lies between Supervised and
Unsupervised machine learning. It represents the intermediate ground between Supervised (W ith
Labelled training data] and Unsupervised learning (with no labelled training data) algorithm s and
uses the combination of labelled and unlabelled datasets during th e training period.
Although Semi-supervised learning is the middle ground b etw een supervised and unsupervised
learning and operates on the data th at consists of a few labels, it m ostly consists of unlabelled data. It
is com pletely different from supervised and unsupervised learning as they are based on the presence
& absence o f labels. In semi-supervised learning, the algorithm is train ed on a dataset that contains a
small am ount of labeled data and a larger amount of unlabeled data.
m
Fundamentals of Machine learning ^ ^ 1.27

? ( What is Semi-Supervised Machine Learning ?

Semi-supervised learning is a machine learning paradigm where the algorithm learns from a
combination of labeled and unlabeled data. Unlike supervised learning that relies solely on labeled
examples and unsupervised learning that operates on unlabeled data, semi-supervised learning
strikes a balance between the two. The goal is to leverage the labeled data to guide the learning
process and utilize the unlabeled data to discover underl}dng patterns and relationships within the
data.

To overcom e the drawbacks of supervised learning and unsupervised learning algorithms, the
concept o f Semi-supervised learning is introduced. The main aim o f semi-supervised learning is to
effectively u se all the available data, rath er than only labelled data like in supervised learning. Initially,
similar d ata is clustered along with an unsupervised learning algorithm , and further, it helps to label
the unlabelled data into labelled data. It is because labelled data is a comparatively more expensive
acquisition th an unlabelled data.
We can im agine these algorithms with an example. Supervised learning is where a student is under
the su p erv isio n of an instructor at hom e and college. Further, if th a t student is self-analysing the
same co n ce p t without any help from the instructor, it comes u n der unsupervised learning. Under
sem i-supervised learning, the student has to revise himself after analysing the same concept under
the guidance o f an instructor at college.

Key Components of Sem-Supervised Machine Learning

In sem i-supervised learning, we w ork w ith a mix of labeled and unlabeled data to train our algorithm.
The key com ponents are:
1. L a b e le d and Unlabeled D ata: We have some data with lab els (like pictures of cats labeled as
"c a t") and a lot of data without labels. The algorithm learns from both types of data to improve
its understanding.
2. L ab el Propagation: This technique spreads the known labels to similar unlabeled data points.
If a labeled picture of a cat looks similar to an unlabeled picture, we can assume the unlabeled
on e is also a c a t This helps the algorithm learn more from th e unlabeled data.
3. Pseudo-Labeling: The algorithm predicts labels for the unlabeled data based on its current
understanding. These predicted labels are then used to train th e model further. It's like making
ed u cated guesses to teach the algorithm.
4 . Self-Training: The algorithm trains on the labeled data, th en uses its knowledge to predict
la b els for unlabeled data. If it's confident about these predictions, it adds them to the labeled
d a ta se t for future training. This process helps the model learn more from the unlabeled data.
5. Transfer Learning: In sem i-supervised learning, we can use knowledge from pre-trained
m od els on labeled data to assist in tasks with limited labeled examples. This transfer of
know ledge helps the algorithm learn more efficiently and improve its performance in the
sem i-supervised task.
 Example |How SemrSupervised Learning Works?__________ _-------------------------------------------------
Let's consider an example of seml-supervlsed leamlni using the scenario of classilying entails as either spam
or not spam.
1 Labeled and Unlabeled Data: We have a small set of labeled emails where some are marked as spam
■ and others as non-spam. The majority of emails in our dataset are not labeled as spam or non-spam.
2. LabelPropagation:ThealgorithmlooksatthelabeledemailsandtheircharacteristicsOikekeywords,
senderinformation)[Link],hrun abe edem a s
For instance, if a labeled email with the word "discount" is marked as spam, similar unlabeled emails
with "discount" may also be considered spam.
3. Pseudo-Ubeling: The algorithm predicts labeU for the unlabeled emails based on its InlM
understanding. If It predicts that an email is likely spam based on its content, it assigns a pseudo la
of "spam" to that email.
4. Self-Training: The algorithm trains on the labeled emails and uses this knowledge to predict labels for
the unlabeled emails. If it is confident in its predictions (e.g., high probability of an email being spam),
it adds these emails with pseudo-labels to the training set for further training iterations.
5. Transfer Learning: If there are pre-ti^ined models for email classification taste with labeled data,
we can leverage their knowledge to improve our semi-supervised learning model s performance. The
insights gained from these pre-trained models can help our algorithm better understand and classify
emails in the semi-supervised setting.
By combining these techniques in a semi-superrised learning approach, our algorithm can
a larger set of emaiU as spam or non-spam, ewn with limited labeled data, leadmgto improved email flltenng

T here are several types or categories ofsemi-supetvised m achine learning algorithms. Here are some
com m on ones:
1 Self-Training A lgorithm s: These algorithms iteratively train on the labeled data and then use
' the model to predict labels for the unlabeled data. T h e high-confidence predictions are added
to the labeled dataset for further training.
2 Co-Training A lgorithm s: In co-training, the algorithm trains multiple models on different
subsets of features or data. Each model then provides predictions for the unlabeled data, and
the agreement between th e models helps in labeUng th e unlabeled instances.
3. Semi-Supervised Support Vector MaciUnes (S3VM): S3VM extends traditional Support
Vector Machines [SVM) to incorporate unlabeled d ata in the learaing process. It aim s to And a
decision boundary that n o t only separates the labeled d ata but also considers the distribution
o f the unlabeled data.
4 Graph-Based A lgorith m s: These algorithms re p rese n t the data as a graph w here nodes are
' data points and edges represent relationships b etw een them. By propagating labels^ through
the graph,thesealgorithmscanleverage the structure ofthedataforsem i-supervisedlearm ng.

5. Generative M odels: Generative models, such as Generative Adversarial N e tw o rk (GANs] or

Variational Autoencoders (YAEs), can be used for semi-supervised leam m g by learnmg the
 .,.,.............................
Funddmentais of Machine Learnir

underlying distribution o f the data. They can generate new data points and help in improving
the model's understanding of the data distribution.
6 . Low-Density S e p a ra tio n Algorithms: These algorithm s aim to find a decision boundary
that separates high-density regions (labeled data) fi-om low-density regions (unlabeled data].
By considering the density of the data points, th ese algorithms can effectively classify both
labeled and unlabeled instances.
Each type of semi-supervised learning algorithm has its strengths and is suitable for different types
o f datasets and learning tasks. Researchers and practitioners choose the most appropriate algorithm
b ased on the characteristics o f the data and the specific learning objectives.

Applications of Semi-Supervised Machine Learning

Semi-supervised learning has various applications across different domains due to its ability to
leverage both labeled and unlabeled data efficiently. Here are some common applications of semi-
supervised learning:
1. Text Classification: In natural language processing tasks like sentiment analysis, document
categorization, or spam detection, sem i-supervised learning can be used to improve
classification accuracy by utilizing a combination o f labeled and unlabeled text data.
2 . Image Recognition: Semi-supervised learning is beneficial in image recognition tasks where
there is a large am ount o f unlabeled image data. By training on a small set of labeled images
and propagating labels to similar unlabeled images, the model can learn to recognize patterns
and objects more effectively.
3 . Speech R ecognition: Semi-supervised learning can Enhance speech recognition systems
by utilizing both labeled and unlabeled speech data. By leveraging the sim ilarities between
labeled and unlabeled speech samples, the model can improve its accuracy in transcribing
speech.
4 . Anomaly D etection: In cybersecurity and fraud detection, semi-supervised learning can help
identify anomalies in data by learning the norm al patterns from labeled data and detecting
deviations in the unlabeled data.
5 . Drug Discovery: In the pharmaceutical industry, semi-supervised learning can be applied to
predict the properties o f new drug compounds by training on a small set of labeled compounds
and leveraging the vast amount of unlabeled chem ical data available.
6 . Recom m endation System s: Semi-supervised learning can enhance recommendation systems
by utilizing both explicit user ratings (labeled d ata) and implicit user behavior (unlabeled
data) to provide m ore personalized and accurate recommendations.
7 . Medical Image A nalysis: In medical imaging tasks such as tumor detection or disease
diagnosis, sem i-supervised learning can assist in analyzing large volumes of medical images
by combining labeled images with similar unlabeled images to improve diagnostic accuracy
8 . Social Network A nalysis: Semi-supervised learning can be used in social network analysis
to predict connections o r identify communities w ithin a network by leveraging both labeled
connections and the netw ork structure of unlabeled data.
AdvantageB and Disadvanatges [Link] M a t o l ^ n g

Advantages o f ijemi-oupei
kdvantages ot Semi-Superviscd
viwu Machine Learning ...................................................... ...........
................. .
"Efficient Use Data: Setni-^pervised learning allows for the utilization of large amounts
ofunlabeled data,Which isoftenmoreabundantandeasiertoobtainthanlabeled data. This canlea

improved model performance without the need for extensive labehng efforts.
3 Cost-Effective: By reducing the reliance on labeled data, semi-supervised learning can be more
cost-effective compared to supervised learning, especially m scenarios where a e mg a i
consuming or expensive. .

=r :r r r r
unlabeled instances. . . r
O Performance: Seml-supeivised learning can lead to improved model
especially in cases where labeled daU is scarce. By incorporaSng unlabeled data, the model can
more robust representations and make better predictions.
S Scalability:Semi-supervisedlearningtechnlquescanscalewelltolargedatasets,astheycaneffectively
I------ r - nf iinlabeled data to enhance the model’s learning process^ -----------
^ ^f-'...
_ _ ------ T-;—-T— -------- 7 ^ ------ , - • ... TT'T^Tsmr’ i - - ^
r a l D isady^teges o f Semi-Supervised Machine Leari^^ng y^v-. ____
e OuaitoofUnlabeledData:Theeffectivenessofseml-supervisedlearningheavilyreliesontheqmlity
and reTevance of the unlabeled data. If the unlabeled data is noisy or contains irrelevant mformaOon,
aiiu leicvain^t; \ji --------------------
can negatively impact the model's performance.
C Model Complexity: Implementing semi-supervised learning algorithms can be more complex than
[Link],[Link],on

or pseudo-labeling.
e Risk of Overntting: In some cases, semi-supervised learning models may be P™ "' “
^ ^ d llly when the unlabeled data Introduces noise or biases that are not effecttveiy handled dunng

C u I7 d control: Unlike supervised learning where the labels are explicitly

learning relies on the model's ability to infer labels from the unlabeled data. This lack
supervision can make it challenging to interpret and debug the model s decisions.
C Domain Dependency: The effectiveness of semi-supervised learning techniques can vary depending
o n T e s p e l c d o m l and dataset characteristics. Some algorithms may perform better ,n certam
domains while being less effective in others^-------------------- ------------------- -------------------------

1111111 R e i n f o r c e m e n t L e a rn in g

T-»or*fr^rmanrP.
improves its perform ance.
 - J ' Fa^am entals of Machine Learning Mi .31

Trial, error, an d d elay are the most relevant characteristics of rein fo rcem en t learning. In this
technique, the m odel keeps on increasing its performance using Rew ard Feedback to learn the
behaviour or pattern.
E n v i r o n m erit:

R e w a r d , S t:a te
A c tio n

A g e n t:

R einforcem ent Machine Learning

Let us consider th e robotic dog as the agent, the movement of its arm s as th e actions it takes, and
the environment as the space in which it operates. By receiving feedback in the form of rewards or
penalties based on th e success of its arm movements, the robotic dog can a d ju st its actions over time
to optimize its perform ance.

Key Components o f Reinforcement Learning

1. Agent: The entity that learns and m akes decisions based on the environm ent's feedback. The
agent takes actions in the environment to achieve a specific goal.
2. Environm ent: The external system with which the agent interacts. T h e environment provides
feedback to th e agent in the form of rew ards or penalties based on th e agent's actions.
3. Actions: T h e s e t of possible choices th at the agent can take in a given state. The agent selects
actions based on its policy, which defines how it chooses actions in different states.
4. State: The cu rren t situation or configuration of the environment at a particular time step. The
agent's actio n s influence the transition from one state to another.
5. Rewards: Numeric feedback provided by the environment to the ag en t after each action. The
agent's objectiv e is to maximize the cumulative reward it receives o v er time.
6. Policy: The strategy or set of rules that the agent uses to select actio n s in different states. The
policy can b e deterministic or stochastic.
Reinforcement learning algorithms aim to learn an optimal policy th a t maximizes the expected
cumulative rew ard over time.
Jig Example ' How Reinforcement Learning Works?

Let's consider an example of training an autonomous vehicle using reinforcement learning.

1. Agent: Imagine a smart car as the agent It's like a student learning to drive.
2. Environment: The simulated environment represents the external system with which the autonomous
vehicle interacts. It provides feedback to the vehicle based on its actions, such as collisions or successful
navigation.
3. Actions: The smart car can speed up, slow down, turn left turn right, or keep going straight These are
like the choices it makes v\^ile driving.
4. State: The state is like the current situation on the road - wdiere the car is, how fast it's going, and if
there are any obstacles nearby.
5. Rewards: When the smart car drives well, it gets points (rewards). For example, reaching a destination
gives it points, but hitting an obstacle takes points away.
6. Policy: The smart car's policy is its driving strategy. It decides how to drive based on what's happening
around i t It could follow simple rules like "avoid obstacles" or more complex strategies based on its
surroundings.
In this scenario, the smart car learns to drive safely and efficiendy by moving around the virtual road, earning
points for good driving, and adjusting its driving strategy to maximize its total points.____________________

Some of most common reinforcement learning algorithms

Q -learn irig i Q-leaming is a m odel-free RL algonAm that le a m s a Q-function, which maps

states to actions. The Q-function estim ates the expected rew ard o f taking a particular action in
a given state.
SARSA (State-A ction-R ew ard -State-A ction ); SARSA is an o th e r model-free RL algorithm
that le a m s a Q-function. However, unlike Q-leaming, SARSA updates the Q-function for the
action th at was actually taken, ra th e r than the optimal action.
D eep Q -leam in g : Deep Q-leaming is a combination of Q -learning and deep learning. Deep
Q -leam ing uses a neural network to represent the Q-fimction, w hich allows it to learn complex
relationships between states and actions.

Applications of Reinforcement Machine Learning

> Gam e P la 5ang: RL can teach agents to play games, even com plex ones.
> R o b o tics: RL can teach robots to perform tasks autonomously.
> A u to n om ou s Vehicles: RL can help self-driving cars navigate and make decisions.
> R eco m m en d atib n Systems: RL can enhance recommendation algorithms by learning user
preferences.
> H e a lth ca re : RL can be used to optim ize treatment plans and d rug discoveiy.
> N atural Language P rocessing (N LP): RL can be used in dialogue systems and chatbots.
> F in a n ce a n d Trading: RL can be used for algoritiimic trading.
> Supply C hain and Inventory M a n a g e m e n t RL can be u sed to optimize supply chain
operations.
 bIs of Machine Learning

> E n erg y M anagem ent RL can be used to optimize energy consumption.

> Game AI: RL can be used to create m ore intelligent and adaptive NPCs in video games.
> A daptive Personal Assistants: RL can be used to improve personal assistants.
> V irtu al Reality (VR) and A ugm ented Reality (AR): RL can b e used to create immersive and
interactive experiences.
> In d u strial Control: RL can be used to optimize industrial processes.
> E d u catio n : RL can be used to create adaptive learning systems.
> A gricu ltu re: RL can be used to optimize agricultural operations.

Advantages and Disadvantages of Reinforcement Machine Learning

1 1 ^ 1 A d W n tages of Reinforcement M achine Learning

O It has autonomous decision-making that is well-suited for tasks and that can learn to make a sequence
of decisions, like robotics and game-playing.
O This technique is preferred to achieve long-term results that are very difficult to achieve.
O It is used to solve a complex problems that cannot be solved by conventional techniques.---------------

D isadvantages of Reinforcement M achine learn in g' :

C Training Reinforcement Learning agents can be computationally expensive and time-consuming.

C Reinforcement learning is not preferable to solving simple problems.____________________ __

m i Difference between Supervised and Unsupervised Learning

■*‘'Supemsed beaming ' ’ Unsupervised Leai^ing. -

Supervised learning algorithms are trained using Unsupervised learning algorithms are trained using
labelled d ata.______________________ ____________ unlabelled data.
Supervised learning model takes direct feedback to Unsupervised learning model does not take any
check if it is predicting correct output or not. feedback.
Supervised learning model predicts the output. Unsupervised learning model finds the hidden
patterns in data.
In supervised learning, input data is provided to the In unsupervised learning, only input data is provided
model along with the output to the model.
The goal of supervised learning is to train the model so The goal of unsupervised learning is to find the
that it can predict the output when it is given new data. hidden patterns and useful insights from the unknown
dataset
Supervised learning needs supervision to train the Unsupervised learning does not need any supervision
model._________________________________________ to train the model.
Supervised learning can be categorized Unsupervised Learning can be classified
in Classification and Regression problems. in Clustering and Associations problems.
Supervised learning can be used for those cases where Unsupervised learning can be used for those cases
we know the input as well as corresponding outputs. where we have only input data and no corresponding
output data.
Supervised learning model produces an accurate Unsupervised learning model may give less accurate
result. result as compared to supervised learning.

Supervised learning is not close to true Artificial Unsupervised learning is more close to the true
intelligence as in this, we first train the model for each Artificial Intelligence as it learns similarly as a child
data, and then only it can predict the correct output learns daily routine things by his experiences.

Machine learning is a buzzword for today's technology, and it is growing very rapidly day by day. We
are using machine learning in our daily life even w ithout knowing it such as Google Maps, Google
assistant, Alexa, etc.
Below are some most trending real-world applications o f Machine

1. Image R ecognition:
Image recognition is one of the most common applications of machine learning. It is used to
identify objects, persons, places, digital images, etc. The popular use case of im age recognition
and face detection is, A utom atic fiiend tagging suggestion.
Facebook provides us a feature of auto friend tagging suggestion. Whenever we upload a photo
with our Facebook friends, then we automatically get a tagging suggestion w ith name, and the
technology behind this is machine learning’s face d etectio n and reco g n ition algo rith m .
It is based on the Facebook project named "Deep Face,” which is responsible for face recognition
and person identification in the picture.
2. Speech R ecognition
While using Google, we get an option of "S e arch by voice," it comes under sp eech recognition,
and it's a popular application of machine learning.
Speech recognition is a process of converting voice instructions into text, and it is also known
as "Speech to text", o r "Com puter speech recognition ."
At present, machine learning algorithms are widely used by various applications of speech
recognition. Google assistan t, Siri, C ortana, and Alexa are using sp eech recognition
technology to follow the voice instructions.
 '" •''•^’■‘iA ■ ■ Fundamentals of Machine Learning

3. Traffic Prediction:
If we want to visit a n ew place, we take help o f Google Maps, which shows us the correct path
with the shortest ro u te and predicts the traffic conditions.
It predicts the traffic conditions such as w hether traffic is cleared, slow-moving, or heavily
congested with the h elp o f two ways:
o Real Time lo c a tio n o f the vehicle form Google Map app and sensors
o Average tim e h a s taken on past days at the sam e time.
Everyone who is u sing Google Map is helping this app to make it better. It takes information
from the user and sen d s back to its database to improve the performance.
4 . Product R ecom m en d atio n s:
Machine learning is w idely used by various e-com m erce and entertainment com panies such
as Amazon, Netflix, etc., for product recom m endation to the user. W henever w e search for
some product on A m azon , then we started getting an advertisement for the sam e product
while internet surfing on the same browser and this is because of machine learning.
Google understands th e user interest using various machine learning algorithm s and suggests
the product as per cu stom er interest.
As similar, when w e use Netflix, we find som e recommendations for entertainm en t series,
movies, etc., and this is also done with the help o f machine learning.
5. Self-Driving C ars:
One of the most exciting applications of machine learning is self-driving cars. M achine learning
plays a significant ro le in self-driving cars.
Tesla, the most p opu lar car manufacturing com pany is working on self-driving car.
It is using unsupervised learning method to train the car models to detect people and objects
while driving.
6. Email Spam and M alw are Filtering:
Whenever we receive a new email, it is filtered automatically as important, norm al, and spam.
We always receive an important mail in our inbox with the important symbol and spam emails
in our spam box, and th e technology behind this is Machine learning.
Below are some spam filters used by Gmail:
o Content Filter
o Header filter
o General black lists filter
o Rules-based filters
o Permission filters
Some machine learn in g algorithms such as M ulti-Layer Perceptron, D ecision tre e , and Naive
Bayes classifler a re used for email spam filtering and malware detection.
7. Virtual P erson al A ssistan t:
We have various virtu al personal assistants such as Google assistant, Alexa, C ortana, Sin. As
the name suggests, th ey help us in finding the inform ation using our voice instruction. These
1 .3 6

assistants can help us in various ways ju st by our voice instructions such as Play music, call
someone. Open an email, Scheduling an appointment, etc.
These virtual assistants use machine learning algorithms .These assistant record our voice
Instructions, send it over the server on a cloud, and decode it using Machine U am m g
algorithms and act accordingly.
8. Online F rau d Detection:
Machine learning is making our online transaction safe and secure by detecting fraud
transaction.
Whenever we perform some online transaction, there may be various ways that a fra';‘dulent
transaction can take place such as fake accou n ts, fake ids, and s te a l m o n ey in A e middle of
a transaction. So to detect this. Feed Forw ard Neural netw ork helps us by checking whether
it is a genuine transaction or a fraud transaction.
Foreach genuine transaction, the output is converted into some hash values, and these values
become the input for the next round.
For each genuine transaction, there is a specific pattern which gets change for the fraud
transaction hence, it detects it and makes our online transactions m ore secure.

9. Sto ck M a rk et Trading:
Machine learning is widely used in stock market trading. In the stock market, there is always
a risk of up and downs in shares, so for this machine learning’s lo n g sh o rt term m em ory
n eu ral n e tw o rk is used for the prediction of stock market trends.

10. Medical D iagnosis: . u- ^- i

In medical science, machine learning is used for diseases diagnosis. With this, medical
technology is growingveryfest and able to buildSD models that can predict theexact position
of lesions In the btain. It helps in flnding brain tumors and other brain-related diseases easily.

11. A utom atic Language Translation:

Nowadays, if we visit a new place and we are not aware of the language then it is not a problem
atalU sforthisalsom achinelearninghelpsusbyconvertingthetextintoourknow nlanguages.

Google's GNMT (Google Neural Machine Translation) provide this feature, which is a Neural
Machine Learning that translates the text into our familiar language, and it called as automatic
translation.
The technology behind the automatic translation is a sequence to sequence learning algonthm,
which is used with image recognition and translates the text fi-om one language to another
language.

R f l T M a c h i n e l e a r n i n g L ife C y c le
HScfflne learning has given the computer systems the abilities to automatically
explicitly programmed. But how does a machine learning system work? So, it can be described us g
the life cycle o f m achine learning.
Machine learning life cycle is a cyclic process to build an efficient machine learning project The main
purpose of the life cycle is to find a solution to the problem or project.
I m
Fundamentals of Machine Lea

Machine learning life cycle involves seven m ajor steps, which are given below :
• Gathering D ata • Data preparation
• Data W rangling • Analyse Data
• Train the m od el • Test the model
• Deployment

In the complete life cycle process, to solve a problem, we create a m achine learning system called
“model" and this m odel is created by providing "training". But to train a m odel, we need data, hence,
life cycle starts b y collecting data.
1. Gathering Data:
Data G athering is the first step of the machine learning life cycle. T he goal of this step is to
identify and obtain all data-related problems.
In this step , w e need to identify the different data sources, as data can be collected from
various so u rce s such as files, d atab ase , in tern et, or m obile d ev ice s. It is one of the most
important step s of the life cycle. The quantity and quality of the collected data will determine
the efficiency o f the output. The more will be the data, the more accu rate will be the prediction.
This step includes the below tasks:
o Identify various data sources
o Collect data
o Integrate the data obtained from different sources
By perform ing the above task, we get a coherent set of data, also called as a dataset, It will be
used in fu rth e r steps.
2. Data Preparation
After collectin g the data, we need to prepare it for further steps. Data preparation is a step
where we p u t our data into a suitable place and prepare it to use in our machine learning
training.
In this step , first, we put all data together, and then randomize the ordering of data
This step ca n be further divided into two processes:
o DataExploration: It is used to understand the nature o f data th at we have to work
with. W e need to understand the characteristics, format, and quality of data. A better
 Machine Learning - ^ ■

understanding of data leads to an effective outcome. In this, we find Correlations, general -

trends, and outliers.

o D a o Pre-processing; Now the next step is pre-processing of data for its analysis.

3. D ata Wrangling
Data wrangling is the process o f cleaning and converting raw data into a useable

important steps of the com plete process. Cleaning of data is required to address qu ty

issu 0s
It is not necessary that data w e have collected is always o f our use as some of the d ata may not
be usefiil. In real-world applications, collected data may have various issues, including.

o Missing Values
o Duplicate data
o Invalid data
o Noise
So, we use various filtering techniques to dean the data.
It is mandatory to detect and remove the above issues because it can negatively affect the

quality of the outcome.

N ow ftrc ta n e d and prepared data is passed on to the analysis step This step involves:

o Selection of analytical techniques

o Building models
o Review the result
The aim of this step is to build a machine learning model to analyze the d ata usmg
L r i o u r a n i* ic a l techniques and review the outcom e. It starts with the determ m anon
7 Z ^ e o f the problem s, where we select the machine leammg tech niques su h
as Classification, R egression, Cluster analysis. A ssociation, etc. then build the m odel usmg
prepared data, and evaluate the model.
Hence, in this step, we take th e data and use machine learning algorithms to build th e model.

N o w "th e °tL step is to train the model, in this step we train our model to improve its
performance for better outcom e of the problem.
We use datasets to train th e model using various m achine learning algorithm s. Training a
m L e U s required so that it can understand the various patterns, rules, and, features.

T c e ^ w machine learning model has been trained on a

In this step, we check for th e accuracy of our model by providing a test d ataset
Testing the model d eterm ines the percentage accuracy o f the model as per the requirem ent of

project or problem.
i Fundanieiitalj«#lto 1.39

7 . D eploym ent
The last step of machine learn in g life cycle is deplo 5m ient, where we deploy the m odel in the
real-world system.
If the above-prepared m odel is producing an accurate result as per our req u irem en t with
acceptable speed, then we deploy the model in the real system. But before deploying the
project, we will check w h eth er it is improving its perform ance using available data o r not. The
deployment phase is sim ilar to making the final report for a project.

Main Challenges of M achine Learning

Machine learning faces several challenges that can im pact the performance and reliab ility of
models. Each of these challenges plays a critical role in the su ccess of machine learning applications
and requires careful consideration and mitigation strategies to ensure accurate pred ictions and
generalization to new data.
1. In su fficien t Quantity o f T r a in in g D a ta :
Machine learning model generally require large am ounts of data to perform w ell. With
insufficient training data, m od els have a limited ability to learn, leading to poor perform ance
on unseen data. Insufficient training data can hinder the ability of machine learning m odels to
learn complex patterns and m ake accurate predictions.
Exam ple: Consider a facial recognition system designed to identify individuals at an event. If
the training data only includes a few images per person, the model might not learn to generalize
well, leading to inaccurate identification. *
2 . N onrepresentative T ra in in g D a t a :
Nonrepresentative training d ata means that the data used to teach a machine learning model
does not show a complete pictu re of what the model will face in the real world. This can lead
to the model making m istakes o r having biases because it hasn't learned from a w ide enough
range of examples. To avoid this, it's important to train the model on a diverse set o f d ata that
covers all possible scenarios it might encounter. This way, the model can learn m ore effectively
and make better predictions w hen faced with new, unseen data.
Exam ple: Consider a scen ario where a machine learning model is being trained to classify
different types of animals b ased on their features. If the training dataset only includes images
o f dogs and cats but lacks im ages o f birds and fish, the model may struggle to accurately classify
these missing animal types w hen presented with them in real-world scenarios. This lack of
representation in the training data can lead to the model making errors or show ing biases
towards the animals it was train ed on, resulting in unreliable predictions. To ensure th e model
can effectively classify a w ide range of animals, it is crucial to train it on a diverse d ataset that
includes various animal sp ecies with different characteristics and features.
3. Poor-Quality D a ta :
Poor-quality data Including m issing values, errors, o r inconsistencies can introduce noise and
biases into machine learning models. It affects the model performance and reliability. Data
cleaning and preprocessing a re crucial to address issues related to poor-quality data.
M a e h in e ^ rn in g : ■ .r., ‘

customer satisfaction through personahzed recommendations.

housing prices based on various features such f o f th e house or the

for buyers and sellers in the real estate market.

instead of learning the true relationships within the data.

We can tackle this issue by:

Analyzing the d ata with the utmost level o f perfection
o Use data augm entation technique
o Remove outliers in the training set
o Select a model w ith lesser features

due to its focus on irrelevant details from the training data.

 data. In essence, the m odel is not able to learn enough from the training exam ples to make
accurate predictions.
To overcome this issue:
o Maximize the training time
o Enhance the com plexity of the model
o Add more features to the data
o Reduce regular param eters
o Increasing the training time of model
Example: Let's con sid er a scenario where a linear model is used to predict housing prices
based solely on the nu m ber of bedrooms in a house. If the model is underfitting, it may
oversimplify the relationship between the num ber o f bedrooms and the price, assum ing that
all houses with the sam e number of bedrooms have identical values. This sim plistic approach
overlooks other crucial factors influencing housing prices, such as location, square feet, and
amenities.

Why Python?
P )^ o n is used in numerous d ata science and machine learning applications due to its versatility and
user-fnendly nature. It seam lessly combines the robust capabilities of general-purpose programming
languages with the sim plicity o f domain-specific scripting languages like MATLAB or R. W ith a rich
ecosystem of libraries caterin g to data loading, visualization, statistics, natural language processing,
image processing, and m ore. Python equips data scientists with a diverse toolkit encom passing both
general and specialized functionalities.
The flexibility of Python extend s to its interactive nature, allowing users to engage directly with the
code through interfaces like term inals or popular tools such as Jupyter Notebook. This interactive
capability is particularly advantageous in the iterative nature of machine learning and d ata analysis,
where insights are derived from the data itself.
Python has emerged as the preferred language for m achine learning due to its exceptional qualities.
P}^hon is renowned for its simplicity, readability, and extensive library ecosystem tailored for data
science and machine learning tasks. Libraries like NumPy, Pandas, SciWt-learn, TensorFlow, and
PyTorch provide powerful to o ls for data manipulation, visualization, and model building.

Why Python is Preferred Choice for Machine Learning Applications ?

Python is the preferred language for machine learning due to several key reasons.
• Rich Ecosystem o f Libraries: Python includes a vast array of libraries and frameworlcs specifically
created for machine learning and data science tasks. Libraries like NumPy, Pandas, Scikit-learn,
TensorFlow, and PyTorch provide powerful tools for data manipulation, visualization, and building
machine learning m odels.
• Ease of Learning and Use: Python is popular for its simplicity and readability. Its clean S3mtax and
extensive documentation enable developers to write code efficiently, accelerating the development
process. ^ ___________________________________
f ,42

Community Support: Python has a large and active community ot developers and data sa en u s^
who contribute to open-sounre projects and offer support through forums, tutorials, and onhne
resources. This collaborative environment fosters knowledge sharing and mnovation.
Versatility: Python is a ver^tile language that extends beyond machine learning to various
domains like web development, automation, scientific computing, and more. Its flexibility makes it
a valuable skill for professionals working across diverse fields.
integration Capabilittes: Python seamlessly Integrates with other languages and tools, faciltoting
easy integration with existing systems and technologies. It can be combmed with languages like C/
C++, Java, and R for building machine learning applications.
Scalability: Python offers scalability for machine learning projects by handling of large datasets
and complex algorithms. It supports deployment on different platforms, including cloud services,
to facilitate scalable machine learning model training and deployment
. Platform Independence: Python is platform-independent, meaning that code written in Python
can run on different operating systems without modification. It provides the flexibility and ease of
deployment across various platforms.

IR ra S c ik it-le a r n

S d k lt-le a m is a widely used open-source m ach in e learning library in Python that provides a simple
and efficient tool for data analysis and modeling- It is built on NumPy, SciPy, and Matplotlib which are
popular libraries for scientific computing and d ata visualization in Python. Scikit-learn is designe o
be user-friendly, accessible to both beginners and experts, and offers a wide range of machine learning
algorithms and tools for various tasks such as classification, regression, clustering, dimensionality
reduction, and more.

What is Scikit-learn ?

Scikit-learn is a popular open-source machine learning libraiy in Python that offers a comprehensive
set of tools and algorithms for data analysis, modeling and machine learning tasks. It is built on
foundational libraries like NumPy, SciPy. and Matplodib. Scikit-learn provides a user-fiiendly and
efficient framework for both beginners and experts in thejeld o fd ata^ aen ce^ ^ ^

|j| | j j |Features Scikit-learn

The key features and components of S cik it-leam are;
1. Sim ple and C onsistent API: Scikit-learn features a straightforward and consistent API that
simplifies the process of implementing machine learning algorithm s. This umform interface
allows users to seamlessly switch b e tw e e n different models and techniques without the need
for extensive code modifications.
2. D iverse A lgorithm s: The library o ffers a diverse collection of supervised and unsupervised
team ing algorithms induding su p p o rt vector machines, decision trees, _^random forests
k-means dustering, and more. The huge collecHon of algorithms helps in choosing the m ost
suitable algorithm for machine learning tasks.
 .^hndamentals of Machine Learning

3. Model E valu ation and Selection: S cikit-learn provides robust tools for model evaluation,
parameter tuning, and selection. Techniques such as cross-validation, grid search, and
performance m etrics like accuracy, p recision, recall, and F I score help users assess and
optimize the performance of their m achine learning models.
4. P rep rocessin g and Feature Engineering: The library includes utilities for data preprocessing,
feature scaling, feature selection, and transformation. These capabilities enable users to
prepare and clean their data effectively b e fo re training machine learning models to improve
model perform ance and generalization.
5. Integration w ith NumPy and Pandas: Scikit-learn seamlessly integrates with NumPy arrays
and Pandas DataFrames to facilitate d ata manipulation and com patibility with other data
science tools in the Python ecosystem. T h is integration stream lines the workflow for data
preprocessing and model building.
6. Extensive Documentation and C om m u n ity Support: Scikit-learn offers detailed
documentation, tutorials, and examples to assist users in understanding and utilizing machine
learning algorithm s effectively. The lib ra ry benefits from a strong community of users and
contributors who provide support, sh a re knowledge, and contribute to its development,
enhancing its usability and reliability.
7. Scalability a n d Perform ance: While prim arily designed for small to medium-sized datasets,
Scikit-learn offers scalability through integration with parallel processing libraries like Dask
and joblib. T his scalability feature allow s users to handle larger datasets efficiently and
leverage distributed computing resources w hen necessary

J J J j I n s t a i r i n g scikit-learn
To install scikit-leam on windows follow the ste p s given below:

Prerequisites
Python: Ensure P)^hon is installed on th e system. Scikit-learn is com patible with Python 3.6
or higher.
pip: Ensure pip is installed, which is the package installer for Python.

Steps to Install Scikit-learn

Step 1: Open a Term inal or Command P ro m p t:

Press th e Windows key and the R key simultaneously. This will open the "Run" dialog box.
In the "Run" dialog box, type "cm d" (w ithout quotes) and press Enter or click OK.
Type th e following commands to c h e ck whether P)rthon is installed or not
 If Python is not installed, th e Install python lh.m [Link]

Step 2: U pdate pip (op tion al):

before installing any packages by running follow ing
Update pip to the latest v e rsio n
command.
[Link] -m pip install -upgrade pip
In sta ll sd W t-le a rn U b ra ry
Step 3:
sclkit-leam can be installed using pip. We can run the following command m com m and
prom pt to install scik it-leam .

downloading and installing packages related to the scikit-

Step 4 : V e r ify Installation of scikit-leam Libraiy d

After installation verify that Scikit-leam is installed correctly by importmg it m a Python
a ^ o n IDLE or a Jupyter N otebook and n.n the below com m ands.

import sklearn
print(sklearn. ^version— )

^ IDLE Shell 3.1T.1

File Edit Shell Debug Options
Python 3 . 1 1 .1 { t a g 3 / v 3 . 1 1 . 1 - a 7 a 4 S 0 f , Dec .6 2022,
1 9 :5 8 1 3 9 ) [ B S C v A 9 3 4 64 b i t (M D 64) 1 on Win32 ^
Type - h e l p ', » c c ^ y r i g h c - , ^ c r e d i t s ” o r -li c e n s e ()
f o r m o re. I n f o n a a t i o n .

> » ' a k le a m
p r i i t '( 3 k l e a t n * ^ v e r 3 i o n ^ ) .
» >

> »
 Fu ndam entals of Learning ^ 1.4 5

Scikit-leam has dependencies on several other Python Ubraries, which are usually installed
automatically when installing Scikit-learn using pip. Some key dependencies include:
• N um Py: Fundamental package for scientific computing with Python.
• S ciP y : Library for mathematics, science, and engineering.
• jo b lib : Library for ligjitweight pipelining in Python.
• th re a d p o o lctl: Library for controlling the number of threads used by native libraries.

E s s e n t i a l L ib r a rie s a n d T o o ls
Understanding scikit-leam is im portant for machine learning applications. However, the additional
libraries such as NumPy, SciPy, pandas, and matplotlib enhance the overall experience. The Jupyter
Notebook, an interactive programming environment, is introduced for improved workflow. Proficiency
in these tools is essential for maxim izing the benefits of scikit-learn in real-world scenarios.
Understanding and utilizing these tools can significantly enhance the workflow and productivity of
data scientists and machine learning practitioners.

What are the Essential Libraries and Tools required for Machine Learning Projects ?
? l
Essential libraries and tools for effective implementation of machine learning projects is very important.
Understanding and utilizing these tools can significandy enhance the workflow and productivity of data
scientists and machine learning practitioners. ^
1. scikit-leam : A widely-used machine learning libraiy in Python that provides a simple and efficient
tool for data analysis and modeling.
2. NumPy: Fundamental package for scientific computing in P3rthon, providing support for large,
multi-dimensional arrays and matrices.
3. SciPy: SciPy is a library for mathematics, science, and engineering. It offers modules for optimization,
integration, interpoladon, and more. It is built on NumPy.
4 . pandas: Data manipulation and analysis library that offers data structures and functions to
efficiently work with structured data.
5. matplotlib: It is a libraiy for creating static, animated, and interactive visualizations In Python,
essential for data visualization tasks.
6. Jupyter Notebook: It is interactive web-based tool for creating and sharing documents that contain
live code, equations, visualizations, and narrative text. The Jupyter Notebook is an interactive
environment for running code in the browser. _________________ __

J u p y t e r ffateb o o k ~
Jupyter Notebook is an open-source w eb application that allows you to create and share documents
that contain live code, equations, visualizations, and narrative text. It is widely used in data science,
machine learning, scientific computing, and educational purposes.
01 .j^chiije^rning . •

Key Features

1. Interactive Coding: Jupyter Notebook supports over 40 programming languages, including

Python, R, Julia, and Scala. Code can be w ritten in cells and executed interactively, which allows
for immediate feedback and iterative data exploration.
2. Rich Media Su p p ort: In addition to code cells, Jupyter Notebooks support integrating rich
media such as images, videos, HTML, LaTeX, and m ore.
3. Visualization: It integrates with data visualization libraries like M atplotlib, Seaborn, and
Plotiy. This allows users to create dynamic visualizations that are essential for data analysis,
directly within the notebook.
4 . Shareability: Notebooks can be shared with o th e rs via email, Dropbox, GitHub, or the Jupyter
Notebook Viewer. This sharing capability facilitates collaboration on p rojects and education
by allowing others to see and execute the n o teb oo k documents live.
5. Conversion: Jup3^er Notebooks can be converted to a number of open standard output formats
' (like HTML, presentation slides, LaTeX, PDF, Python script, and more) through "nbconvert".
6. Extension and In teg ratio n : A large num ber o f extensions are available for Jupyter, which
extend its capabilities. It can also be integrated w ith big data tools like Apache Spark.

Common Use Cases

1. Data Cleaning an d Transform ation: Data scien tists use Jupyter Notebook for data cleaning,
transformation, and feature extraction to prepare data for statistical m odelingand visualization.
2. Numerical S im u latio n : Scientists and re search ers use it to write and run code that simulates
real-world processes or experiments.
3. Statistical M odeling and Machine L ea rn in g : It provides an interactive interface for
exploratory data analysis and model developm ent, which is central to m achine learning and
statistical modeling.
4. Educational P u rp o se s: Educators use it to cre ate and share documents that contain live
code, equations, visualizations, and explanatory text. It has become a valuable tool m teaching
coding, data science, and computational thinking.
5. Reporting and P resen tatio n : With the ability to convert notebooks to o th er formats like
HTML and PDF, Jup3n;er is used to create outputs that can serve as final p ro ject reports or
presentations.

Installation
Jupyter Notebook can be installed using Python's package manager pip:

pip i n s t a l l notebook
• After installation, it can be started with the com m and;
jupyter notebook
This command launches a local web server and opens a notebook interface in the default web
browser
I Fundanienials o f M a ch in e Lc

NiimPy

NumPy (Numerical Python) is a fundamental package for scientific

computing in Python. It provides support for large, multi-dimensional
arrays and matrices, along with a collection of mathematical functions
to operate on these arrays efficiently.

NumPy is a core library used in various fields such as machine learning, data science, and scientific
research due to its powerful array manipulation capabilities.
In machine learning, NumPy arrays are used for storin g and manipulating data, serving as inputs to
machine learning algorithm s for tasks like classification, regression, and clustering.

Key Feature

1. M ultl-dim ensional Arrays: NumPy's main o b je ct is the nd arn^ , a multi-dimensional array

that allows efficien t manipulation of large datasets. These arrays can be created, indexed,
sliced, and reshaped easily. It is ideal for storin g and processing data in machine learning
algorithms.
2. M athem atical Fu nction s: NumPy provides a wide range of mathematical functions that
operate elem ent-w ise on arrays, such as [Link], [Link], [Link], [Link], [Link], etc. These
functions enable efficient computation and manipulation of numerical data in machine
learning tasks.
3. Broadcasting: NumPy's broadcasting feature allows operations to be performed on arrays
of different shapes. This feature simplifies th e implementation of mathem atical operations in
machine learning algorithms.
4. Linear A lgebra O perations: NumPy provides a rich set of linear algebra functions for matrix
operations, including matrix multiplication, m atrix inversion, eigenvalues, singular value
decomposition (SVD), and more. These operations are essential for many machine learning
algorithms such as regression, clustering, and dimensionality reduction.
5. Random N u m ber Generation: NumPy includes functions for generating random numbers
and random arrays, which are useful for tasks like data shuffling, initialization of weights in
neural networks, and creating synthetic d atasets for testing machine learning models.
6. Integration w ith O th er Libraries: NumPy seam lessly integrates with other Python libraries
commonly used in machine learning such as SciPy, pandas, scikit-learn, and matplotlib. This
interoperability allows for a smooth workflow when working on machine learning projects.

Common Use Cases

1. Data A nalysis an d Statistical O p eration s: NumPy is extensively used for performing

statistical analysis on data. It helps research ers and analysts to extract insights and make
informed decisions.
2. Machine L earn in g : In machine learning, NumPy arrays are used for storing and manipulating
data, serving as inputs to machine learning algorithm s for tasks like classification, regression,
and clustering.
 i
'-^m >
1.48 \> f"J^ eliine learning

3. Im age P ro ce ssin g and Com puter G rap h ics: NumPy arrays facilitate the storage and
manipulation o f pixel values for images. It can be used in various image processing tasks, such
as filtering, transformation, and visualization.
4. S cien tific Sim ulation s: Its efficient com putation capabilities make it ideal for simulations in
physics, chemistry, and engineering, w h ere large arrays of data and numerous computations
are common.

In stall Num Py : To install NumPy, type 'p ip install numpy' in the Command Prompt and
press 'Enter'. This command instructs pip to download and install the numpy package from
the P 5^ o n Package Index (PyPI).

5. Confirm th e In sta lla tio n ; After the installation process is complete, the successful installation
of NumPy can be verified. Type 'pjrthon' in the Command Prompt to open Python's interactive
mode, then type 'im port numpy a s n p ' and press 'Enter'. If no erro r message is displayed, it
confirms th at NumPy has been successfully installed. To exit the interactive mode, type 'exitQ'.

Example 1
import numpy as np Matrix Addition:
[[ 6 8]
[10 12 ]]
a = [Link]([[l, 2 ], [3 , 4]])
Matrix Subtraction:
b = [Link]([[5, 6], [7, 8]])
[[-4 -4]
[-4 -4]]
print("Matrix Addition:\n", a + b) Matrix Multiplication (element­
printC'Matrix Subtraction:\n", a - b) wise) :
print("Matrix Multiplication (ele«ent-wise) :\n ", a* b) [[ 5 12]
printC'Matrix Multiplication:\n", [Link](a, b)) [2132]]
print("Tnanspose of a:\n", [Link](a)) Matrix Multiplication:
[[19 22]
[43 50]]
Transpose of a:
[[1 3]
[2 4]]
 . Fundamentals of Machine I

Explanation
In the above example, the various matrix operations are performed using the NumPy library.
Specifically, the matrices 'a' and 'b' are added and subtracted element-wise, yielding the sum and
difference respectively.
The operation a * b performs element-wise multiplication.
The [Link](a, b) function carries out matrix multiplication by following the rules of linear algebra.
The [Link](a) function flips the matrix 'a' over its diagonal, effectively swapping its row and
column indices, to create a new matrix.

Example 2 Statistic^ PuncJtions on Arrays

import numpy as np
Standard Deviation:1.4142135623730951
a = [Link]([l, 1 , 3, 4, 5]) Variance: 2.0
Median: 3.0
printC'Standard Deviation:", [Link](a)) Percentile: 3.0
printC'Variance:", [Link](a))
print("Median:", [Link](a))
printC'Percentile:", [Link](a, 50)) #50th percentile also known as aedian

Explanation
In the above example, multiple statistical measures are calculated on the array 'a' using NumPy’s built-
in functions.
The standard deviation of the array, represented by [Link](a), measures the amount of variation or
dispersion of the set of values.
The variance of the array, calculated by [Link](a) is another measure of dispersion, which is essentially
the square of the standard deviation.
The median found using [Link](a) is the middle value in the sorted list of numbers that separates
the higher half from the lower half of the data set
The percentile is computed with [Link](a, 50), which represents the value below which a given
percentage of the data falls. In this case, the 50th percentile is calculated, which is also known as the
median.

S ciP y

SciPy is an open-source Python library th at is used for scientific and technical computing. It builds on
top of NumPy and provides a wide range of functions for numerical integration, optimization, signal
processing, linear algebra, statistics, and much more. SciPy is a powerful tool for scientific computing
and is widely used in various fields, including machine learning, physics, engineering, and biology
1 Inteeration and Optimization: SciPy Includes functions for numerical integration
' in ter^ lation , and opamizatlon. These capabilities are essential for sol«ng optimization
problems in machine learning, such as parameter tuning in algorithms like support
m achines CSVM) o r n e u r a l n e t w o r k s .

2 Signal Processing: SciPy offers tools for signal processing tasks like filtering, spectral ana^sis
fn^w a^form generation. These hinctions are valuable forprocessingandanalyzingsignalsm

machine learning applications, such as speech recognition o r image processing.

3 LinearAlgebn.:SciPyprovidesacom prehensi»esetoffunctionsforlinearalgebraoperanons
MudingLtrixdec7mposition,eigenvalueproblems,andsolvingUnearsystemsofequations^

These operations are crucial for many machine learning algorithms that involve matr

computations. .
4 Statistics- SciPy includes statistical functions for probability distributions, hypothesis testing
a^d d e s c r i p ~ ^ These functions are useMl for d ata analysis, model evaluation, and
understanding the significance of results in machine le a rn in g experiments.

5 S p arse Matrices: SciPy supports sparse matrix representations and provides efficien
■ afgorithms for working with large, sparse datasets. /"^trices are c o m j n l y
mwhine learning for tasks like collaborative filtering text minmg and graph analysis.
6 im ag e Processing: SciPy includes modules for image processing tasks such as filtering edge
A c t i o n , and morphology. These ftmcaons are beneficial for preprocessmg image data m
machine learning applications like computer vision and object recognlOon.
7 interoperability with N um Py: SciPy seamlessly integrates with NumPy, making it “
■ combinethearraymanipulationcapabiilttesofNumPywiththeadvancedsciennficcomput^^^^
functions of SciPy This integration enhances the efficiency and productivity
learning workflows.

1. Scientific Analysis: For task s that require precise calculations and date
physics and chemistry, SciPy provides robust algorithm s th a t are dependable and efficien t
2. Engineering Applications: Many engineering disciplines use SciPy for simulating real-world
processes, optimizing system s, and analyzing data.
3. Academic Research: Researchers in fields like econom ics, sociology, and psychology utilize
SciPy’s statistical tools to analyze experimental data.
4 . Im age Processing: SciPy's sub-package ndimage supports tasks in multi-dimensional image
processing widely used in fields such as medical image analysis and computer vision.

P r e r e q u is ite s : SciPy requires Python and NumPy. Before installing SciPy, ensure that Python and

NumPy are installed.

 Fundamentals of Machine Learning 'd 1.51

Install SciPy: We can install SciPy using pip. Python’s package manager. To install SciPy, open
command prompt or term inal and type:
pip install scipy
Verify Installation: To confirm that SciPy has been successfully installed, launch Python in
interactive mode and try importing SciPy:
import scipy
If no error appears, the installation is successful.

Example Solving a System of Linear Equations

from [Link] import solve Solution of the system: [2. 3.]

# Define the coefficient matrix A and the constant

vector B
A = [ [ 1 , 2 ],
[3 , 4 ]]
B = [8, 18]

# Solve for x and y

solution = solve(A, B)

print("Solution of the system:", solution)

Explajaation
Consider solving the system of equations:

x+2y=8
3x+4y-18
This system can be written in matrix form as:

[1 2 x 8
3 4 y 18

terms (B).
The matrix A represents the coefficients of the variables in the system of linear equations, and vector
B is the right-hand side of these equations.
The solve function computes the exact solution to the linear equations, offering a direct method
to handle such problems without manually implementing Gaussian elimination or other solving
techniques.
The script will output: Solution of the system: [2. 3.]
This means x=2 and y=3 are the solutions to the system of equations.______________________________
 ’I*

■ I
Pandas

s r ; r. ; ™ r r : i r i C i p . :

Pandas i
transform ation, exploration, and analysis.

Pandas is mainly used for data analysis. TOe

structures.

d a ts t^ i— otyforanalysisandexporting .

‘ = = ~ = S = = S S =

• ^ ” 5 H = .S ~ = =

• i-3 S = 3 = r 2 H = r ~

•= = = = = — = = = =

■ s r ~ 3 - = s s s =
 - i ‘r:FunaamentaIs of Machine LMrhing'/^
m m k

1. Data Cleaning: Data scientists often spend a large amount of their tim e cleaning data, and
Pandas provides powerful tools to perform this task efficiently.
2. Data E xp loration a n d Analysis: Pandas provides a high-level, flexible, and fast tool for data
analysis.
3. Data V isualization: It seamlessly integrates with the data visualization libraries such as
Matplotlib to plot data directly from data fram es.
4. Building M achine Learning Models: Before building models, data needs to be preprocessed
and transformed effectively. Pandas is often used for these tasks, ensuring that data is in the
correct form and ready for training models.
5. Tim e S e rie s A nalysis: Pandas has built-in support for handling time series data. Whether it's
resampling of tim e series data to convert frequencies, generating date ranges, or shifting and
lagging values. Pandas has tools to handle all th ese tasks effectively.

It's essential to have Python and pip (Python's package installer) pre-installed to install Pandas on a
system running Windows. The installation steps are:
1. Open Com mand P ro m p t: The Command Prom pt can be opened by searching for 'cmd' in the
Start menu and clicking on the Command Prom pt app.
2. Install P and as : To install Pandas, type 'p ip in sta ll pandas' in the Command Prompt and
press 'Enter'.
3. Confirm th e In s ta lla tio n : After the installation process is complete, the successful installation
of Pandas can be verified. Type 'python' in the Command Prompt to open Python's interactive
mode, then type 'im p o rt pandas as pd' and press 'Enter'. If no error m essage is displayed, it

Creating a'DataFrame from a Dictionary

# Importing the pandas library Name Age

import pandas as pd 0 Srikanth 45

1 Snigdha 20
# Creating a pandas DataFrame from a dictionary 2 Bhagya 40
df = [Link]({
■Name': ['Srikanth', 'Snigdha', 'Bhagya'],
•Age': [45, 20, 40],

# Printing the DataFrame to console

print(df)
 m
Explanation
* e code, the
DataFrame is created using the p . values form the rows of the DataFrame.
The keys of the dictionary form the column names and the values
Finally,theprintstatementdlsplaysthecreatedDataFra,ne. , other column

:r a T r .“ e ; "
the respective data.

Matplotlib is a popular plotting library in PythonJhat provides a

creating static, intenctive, and animated data science,

plots, charts, and graphs for data visualization and analysis m
machine learning, scientific research, and more.

1. Sim ple and Flexible: MatplotUb offers a

interactive backends for use in ppp, SVG) or embedding in web

interactive backends for saving plots to image files (e.g.,

directly and perform mathematical operation

 Fundamentals of Machine Learning

6. C u sto m izatio n and Styling: Matplotlib provides extensive customization options for styling
plots, such as setting plot colors, line styles, markers, fonts, and plot sizes. Users can create
visually appealing plots by adjusting the appearance of elem en ts to suit their preferences.
7. S u b p lo ts and Figures: Matplotlib supports the creation o f m ultiple subplots within a single
figure, allowing users to display multiple plots in a grid layout. This feature is useful for
com paring different datasets or visualizing related inform ation in a single plot window.
8. In te ra c tiv e Plotting: Matplotlib can be used in interactive m ode to create dynamic plots that
resp ond to user interactions, such as zooming, panning, and selecting data points. Interactive
p lottin g is beneficial for exploring data and gaining insights through visual exploration.

Common Use Cases

1. D ata Visualization; It is widely used for exploring and understanding data through
visualizations, especially where the data is time-series data o r ordered categories.
2. S c ie n tific Plotting: In academic and scientific publications, Matplotlib is used extensively to
cre ate high-quality plots, charts, and figures.
3. A lg o rith m Visualization: For data scientists and developers, visualizing the algorithm's
beh av ior can be crucial for diagnosing problems, and M atplotlib provides the tools necessary
to c re a te these visualizations.
4. In te ra c tiv e Applications: M atplotlib can be used to create desktop graphical user interfaces
or dynam ic dashboards for visualizing data in real-time.
5. E d u ca tio n : Its ease of use and wide range of plotting capabilities make it an excellent tool for
teach in g concepts in data science, statistics, and com putational mathematics.

Installation

The step -by-step process of installing Matplotlib using pip in the com m and prompt:
1. Open th e command prompt
2. Type in the following command to install Matplotiib:
pip install matplotlib
3. P ress the Enter key to execute the command. This will s ta rt the installation process for
M atplotlib.
4. W ait fo r the installation process to complete. We should see a message indicating that
M atplotlib has been successfully installed.
5. To verify the installation, we can type in the following com m and:
pip show matplotlib
6. This should display information about the installed version o f Matplotlib, including its location
and version number.
7. We can now start using Matplotlib in a Python projects. To use Matplotlib in a code, we need to
im p o rt the library using the following line of code:
import [Link] as pit
 MacKine Uarniiig'.'

Example Plotting a Simple Line Chart

import [Link] as pit

# Prepare the data

Simple Line Graph
X = [ 1 , 2 , 3, 4 , 5]
y = [1, 4, 9, 16, 25]

# Plot the line graph

[Link](x, y)

# Add title and axis labels

[Link]("Simple Line Graph")
[Link]("Number")
[Link]("Square")

# Show the plot

[Link]

# Save the plot

[Link]("simple_line_graph.png")
Explanation
This code will create a simple line chart with the x-axis labeled "Number" and the y-axis labeled "Square",
and with a title of "Simple Line Graph". The line chart will be displayed in a new window, and will be saved
to a file named "simple_line_graph.png". _____ ________________________________________________

R e v ie w Q u estio n s

1. Define Learning. Give an example.

2. W hat is Human Learning? Give an example.
3. W hat is Machine Learning? Give an example.
4. W hat is Machine Learning Problem ? Give an example.
5. W hat is Supervised Machine Learning? Give an example.
6. Mention the Real Life Examples of Machine Learning.
7. Mention the T3q)es of Supervised Machine Learning.
8. W hat is Unsupervised Machine Learning? Give an example.
9. Mention the different types o f^ a ch in e Learning.
10. Mention any 4 Features of Machine Learning.
11. W hat is Semi-Supervised Machine Learning ? Give an example.
12. Define Regression.
 Fundamentals of Machine learning

13. W liat is Classification?

• 14. W hat is Clustering? Mention Clustering Algorithms.
15. Define Association. Mention Association Rule Learning Algorithms.
16. W rite any two applications of Supervised Machine Learning.
17. W rite any two applications of Unsupervised Machine Learning.
18. W hat is Reinforcement Learning? Give an example.
19. W hat is Scikit-learn ?
20. W h atisN u m P y?
21. W hat is Pandas?
22. W hat is Jupyter Notebook?
23. W h atisS ciP y ?
24. W hat is Matplotlib?
25. Why p)^hon is used for Machine Learning
26. W hat exactly is a numpy array?
27. Mention any four main challenges in Machine Learning.

1. W rite the Goals of Machine Learning.

2. W rite the History of Machine Learning.
3. Explain the differences betw een Traditional Programming Approach Vs Machine Learning
Approach.
4. Why Use Machine Learning ?
5. W hat is Supervised Machine Learning? Explain the Key Components of Supervised Machine
Learning.
6. W rite the Applications of Supervised Learning.
7. W rite the Advantages and Disadvantages of Supervised Learning.
8. W hat is Unsupervised Machine Learning? Explain the Key Components of Unsupervised
Machine Learning.
9. W rite the Applications of Unsupervised Machine Learning.
10. W rite the Advantages and Disadvantages of Unsupervised Machine Learning.
11. W hat is Semi-Supervised Machine Learning? Explain the Key Components of Semi-Supervised
Machine Learning.
12. W rite the Applications of Semi-Supervised Machine Learning.
13. W rite the Advantages and Disadvantages of Semi-Supervised Machine Learning.
14. W hat is Reinforcement Machine Learning? Explain the Key Components of Reinforcement
Machine Learning.
 Machine Uarning :

15 Write the Applications of Reinforcement Machine Learning.

16. Write the Advantages and Disadvantages of Reinforcement Machine Learning.
17. Explain the Differences between Supervised and Unsupervised Learning.
18. Why Python is Preferred Choice for Machine Learning Applications ?
19. What is Scikit-learn ? Explain its features.
20. What are the Essential Libraries and Tools required for Machine Learning Projects ?

21. What is NumPy ? Why it is needed for ML? Explain its features.
22. What is Pandas? Why it is needed for ML? Explain its features.
23. What is Pandas? Why it is needed for ML? Explain its features.
24. What is Jupyter Notebook? Why it is needed for ML? Explain its features.
25. What is SciPy? Why it is needed for ML? Explain its features.
26. What is Matplotlib? Why it is needed for ML? Explain its features.

Eight Marks Questions

1. What is Machine Learning? Write the Features of Machine Learning.

2. Explain the Operational Mechanisms o f Machine Learning.
3. Explain the Types of Machine Learning.
4. Explain the Types of Supervised Machine Learning Algorithms
5. Explain the Types of Unsupervised M achine Learning Algorithms.
6. How Supervised Machine Learning W orks? Explain with an example.
7. How Unsupervised Machine Learning W orks? Explain with an exam ple.
8. How Semi-Supervised Machine Learning Works? Explain with an exam ple.
9. How Reinforcement Learning Works? Explain with an example.
10. Write the Applications of Machine Learning.
11. Explain the Machine learning Life Cycle.
12 Explain the Main Challenges of Machine Learning.
13. Explain the Essential Libraries and Tools required for Machine Learning Projects. How to
install the required libraries?
 DATA JPI®PARATIO)M

' !■v:»;&

Introduction
Data Preparation in Machine Learning
Working w ith Real Data
Look at the Big Picture
Get the Data
Discover and Visualize the Data to Gain Insights
Prepare the Data for Machine Learning Algorithms

Select and Train a Model

Review Q uestions
 Machine Learning
O k

M I B Introduction i onino
Data preparation Is a fundamental s t e p T h e quality and
transforming, and organizing raw data im oact the performance and accuracy of ML

r r r r j j ™ -

; r s : . : r x r r r : : ' = ' . r - . - « . _ -
machine learning model produces more reliable and accurate [Link] .

i M H n u ^ n g o f P g it^ l^ M ^ in e L e a rn i^ . ^ ^,

In general context, D ata refers to raw facts, statistics. ta tg ^ o ^ r ^ m u lt o d ia . Data

and analyzed. It can exist in various nroblem-solving in various fields, including
is fundamental for decision-making, analysis, and problemsolving
technology, business, science, statistics, and more.

Meaning of Data in Machine Learning

£
i t i r ™ r d i ,T " i " ,t d ^
predictions, or perform t a s k s w i t t ^ ^ ------- “

Data is a crucial com ponent in the quantity of

or measurements th at can be used to m nt rnle in determining the performance of a
data available for training and testing play a c a te g o r ic a l, or time-series
machine-learning model. Data can be d e b a s e s spreadsheet^, o r APIs. Machine learning

which can then be used for prediction or classification tasks.

— i r.5 ^ tA > ro r ie s ^ n 5 a ta in M a c h in e L e a r n i n g

evaluation (training data, validation data and testing data)

Labeled Data and Unlabeled Data

inmachine,earning [Link],includ,ngwhetherthe

data is labeled or unlabeled.

 1. Labeled Data: Labeled data is data th at has both input features and corresponding output
labels or targ et variables. In supervised learning, labeled data is used to train a model to make
predictions by learning the relationship betw een input features and output labels.
Labeled data is tj^ically used for training supervised learning models, where the model learns
from the input features and corresponding output labels. This labeled training data is then
used to make predictions on new, unseen data.
Example: In a dataset of images with labels indicating whether each image contains a cat or a
dog, the im ages along with their corresponding labels (cat or dog) are considered labeled data.
2. Unlabeled Data: Unlabeled data is data th at consists of input features without corresponding
output labels. Unlabeled data is commonly used in unsupervised learning tasks where the goal
is to discover patterns, relationships, or structures within the data without explicit labels.
Unlabeled data can be used in various ways, such as for unsupervised learning tasks like
clustering, dimensionality reduction, o r anom aly detection. Unlabeled data can also be used in
sem i-supervised learning, where a model is trained on a combination o f labeled and unlabeled
data to improve performance.
Example: A d ataset containing custom er demographic information [e.g., age, income, location)
without any specific target variable or label is considered unlabeled data.

Training Data, Testing Data, and Validation Data in Machine Learning

In machine learning, datasets are typically divided into three main subsets for model development
and evaluation: training data, testing data, and validation data.

DATA in Machine^
Learning ,

I^Training Data^ Validation D a^ ^Testing Data ^

1. Training Data:
• Definition: Training data is the initial dataset used to train a Machine Learning model,
com prising input features and corresponding target labels or outcomes.
• Example: In a housing price prediction project, a dataset of 1,000 houses with features
like square footage, number of bedroom s, and location, along with their actual selling
prices, serves as the training data.
• Explanation: The model learns from the training data by analyzing the relationships
betw een the input features (square footage, bedrooms, location) and the target labels
(selling prices). Through processes like gradient descent, the model adjusts its internal
param eters to minimize prediction errors and enhance accuracy.
2. Validation Data:
• Definition: VaUdation data is a separate dataset utilized to fine-tune the model's
hyperparam eters and evaluate its performance during training.
 Machine l^irning . V?- ’ "

• Exam ple: For the housing price prediction task, a su b set of 200 houses with sim ilar
features but different p rices is designated as the validation set.
• Explanation: During training, the model's perform ance with various hyperparam eter
configurations (e.g., learning rate adjustments) is assessed using the validation set.
By comparing the m odel's performance on the validation data for each configuration,
optimal hyperparameters are selected to improve the m odel's predictive accuracy.

3. T estin g Data:
• D efinition: Testing data is a dataset employed to assess the model's perform ance and
generalization capabilities on unseen data.
• Exam ple: Following training and validation, a fresh d ataset of 300 houses, com pletely
new to the model with undisclosed selling prices, is reserved for testing. f
• Explanation: The model is evaluated on the testing s e t to gauge its ability to predict
selling prices of unseen houses. Testing data provides an unbiased evaluation o f the
model's performance in real-world scenarios, indicating its capacity to generalize
accurately to new instances.

H ow do w e split d ata in M a c h in e Learning ? ^

• The training data teaches the model to recognize patterns, the validation data helps fine-tune the
model's settings, and the testing data evaluates the model's performance on unseen instances. Each
type of data serves a specific purpose in the Machine Learning workflow, ensuring that the model is
trained effectively, optimized for performance, and capable of making accurate predictions on new
data.

Data Preparation in M achine Learning

Data preparation is a critical step in the machine learning pipeline that involves processing and
transform ing raw data into a form at suitable for building and training machine learning models.
This process ensures that the data is clean, relevant, and structured in a way that optim izes the
perform ance of machine learning algorithm s.
Each machine learning project requires a specific data format. To do so, datasets need to be prepared
well before applying it to the projects. Sometimes, data in data sets have missing or incom plete
information, which leads to less accurate or incorrect predictions. Further, sometimes data sets are
clean but not adequately shaped, such as aggregated or pivoted, and some have less business context.
Hence, after collecting data from various data sources, data preparation needs to transform raw data.

W h at is Data Preparation ? ^

• Data preparation is defined as a gathering, combining, cleaning, and transforming raw data to
make accurate predictions in Machine learning projects. It is the later stage of the machine learning
lifecycle, which comes after data collection. ________
i ■‘Ottta Preparation 2.5

I m p o r ta n c e a n d B e n e f i t s o f D a ta P r e p a r a t i o n

Data preparation is important for the below reasons.

1. Data preparation ensures reliable prediction outcomes in analytics operations.
2. It helps identify data issues or errors, significantly reducing the chances of errors.
3. P roper data preparation increases decision-making capability.
4. Effective data preparation reduces overall project costs, including data m anagem ent and
analytics.
5. It removes duplicate content, making data more valuable for different applications.
6. Data preparation enhances model performance by optimizing input data quality.

P r e p a r a t io n I s s u e s in M a c h in e L e a r n i n g

Various issues have been reported during the data preparation step in machine learning as follows:
> M issin g data: Missing data o r incomplete records is a prevalent issue found in m ost datasets.
Instead of appropriate data, sometimes records contain empty cells, values [e.g., NULL or
N/A), or a specific character, such as a question mark, etc.
> O u tliers or Anom alies: ML algorithms are sensitive to the range and distribution o f values
w hen data comes from unknown sources. These values can spoil the entire machine learning
training system and the perform ance of the model. Hence, it is essential to detect these outliers
o r anomalies through techniques such as visualization technique.
> U nstru ctu red Data F o rm at: Data comes from various soiirces and needs to be extracted into
a different format Hence, before deploying an ML project, always consult with domain experts
o r import data from known sources.
> L im ited Features: W henever data comes from a single source, it contains limited features, so
it is necessary to import data from various sources for feature enrichment or build multiple
features in datasets.
> U nd erstanding Featu re E n gin eerin g : Features engineering helps develop additional content
in the ML models, increasing model performance and accuracy in predictions.

S t e p s in D a ta P r e p a r a t i o n P r o c e s s

Data Preparation is a crucial step in the Machine Learning process, and it involves various key steps to
ensure the data is suitable for training models effectively. The key steps involved in data preparation
are listed below:
1. D ata Collection: The initial step involves gathering raw data from a variety of sources such
as databases, file systems, sensors, or external APIs. This process lays the foundation for
subsequent data processing and analysis.
E xam p le: Gathering custom er information from a GRM system (such as customer IDs, purchase
history), transaction data from a database (including order amounts, timestamps), and social
media interactions from an API (like customer engagem ent metrics, comments) to analyze
custom er behavior for a targeted marketing campaign.
 Ml

2 . Data Cleaning: Data cleaning is a fundamental step in ± e data preparation process that
involves identifying and rectifying errors, inconsistencies, and missing values in the datasetto
ensure its quality and reliability for subsequent analysis and modeling.
> Handling M issin g Values: Dealing with m issing data by either filling them with
appropriate values (e.g., mean, median, mode) or removing them to prevent inaccuraaes
in the model.
Example ; If som e customer records have m issing age and income columns. Replace
missing values in the "Age" column with the m ean age o f the avaiUble data and missmg
values in the "Incom e" column with the mean income.
> Filtering O u tliers: Identifying and removing anomalies that significantly deviate from
the rest of the data using statistical methods like Z-scores or IQR.
Example: Identifying and removing outliers in the customer age field, such as entries
with ages over 1 0 0 years, to ensure the data's accuracy.
3. Data Transformation: Data transformation involves converting and standardizing the dataset
to make it more suitable for Machine Learning algorithm s by ensuring consistency, reducing
redundancy, and improving interpretability.
> Normalization and Standardization: Scaling data to a standard range (e.g., 0 to
1 for norm alization) or transforming data to have zero mean and unit variance
(standardization) to ensure consistency, especially for models sensitive to feature scales.
. Think of normalization as adjusting values to fit within a specific range, like
resizing a photo to fit a frame. It ensures all data points are on a sim ilar scale,
preventing any one type of data from overshadowing others.
. Standardization is like making data follow a standard pattern, such as ensuring all
ingredients in a recipe are measured in the same units. Ithelps m odels understand
and com pare different types of data m ore easily
Example : Scaling customer purchase amounts to a standard range (e.g., 0 to 1) to ensure
consistency in the analysis, especially when comparing with other features like customer
engagement levels.
> Encoding C ateg orical Variables: Converting categorical variables into a form at suitable
for ML algorithms, such as one-hot encoding or label encoding.
. O ne-H ot Encoding: Imagine creating a list of checkboxes for different categories,
where each checkbox is either ticked (1 ) or unticked (0). This method helps the
model understand and use categorical data effectively.
. L abel Encod ing: Think of assigning a unique number to each category, like giving
each type o f fh iit a specific code. This simplifies the data for the model to process,
making it easier to work with different categories.
Example: Converting categorical data like customer segmentation (e.g., premium,
standard, basic) into numerical values using one-hot encoding for model compatibility.
4 Data Reduction: Data reduction techniques aim to simplify and con d en ^ the dataset while
retaining its essential information, making it m ore manageable and efficient for Machine
Il»
Data Preparation

Learning algorithms to process. These techniques help in reducing computational complexity,

improving model performance, and enhancing interpretability.
> D im ensionality Reduction: Reducing the number of variables by extracting principal
components using techniques like Principal Component Analysis (PCA) or t-Distributed
Stochastic Neighbor Embedding (t-SNE).
Example : Imagine using PCA to condense customer demographic features like age,
income, and location into a few key components that capture the m ost important
patterns in the data. By doing this, we simplify the dataset while retaining the essential
information needed to understand custom er behavior and preferences effectively.
> Feature S e le c tio n : Feature selection involves choosing the most relevant features from
the dataset while discarding irrelevant or redundant ones. The goal is to improve model
performance by focusing on the most informative attributes.
Selecting the right features can help reduce overfitting, improve model interpretability,
and enhance prediction accuracy by elim inating noise and irrelevant information.
Feature selection techniques include statistical tests, correlation analysis, model-based
selection, and recursive feature elimination. These methods help identify the subset of
features that contribute most to the predictive power of the model.
Exam p le: Think o f choosing critical custom er attributes such as purchase frequency and
average order value based on their significance in predicting how custom ers respond to
marketing campaigns. By focusing on these key attributes, we can focus on marketing
strategies m ore effectively to target specific customer segments and improve campaign
outcomes.
5. Feature E n g in eerin g : Feature engineering is the process of creating new features or
transforming existing features in a dataset to improve the performance of machine learning
models. It involves selecting, modifying, or creating relevant features that help the model
better understand the underlying patterns in the data. Feature engineering plays a crucial role
in enhancing the predictive power of machine learning algorithms by providing them with
more informative input variables.
> Feature T ran sfo rm atio n : Feature transform ation involves converting existing features
into a more suitable format for the model. This process can include scaling numerical
values to a com m on range, encoding categorical variables into numerical representations,
or appl}dng m athematical transformations to the data.
Transforming features ensures that the data is in a format tiiat the model can effectively
learn from. Standardizing or normalizing numerical features, for example, can prevent
bias towards variables with larger scales.
Common techniques for feature transform ation include Min-Max scaling. Standardization,
One-Hot Encoding for categorical variables, and log transformations for skewed data.
Example : Suppose the dataset includes a categorical feature "Region" representing
different geographical regions where custom ers reside. To transform this categorical
variable into a numerical format that the model can interpret, we can use One-Hot
^ 7 ^ 1 ^ Machine l^ rning .. ,. '

Encoding. This technique creates binary columns for each category within the "Region" .
feature, assigning a value o f 1 if the customer belongs to th at region and 0 otherwise. By
encoding categorical variables in this manner, we ensure th at the model can effectively
utilize this information for making predictions w ithout introducing any ordinal
relationship between the regions.
> F e a tu re Creation: Feature creation involves generating new features by combining
existing ones or extracting additional information from the data. This process aims to
provide the model with more relevant and informative input variables.
Creating new features can capture complex relationships in the data that the original
features may not fully represent. It can lead to improved model performance and better
generalization to unseen data.
Feature creation techniques include polynomial features, interaction terms, domain-
specific feature engineering, and text or image feature extraction. These methods
help enrich the dataset with new information that can enhance the model's predictive
capabilities.
Exam p le : Consider com bining customer attributes like purchase history, browsing
behavior, and demographic information to create a new feature representing overall
custom er engagement This feature creation process helps in capturing complex
relationships within the data and providing a more com prehensive view of customer
interactions, enabling better insights for marketing strategies and customer segmentation.
6. D ata Sp ittin g : Data splitting in m achine learning is the process o f dividing the data into
separate subsets to be used at different stages of model building and evaluation. The primary
goal o f data splitting is to ensure that the model trained on one set o f data can generalize well
to new, unseen data. This helps avoid problems like overfitting, w here a model performs well
on the training data but poorly on new data.
Com m on Types of Data Splits
> T rain in g Data: This is the largest portion of the dataset and is used to train the model.
The model learns to identify patterns and make decisions based on this data. Typically,
about 70-80% of the entire d ataset is allocated to the training set.
> V alidation Data: This subset is used to tune the model's hyperparameters and make
decisions about which models or configurations to use. It acts as a check to avoid
overfitting on the training d ataset. Typically, about 1 0 -1 5 % o f the dataset is reserved for
validation.
> T e st Data: This is used to evaluate the final model's perform ance after it has been trained
and validated. The test set should be a completely independent dataset that the model
has not seen during training o r validation. This helps provide an unbiased evaluation o f
how well the model is expected to perform in the real world. Typically, about 10-15% of
the dataset is used as the te st set.
 Data Preparatio

What is Data Collection ? Explain the Key Steps Involved in Data Collection.
%
Data collection is the process of gathering relevant information o r data from various sources
to be used for analysis, decision-making, or research purposes. It is a crucial step in any data-
driven p ro ject as the quality and accuracy of the collected data directly impact the outcomes of
subsequent analyses and modeling.
The Key Steps involved in data collection :
1. Define Objectives: Clearly define the objectives and goals o f the data collection process.
Understand what specific inform ation is needed and how it will be used to achieve the
desired outcomes.
2. Identify Data Sources: D etermine the sources from which the data will be collected.
Sources can include databases, surveys, sensors, web scraping, social media, public
records, etc.
3. Design Data Collection M ethods: Choose appropriate methods for collecting data
based on the objectives and sources. Common methods include surveys, interviews,
observations, experiments, and automated data collection tools.
4 . Develop Data Collection Tools: Create tools such as questionnaires, forms, sensors, or
softw are applications to collect data efficiently and accurately. Ensure that the tools are
designed to capture the required information effectively.
5. D ata Collection: Implement the data collection process according to the defined
m ethods and tools. Collect data from the identified sources while ensuring data quality,
consistency, and relevance to the objectives.
6. D ata Storage and M anagem ent: Organize and store the collected data in a secure and
accessible manner. Establish data management protocols to maintain data integrity and
confidentiality
7. D ata Documentation: Document the data collection process, including details of
sources, methods, tools used, and any modifications made during the process. This
documentation is essential for transparency and r e p r o d u c i b i l i t y . __________

Working with Real Data

Working with real data means using inform ation collected ft'om the real world, like data ft'om sensors,
surveys, o r social media. This data is genuine and reflects the actual complexities and variations
found in real life. Real data is im portant for machine learning because it helps models learn from
real-world situations. The machine learning models will become stronger and more useful with real
data. Unlike artificial or fake data, real data shows the true challenges and diversity of the world by
allowing m odels to adapt better and provide better solutions.
By working with real data, researchers and practitioners can develop m odels that are better equipped
to handle real-world challenges and make more informed decisions.
 Advantages of Working with Real Data in Machine learning Projects
m
O Enhance Model Performance; Real data provides a more accurate representation of the underlying
patterns and relationships in the data, leading to improved model performance and generalization to
unseen data.
O Address Real-World Challenges: Real data often contains noise, missing values, oudiers, and other
complexities that are common in practical applications. Working with such data helps in developing
models that are resilient to these challenges.
O Validate Model Effectiveness: Real data allows for the validation and testing of machine learning
models in real-world scenarios and it ensures that the models perform well when deployed in
production environments.
O Gain Insights and Make Informed Decisions: Analyzing real data can uncover valuable Insights,
trends, and patterns that can inform decision-making processes in various domains such as healthcare,
finance, marketing, and more.
O Improve Ethical Considerations: Real data often contains ethical considerations such as privacy
concerns, bias, and fairness issues. Working with real data helps in understanding and addressing
these ethical challenges in machine learning applications.______________________________________

Sources of Real-World Data

Numerous open datasets are available online across various domains to provide valuable resources
for experim entation and study. These resources offer a wide range of datasets for machine learning
practitioners to explore and utilize in th eir projects by covering diverse topics and domains.
1. UC Irv in e M achine L earning R e p o sito ry : A well-established repository hosting datasets
specifically for machine learning experiments.
W eb site: [Link]
2. K aggle D a ta s e ts : Known for hosting competitions, Kaggle also offers a diverse collection of
datasets contributed by users and organizations, spanning various domains from econom ics
to image data.
W e b s ite : [Link]
3. A m azon's AWS D atasets : Amazon Web Services (AWS) provides a vast array o f public
datasets that can be seamlessly integrated with cloud-based applications.
W e b s ite : [Link]
4. A dd itional R e so u rce s:
• W ikipedia's List o f M ach in e Learning D atasets : A resource listing various d atasets
suitable for machine learning projects.
W e b site :h ttp s ://e n .w [Link]/w iki/List_of_datasets_for_m achine-learning_
research
• Q [Link] : A platform w here users can find discussions and recommendations on
datasets for machine learning.
W e b s ite : [Link]
 • Datasets S u b red d it: A subreddit dedicated to sharing and discussing datasets across
different domains.
W ebsite: [Link] w w .reddiLcom /r/datasets/ ____________________________

Example 1 Real World Data : Indian Liver Patient Dataset__________________________________

The "Indian Liver Patient Records" dataset contains information about liver patients in India and includes
attributes such as age, gender, total bilirubin levels, direct bilirubin levels, alkaline phosphatase levels, and
more. The dataset aims to predict whether a patient has liver disease based on the provided features.
This dataset can be found on the UCI Machine Learning Repository:
Dataset Name: Indian Liver Patient Records
Source; [Link]

By working with this real-world dataset, machine learning practitioners can explore predictive modeling tasks
related to healthcare and medical diagnostics, gaining insights into the factors influencing liver disease in
the Indian population. Analyzing this data can lead to the development of predictive models that assist in
early detection and management of liver-related conditions, showcasing the practical application of machine
learning in healthcare.

Example 2 Real World D ata : Crop Recommendation Dataset

die "Crop Recommendation Dataset" contains informationabout various crops grown in different regions of
India, along witii factors such as soil pH levels, temperature, humidity, rainfall, and crop type. The dataset aims
to predict the most suitable crops to be grown in specific regions based on environmental conditions and soil
characteristics.
This dataset can be found on Kaggle
Dataset Name; Crop Recommendation Dataset
Source: [Link]

This dataset can be utilized for agricultural applications and precision farming in India to help farmers make
informed decisions about crop selection and optimizing agricultural productivity. By analyzing this real-world
data, machine learning models can provide valuable insights and recommendations for crop cultivation,
contributing to sustainable farming practices and enhancing agricultural outcomes in diverse regions of India.

H 2 I Look at the Big Picture

Machine learning (ML) implementation involves understanding the broader context of business
objectives and aligning ML capabilities to meet these goals. By looking at the big picture ensures
that the solutions developed are practical, scalable, and directly contribute to solving real business
challenges.
In the realm of Machine Learning (ML), it is crucial to begin any project by "looking at the big picture."
This involves understanding the goals and objectives of the ML initiative as well as the broader context
in which the project will operate. By taking a holistic view of the problem at hand, stakeholders can
align on the desired outcomes, set clear expectations, and establish a roadmap for success. This
approach sets the stage for effective collaboration, strategic decision-making, and ultimately, the
successful implementation of ML solutions to address real-world challenges.
2.12 V ''M ach in e Learning . •

Structured Approach in Implementing Machine Learning

A structured approach in implementating ML helps stream line the process and ensure alignment with
business objectives. Let us understand the generic fram ew ork applicable across various industries;
1. Define B u sin ess O bjectiv es: Define the specific business objectives that the ML project aims
to address. This step involves understanding the problem domain, identifying key stakeholders,
and establishing measurable goals for the project. The business objectives could range from
reducing costs, improving customer satisfaction, increasing efficiency, or driving innovation.
2. Establish D ata P ip e lin e s: Develop robust data pipelines to streamline the flow of data from
collection to analysis. This involves gathering relevant data sources, preprocessing the data,
and preparing it for model training and evaluation.
3. Evaluate E xisting Solu tions: Assess current processes or systems in place to identify areas
where ML can provide value. By understanding the strengths and lim itations of existing
solutions, organizations can determine the potential benefits of integrating ML technologies.
4. Problem Fo rm u latio n : Define the ML problem type (e.g., supervised learning, unsupervised
learning) and select the appropriate algorithms and techniques based on the nature of the
data and the desired outcomes.
5. Model T rain in g and Validation: Train the model on prepared d atasets and validate its
performance using a separate dataset. This helps in tuning the model to achieve the best
results.
6. Select P erfo rm an ce M etrics: Choose relevant performance metrics such as Root Mean Square
Error (RMSE) to evaluate the effectiveness of the ML models. These m etrics should align with
the business objectives and provide insights into the model's performance.
7. Verily A ssum ptions: Validate the assumptions made during the p roject planning phase
to ensure that the ML models align with the business requirements and deliver actionable
insights for decision-making.
8. Deploym ent: Deploy the model into a production environment w here it can start making
predictions or decisions based on new data.
9. M onitoring and M aintenance: Continuously monitor the model's perform ance in the live
environment and make necessary adjustments. This includes retraining the model with new
data and refining it as business needs evolve.
10. Feedback In teg ratio n : Use feedback from the model’s outputs and business stakeholders to
improve the model and the overall implementation process.

Example : Implementing ML in Real Estate for House Price Prediction

In the dynamic real estate market of India, the integration of Machine Learning presents a
transformative opportunity for accurate house price prediction. By leveraging advanced algorithms
and data analytics, real e state stakeholders can make informed decisions on property transactions.
By integrating sophisticated ML models, businesses can gain a competitive edge, enhancing their
decision-making processes with precise, data-driven insights. This approach not only streamlines
operations but also enriches the customer experience, offering tailored property evaluations
that align closely with m arket djmamics and individual preferences. The implementation of such
technology involves a serie s of structured steps, from defining clear business objectives to deploying
robust models, each crucial for harnessing the full potential of ML in real estate.
1. Define Business Objectives:
• Objective: Develop a model to predict house prices accurately to assist buyers, sellers,
and real estate investors in making inform ed decisions.
• Stakeh old ers: Real estate agents, property buyers, sellers, investors.
2. Establish Data P ip elin es:
• Data C ollection: Gather housing data including features like location, size, number of
bedrooms, bathroom s, amenities, and historical sales data from sources like property
portals (M agicBricks, 99acres), governm ent housing indices, and census data.
For example, collect data for 5,000 properties in Bangalore, with features including built-
up area (in square meters), number o f bedrooms, property age, distance from nearest
metro station, and average local school ratings.
• Data P rep ro cessin g : Preprocess the data to handle missing values, encode categorical
variables such as locality names, and norm alize features like property size (measured in
square feet) and age of the property.
• Data P re p a ra tio n : Split the data into training and testing sets for model training and
evaluation.
3. Evaluate E xisting Solutions:
• The existing solutions may rely on local real estate agents' knowledge or simple
computational models which often fail to capture complex m arket dynamics and can vary
significantly across different regions.
• Assess cu rren t pricing models used by real estate agencies and identify areas where ML
can enhance accuracy and efficiency in predicting house prices.
4. Problem Fo rm u lation :
• ML P rob lem Type: This scenario is a supervised regression problem as the output
variable (house price) is continuous.
• A lgorithm s: Select regression m odels such as Linear Regression for baseUne
performance and more advanced models like Random Forest and Gradient Boosting for
potentially higher accuracy.
5. Model Training a n d Validation:
• Training P ro c e s s : Divide the data into an 8 0 % training set and a 2 0 % testing set. Utilize
cross-validation within the training set to optimize the model param eters.
• Validation : Validate the model's performance on the testing dataset. Tune
hyperparam eters to optimize the m odel's accuracy.
6. Select P erfo rm an ce Metrics:
• P erfo rm an ce M e tric : Use the Root Mean Square Error (RMSE) to m easure the average
error betw een the model's predictions and actual prices, providing a direct measure of
prediction accuracy.
2 .1 ' ? ! . ■ ft

• RMSE Calculation: RMSE = sqrt(Z(predicted price - actual price) V n)

. M easurable Goal: The goal is to develop a predictive model with a Root Mean Square
Error (RMSE) target of less than ^50.00,000, ensuring precise estimations that align with
the diverse needs of buyers, sellers, and investors.
• The choice of ^50,00,000 (50 lakhs) as the target Root Mean Square Error (RMSE) value
in predicting house prices is based on the typical price range of residential properties
in the Indian real estate market. This value serves as a practical benchmark to ensure
that the model's predictions are sufficiently accurate for the majority of house prices
encountered in India. By aiming for an RMSE below ?5 0 ,00,000, the model strives to
provide reliable estimates that align with tiie market dynamics and expectations of
buyers, sellers, and investors in the Indian real estate sector.
7. Verify A ssum ptions;
. Validate that the model aligns with the business objective of accurate house price
prediction. For example, when a user inputs features of a property located in Indiranagar,
Bangalore, measuring 150 square meters, 10 years old, with high local amenities, the
model predicts a price of ?1.5 crores.
• Ensure the model provides actionable insights for real estate decision-making.

8. Deplojmient:
• Deploy the trained model into a production environment where it can receive new data
inputs and make predictions on house prices.
9. M onitoring and Maintenance:
• Monitor the model's performance regularly in the live environment.
. Retrain the model periodically with new data to keep it updated and accurate.
10. Feedback Integration:
. Gather feedback from real estate agents, buyers, and sellers on the model's predictions.
. Use feedback to improve the model's accuracy and refine the implementation process.

(jretthe Data
Working on a m achine learning project begins by obtaining the necessary data, which serves as tlie
foundation for all subsequent modeling and analysis. Let us understand the structured approach to
effectively gather data for ML projects:
1. S ettin g Up Your Environment:
. S ystem Preparation: Ensure Python is installed on the system. If not, it can be
downloaded and installed from Python's official website.([Link]
. W o r k s p a c e C r e a t i o n r - A - d e d i c a t e d - d i r e c t o i i ^ o r m a c h i n e l e a r n i n g p r o je c t s s h o u ld be
created for organizational clarity. This can be set up using Command Prompt:

C:\> mkdir C:\ML_Projects

C:\> cd C:\ML_Projects___
 Data 2.15

2. C reating a n Isolated Environm ent;

• U sin g V irtu al Environm ents: It is recommended to w ork in an isolated environment to
m anage dependencies more effectively and avoid conflicts betw een projects. The virtual
environm ent tool should be installed and a new environment created:

C:\> pip install virtualenv # Install virtualenv

C:\> virtualenv ml_env # Create a new virtual environment named ml_env
C:\> inl_env\Scripts\activate # Activate the virtual environment________
W hile activated, any packages installed using pip will only affect this environment.
To ex it the virtual environment, simply run: deactivate
3. In stallin g N ecessary Tools:
• E s s e n tia l Libraries: Libraries such as Jupyter, NumPy, pandas, Matplotiib, and Scikit-
Learn should be installed if they are not already present. These can be installed using
pip, w hich is Python's package manager. Open Command Prom pt and enter the following
com m ands:
(ml_env) C:\> python -m pip install --upgrade pip
(ml_env) C:\> pip install jupyter matplotiib numpy pandas scikit-learn
4. W riting P 5fth o n Scripts: Using a Text Editor or IDE, Create a new file with a .py extension, for
example, data_fetching.py and save it in C: \ML_Projects
This script is a straightforward tool for data acquisition, automating the process of downloading
a dataset and ensuring its availability on the local machine for further data analysis or machine
learning tasks.

Example A Python Script to Get Data firom a Repositoir.

import OS
import [Link]

DATA_URL="[Link]
housing/housing.C SV "
DATA_PATH = [Link]("C:\\ML_Projects", "datasets", "housing")

def fetch_data(data_url=DATA_URL, data_path=DATA_PATH):

[Link](data_path, exist_ok=True)
csv_path = [Link](data_path, "housing,csv")
[Link](data_url, csv_path)
print ("Data downloaded to:", csv_path)

if __name__ == "__^main__
fetch_data()
 The CSV is downloaded to the C:\ML_Projects as shown below.

> ThisPC > L o c a lP ts k (C ) » M _ P n ^ * datasets > housing

I'
Date modified -fype j Size
Name

1 9-04-2K 41& 57 M ia o so ft Excel C... 1,391 KB

I housmg

Explanation
The above Python script automates the downloading of a dataset from a specified URL and stores it
locally.
import os: This module provides a way of using operating system dependent functionality like
reading or writing to the file system.
. import urllib-request: This module is used for opening and reading URLs, specifically, it is
used here to download data from the internet.
. [Link]: A string that holds the URL of the dataset. This URL points to a CSV file hosted on
GitHub.
. DATA PATH: A string that specifies the local directory path where the dataset will be saved. It
uses o'[Link] to construct the path. This function is platform-independent and ensures the
path is correctly formatted for the operating system.
• fetch_data: The function is defined to handle the downloading of data.
. [Link](data_path, exist_ok=True): Ensures that the directory specified by [Link]
exists. If it doesn’t, this function will create all necessary directories in the path. The parameter
exist_ok=True allows the command to succeed even if the directory already exists.
. csv_path: Constructs the full local file path where the CSV file will be saved after downloading.
. [Link](data_url, csv_path): Downloads the file from data_url and saves it
to the location specified by csv_path.
. printfD ata downloaded to:", csv_path): Outputs a message to the console indicating where
the data has been downloaded.
. Main Block - if _ n a m e _ == "_m ain_": This block ensures that the fetch_dataQ function is
called only when the script is run directly__________ _________ ______________

L o a d th e D a ta a n d E x p l o r e th e D a ta
When working with machine learning projects, the first m ajor step after acquiring the data is to load
it into a usable format for analysis and preprocessing. The m ost common format for data storage is
the CSV (comma-separated values) file, which can easily be loaded into a Pandas DataFrame. After
loading the data, exploring it helps in understanding the structure, content, and initial insights that
guide further data manipulation and analysis.
 1. Load the Data : The process begins with importing necessary libraries and loading the data
into a DataFrame. This is tj^ ically done using Pandas due to its robustness and ease o f use for
handling structured data.
2. Exploring the D ata : Once the data is loaded into a DataFrame, it's crucial to explore it to
understand its characteristics, such as the number o f features, rows, possible m issing values,
and the type of data (num erical or categorical).
Example Load the Data
import pandas as pd
import os

# Define the path where the data is stored

DATA_PATH = "C: \\ML_Projects\\datasets\\housing"

# Function to load data from a CSV file

def load_data(data_path):
csv_path = [Link](data_pathj "[Link]")
return pd.read_csv(csv_path)

# Function to explore the loaded data

def explore_data(data):
# Display the first 5 rows
print("First 5 rows of the data:\n", [Link]())
# Summary statistics for numerical columns
print("\nSummary statistics of the data:\n", [Link]())
# Data types and missing values
print("\nData types and missing values:")
[Link]

# Main block to ensure the script runs only when directly executed
if __name__ == "__main__" :
data = load_data(DATA_PATH) # Load the data
explore_data(data) # Explore the data
Explanation <
• Import Libraries: The script starts by importing necessary libraries, pandas is used for data
manipulation and analysis, and os helps in handling file and directory paths.
• Define Data Path: DATA_PATH holds the directory path where the CSV file is stored. This makes the
script more flexible and easier to modify if the data location changes.
• Load Data Function: load_data function takes the path to the CSV file, constructs the full path to the
file, reads it using pd.read_csvQ, and returns the DataFrame. This DataFrame contains all the data from
the CSV file, ready for analysis.
• Explore Data Function: [Link] function takes a DataFrame as input and performs three key
operations: __________________________________ _________
 Head of tiie DataFrame: Shows the first five rows using [Link], providing a quick view
dataset structure and initial rows.
Stattsacal summary: Uses daK,.describeO to output summary statlsdcs tijat describe the uumen,:al
flelds<,[Link]>it,mean,stdCstandarddeviatloii),[Link].

Data information: The dataJnfoO method prints a concise summary '

total number of entries, the number of non-null values in each column, and the data typ C -g-.
int64, object) of each column. . . ^
Main Block: The if name ==’ m ain_-:blocltensuresthatthedataloadingandexplonngfuncnons
W h lscrlp tls run as the main module. This is useful when you want to Prevent t e e
Ifr o m being autnmatlcallv executed if the script Is imported as a module m another s c n p t _
functions 1
------------------------------ - ^ ^

First 5 rows of the data;

median. median. ocean
housing. totaL households
longitude latitude
rooms income house_value .proximity
[Link]
126.0 8.3252 452600.0
41.0 880.0
1138.0 8.3014 358500.0
21.0 7099.0
177.0 7.2574 352100.0
52.0 1467.0
5.6431 341300.0 NEAR BAY
52.0 1274.0 219.0
3.8462 342200.0 NEAR BAY
52.0 1627.0 259.0

[5 rows X10 columns]

median. .[Link].
housing.
latitude households value
longitude income
[Link]
20640.000000 20640.000000 20640.000000
20640.000000 20640.000000
499.539680 3.870671 206855.816909
-119.569704 35.631861 28.639486
382.329753 1.899822 115395.615874
2.135952 12.585558
1.000000 1.000000 0.499900
32.540000
18.000000 280.000000 2.563400
33.930000
409.000000 3.534800 179700.000000
-118.490000 34.260000 29.000000
605.000000 4.743250 264725.000000
-118.010000 37.710000 37.000000
6082.000000 15.000100 500001.000000
max -114.310000 41.950000 52.000000

Summary statistics of the data:

[8 rows X 9 columns]
Data types and missing values;
<class '[Link] >

Rangelndex: 20640 entries, 0 to 20639

Data columns [total 10 columns):
II
'Data'Prepa Z I9

# Column Non-Null Count Dtype

0 longitude 20640 non-null float64

1 latitude 20640 non-null float64

2 housing_median_age 20640 non-null float64

3 totaLrooms 20640 non-null float64

4 total_bedrooms 20433 non-null float64

5 population 20640 non-null float64

6 households 20640 non-null float64

7 medianjncome 20640 non-null float64

8 median_house_value 20640 non-null float64

9 ocean_proximity 20640 non-null object

dtypes: float64(9), object(l)

memory usage: 1.6+ MB

Create a Test Set

Creating a test set early in a Machine Learning project is crucial for evaluating the model s performance
accurately and ensuring its ability to generalize to new, unseen data.

Importance of Creating a Test Set

Separating a portion o f the dataset for testing is essential as it allows for assessing how well a model
will perform on new data that it has not seen during training. This process sim ulates real-world
scenarios and helps in determining the model's reliability and generalization capabilities.

1. Allocation: W hen creating a test set, it is im portant to allocate a specific portion of the dataset,
typically around 20% , for testing purposes. This allocation ensures that there is a separate
subset of data reserved exclusively for evaluating the model's performance.
2. Purpose: The te st set serves as a critical com ponent in the model development process by
providing a m eans to assess how well the model generalizes to new, unseen data. By withholding
a portion of the data for testing, we can evalute the model's effectiveness in making predictions
on data it has n ot been trained on.
3. Independence: The test set should be independent of the training data to avoid any biases
that may arise from using the same data for both training and evaluation. This independence
ensures that the model's performance is assessed on truly unseen instances, enhancmg its
reliability in real-world applications.
4. Evaluation: Testing the model on a separate te st set allows for a com prehensive evaluation
of its predictive capabilities. By com paring the model's performance on the test set to ite
performance on the training data, we can gain insights into its ability to generahze and make
accurate predictions on new data.
2 .2 0 ^ - Woehine Ufflrning

5. V alidation: The test set acts as a validation mechanism for the model, providmg a benchm ark
for assessing its performance and identifying any potential issues such as overfitting or
underfitting. This validation step is crucial in ensuring the m odel's robustness and reliability
in practical scenarios.
6. Consistency: To maintain consistency in model evaluation, it is recommended to set a random
seed when splitting the data into training and test sets. This practice ensures that the test set
remains consistent across different ru ns of the model, allowing for reliable comparison and
assessm ent of model performance.

Different Techniques for Test Set Creation

1. Random Sam pling: This method is suitable for large d atasets w here randomly selecting
data points for the test set ensures th a t it is representative of the overall dataset. It helps in
evaluating the model's performance on a diverse set of instances.
2. Stratified Sam pling: When certain characteristics or categories in the data are crucial for the
model's predictions, using this technique ensures that these attributes are well-represented in
the test set. It is particularly useful for sm aller datasets w here maintaining the distribution of
important features is essential for accu rate evaluation.______________

Example Data Preparation: .Creating a Test Set in Python_____________ - ■•

import numpy as np
import pandas as pd
from sklearn .model_selection import StratifiedShuffleSplit

# Function to load data from a CSV file

def l o a d _h ou si ng _d at a( dat a_ pa th ):
csv_path = [Link](data_path, "[Link]")
return pd.read_csv(csv_path)

# Define the path where dataset is stored

data_path = "C: \\ML_Projects\\datasets\\housing"

# Load the data into a DataFrame

housing = load_housing_data(data_path)

# Create an income category for stratified sampling

housing["income_cat"] = [Link](housing["median_income ],
bins=[0., 1.5, 3.0, 4.5, 6., [Link]],
labels=[l, 2, 3, 4, 5])

# Stratified splitting of the data using StratifiedShuffleSplit

split = StratifiedShuffleSplit(n_splits=l, test_size=0.2, ra ndom_state=42)
for train_index, test_index in split. split(housing, ho u s i n g [ income_cat ]):
 ' Data Preparation

strat_train_set = [Link][train_index]
strat_test_set = [Link][test_index]

# Remove the income_cat attribute so the data is back to its original state
for set_ in (strat_train_set, strat_test_set):
set_.drop("income_cat", axis=l, inplace=True)

# Print the proportions of each income category in the test set

print("Stratified Sampling Test Set Proportions:\n”, strat_test_set[”median_income"]
value_counts() / len(strat_test_set)) ________________________________ __
Output
Stratified Sampling Test Set Proportions:

4.1250 0.002907
3.1250 0.002907
15.0001 0.002665
3.8750 0.002422
2.1250 0.002422

3.7831 0.000242
2.8056 0.000242
2.1270 0.000242
2.1518 0.000242
4.1111 0.000242
Name: medianjncome, Length: 3446, dtype; float64
Explanation
Load Data : The load_housing_data function is crucial as it allows the seamless reading of structured
data (like CSV files) into a Python environment where it can be easily manipulated and analyzed.
This function uses the pandas library, which is a powerful tool for data analysis and manipulation.
Specifically, pandas.read_csvQ is used to load the data from a CSV file into a DataFrame. A DataFrame
is a 2-dimensional labeled data structure with columns of potentially different types of data.
By loading data into a DataFrame, users can take advantage of the various data manipulation
capabilities of pandas to clean, transform, and preprocess the data effectively before any analysis or
model training.
Income Categoiy : Creating an incom [Link] column is essential for performing stratified sampling
based on the median income of the households in the dataset Stratified sampling is a method of
sampling that involves dividing a population into smaller groups, known as strata, that share a similar
attribute.
The function uses [Link] to categorize the medianjncome into specified bins. This categorization
helps in ensuring that the sampling of data for model training and testing reflects the overall distribution
of income categories in the entire dataset_______________________________ ___ ________________
 Stratification ensures that each category of income is properly represented in both training and test
sets, which helps in building a model that performs well across different income groups, thereby
reducing sampling bias.
• Stratified Sampling: It is used to divide the data into a training set and a test set while maintaining a
consistent percentage of samples for each categoiy of income across both sets.
StratifiedShuffleSplit from skleam.model_selection provides a way of ensuring that the data is
randomly split in such a way that the income categoiy proportions are preserved in both training and
test datasets as compared to the full dataset
This technique helps in maintaining the statistical properties of the original data, which can be critical
for the predictive performance of the machine learning model, especially on unseen data.
• Clean Up : It is used to clean up the data by removing the income_cat column after the stratified
sampling is done. The dropQ method from pandas is used on both training and test sets to remove the
income_cat column, reverting the dataset to its original state before stratification.
This step is crucial for keeping the data tidy and ensuring that only the original attributes are used for
further analysis and model training.
• Printing Proportions; It is to verify that the stratification was performed correctly. The script prints
the proportion of each income category writhin the test set using value_count$0 and normalizing the
results with lenQ to get the percentage.
This confirmation step is essential to ensure that the stratified sampling process has been executed as
expected, providing confidence in the robustness of the subsequent training and validation processes.

Discover and Visualize the Data to Gain Insights

Discover and Visualize Data to Gain Insights refers to the process of exploring and analyzing the
dataset through various techniques to extract valuable inform ation and understand the underlying
patterns, relationships, and characteristics of the data. By analysing the data visually, we can uncover
hidden insights that may not be apparent from raw data alone. This step is essential during data
preparation in a machine learning p ro je c t

Why Visualizing the D a ta is Needed During D ata Preparation?

Discovering and visualizing data during the preparation phase is crucial for several reasons, helping
to ensure the effectiveness and accuracy o f a machine learning model;
1. U nd erstanding the D ata S tru c tu r e : Visualization helps in understanding the underlying
structure of the data, the distribution of key variables, and the relationships betw een different
variables. This insight is vital for selecting tiie appropriate algorithms and tuning them
effectively.
2. Identifying Patterns and A n o m alies : Through graphical representation, it becom es easier
to spot patterns, trends, and anom alies that might not be apparent from raw data. For example,
outiiers that could skew the results of an analysis are m ore readily visible on a plot or graph.
3. F e a tu re Relationships : Visualization helps in understanding how various features relate to
each other and to the target variables. This can help in identifying the most relevant features
to include in a model.
 PK paralior

4 . E rro r Id e n tificatio n : Early in the process, visualizing data can reveal errors in data collection
and processing such as biases and inconsistencies that need to be addressed before further
analysis.
5. Inform ing P rep ro cessin g D ecision s : Insights gained from visualizing data can direct the
preprocessing steps like normalization, handling m issing values, or feature engineering.
For example, if data visualization shows that some variables have a non-linear relationship,
poljmomial features might be created to model these effects.
6. Facilitatin g C om m unication : Visualizations make it easier to communicate findings and
data characteristics to stakeholders, who may not be familiar with the technical aspects of
data science but need to understand the basis of decisions or models built from the data.

Hr^JlBr^ata Visualization Techniques

Data visualization techniques play a crucial role in transform ing raw data into meaningful insights
that drive informed decision-making and facilitate effective communication of findings. By visually
representing data through graphs, charts, and plots, analysts can uncover patterns, trends, and
relationships that may not be apparent from raw data alone.
Data plotting techniques includes a variety of visual tools that aid in exploring and presenting data in
a com prehensible manner. These techniques allow analysts to uncover insights, identify patterns, and
com municate findings effectively to stakeholders.
1. H istogram s:
• Purpose : Histograms display the distribution o f numerical data by dividing it into bins
and showing the frequency of values within each bin through bar heights.
• Usage: Histograms are commonly used in data preparation to visualize the distribution
of features and identify patterns such as outliers, skewness, or gaps in the data.
• Example: When preparing housing price data, a histogram of house prices can reveal
if the data is normally distributed or skewed. This insight can guide decisions on data
transformation techniques like normalization or log scaling to address skew ness before
model training. _______
Marts scored by stiiderts

0-10 10-20 20-30 30-« «-5<>

■ Range o f M a ris scored by students
2.24 ^ > |« g ^ ^ eW rnin 9 V -V.'.-

2. Box Plots:
• P u rp ose: Box plots summarize the distribution of data by displaying quartiles, outliers,
and the overall spread of values in a com p act visual format.
• Usage; Box plots are valuable in data preparation for comparing the distribution of
features across different categories o r groups and identifying potential outliers or
variations.
• Exam ple: In a retail dataset, box plots can b e used to compare sales figures across different
product categories. Detecting outliers in sales data for specific product categories can
prompt further investigation into data quality issues or anomalies before analysis.

3. Tim e S e rie s P lo ts:

• P u rp ose : Tim e series plots visualize data points over sequential tim e periods, enabling
trend analysis and pattern identification in temporal data.
• Usage: Tim e series plots are essential in data preparation for tracking changes in data
over time, detecting seasonality, and understanding historical trends.
• Exam ple: Analyzing monthly w ebsite traffic data using a time series plot can reveal
recurring patterns or spikes in user activity. This insight can inform decisions on data
preprocessing steps like handling m issing values or smoothing out irregularities in the
data.

4. H eatmaps:
• P u rp ose : Heatmaps use color gradients to represent data values in a matrix format,
making it easier to identify patterns and relationships in large datasets.
II Data Preparation

U sage: Heatmaps are beneficial in data preparation for visualizing correlations between
features, identifying clusters, or detecting anomalies.
E xam p le: Creating a heatmap o f feature correlations in custom er survey data can
highlight strong relationships betw een satisfaction scores and purchase behavior. This
can guide feature selection strategies during data preprocessing to enhance model
perform ance.

5. Scatter P lo ts:
• P u rp o se: Scatter plots display the relationship between two variables by plotting data
points on a Cartesian plane, helping to identify patterns, trends, and correlations.
• U sage: Scatter plots are useful in data preparation for exploring associations between
variables and detecting oudiers or data inconsistencies.
• E xam p le: Plotting a scatter plot o f custom er age against purchase frequency in a sales
dataset can reveal any linear or non-linear relationships betw een age and buying
behavior. This insight can guide feature engineering decisions during data preprocessing
to capture relevant patterns for predictive modeling.

Scatterplot o f %Fat vs BM f

45

AO
^ 3S
tE
as 30
25

20

15
15 20 25 35
 w m m ^

6. Line C h arts:
• P u rp o se : Line charts depict data trends over time by connecting data points with hnes,
making them ideal for tracking changes and patterns in sequential data.
. U sage: Line charts are valuable in data preparation for visualizing temporal trends,
m onitoring data quality metrics, o r tracking preprocessing steps over time.
• E xam p le: Tracking the evolution o f data cleaning efforts using a line chart of missing
value percentages over successive data preparation stages can help in assessing the
effectiveness of data cleaning techniques and ensuring data quality before model training.
Product A, Product B and Total Product Sold

Examples Create CSV file

Create CSV file with below data and save it as student_data.csv in C:\ML_Projects folder.
Name,Age,Course,Semester,Attendance_Percentage,Height_cm,Weight_kg

Aditya, 20,BCA,4,88,172,70
Bhavika,19,BBA,2, missing,158,50
Chirag, 21,BCom,5,9 2 ,180,80
Deepa,22,BCA,6,94,160,missing
Esha,20,BBA,3,missing,70,54
Farhan,19,BCA,l,85,165,65
Gita,21, BCom,6,78,170,75
Hitesh, 20,BCA,4,101,165,170
Ila,18,BBA,l,87,190,48
Dai,22,BCom,5,85,180,82
Kavya, 19,BCA,2,missing,159,110
Lalit,20,BBA,3,95,175,80
Mira, 2 1 ,BCom,6,82,missing,70
Nikhil,22,BCA,5,93,174,76
dm,19,BBA,2,75,168,60
Priya,18,BCom,1,95,154,55
Raj, 21, BCA, 5,90,182,83
Sunita, 20,BBA,3, missing,160,120
Tarun,19,BCom,4,88,80,40
Usha,21,BCA,6,92,164,59___________ ___________ ________ __________________ _
 Example Data Visualization
import pandas as pd
import [Link] as pit
import numpy as np

# Load the data

def load_data():
data = pd.read_csv('student_data.csv')
data [ ■Attendance_Percentage'] = p d .to_numeric(data['Attendance_Percentage'], errors='coerce')
data['Height_cm'] = pd.to_numeric(data['Height_cm'], errors='coerce’)
data['Weight_kg'] = pd.to_numeric(data['Weight_kg'], errors='coerce')
return data

# Prepare data by handling missing values

def prepare_data(data):
data [ ■Attendance_Percentage']. fillna (data ['Attendance_Percentage ’]. median (),
inplace=True)
data [' Height_cm'].fillna(data [' Height_cm'].median(), inplace=True)
data [' Weight_kg'].fillna (data [' Weight_kg ’].median (), inplace=True)
return data

# Plotting function using only Matplotlib

def plot_data(data):
[Link](figsize=(15, 10))
# Histogram of Attendance Percentage
[Link](2, 3j 1)
[Link](data['Attendance_Percentage'], bins=10, color='skyblue'j edgecolor='black')
pit. title ('Attendance Percentage Distribution')
[Link] (' Percentage ’)
p i t , ylabel ('Frequency')

# Boxplot for Weight

[Link](2, 3, 2)
[Link] (data ['Weight_kg']. dropna ())
pit. title ('Weight Distribution')
[Link]('Weight (kg)')

# Boxplot for Height

[Link](2j 3, 3)
[Link] (data [-Height_cm']. dropna ())
[Link]('Height Distribution')
pit, ylabel ('Height (cm)')________________ __________________________ ____________
 *
Machine Morning

# Scatter plot of Height vs Weight

[Link](2, 3, Hat.r-Weieht ke'l, alpha=0.6, c=’red')
[Link] at te r(d a t a [' He ig ht .c m ], data[ We ig ht , g j,
[Link]('Height vs Weight Scatter Plot )
[Link](’Height (cm)')
[Link]'Weight (kg)’)
# Bar chart of Course Enrollment
[Link](2, 3, 5)
- H atar' C ou rse' 1 .value_counts()
pu"bar(course_counts.index, [Link], color='green
[Link]('Course Enrollment')
[Link]('Course')
[Link]('Number of Students')
# Correlation Heatmap
[Link](2, 3, 6)

[Link](rlU(len([Link])), ™ ’=ation-«)
[Link](range(len([Link])), [Link])
[Link]('Correlation Heatmap’)
plt.tight_layout()
[Link]

def main():
data = load_data()
prepared_data = prepare_data(data)
plot_data(prepared_data)

if __name__ == — •
 Weight Distnbution Cburae Bmilltnent

Height Distribution CotnlatianHeatnnp

Explanation:
The chart provides a comprehensive look at the distribution and relationship between various features in a
student dataset
1. Attendance Percentage Distribution: This histogram likely shows the frequency of students across
different ranges of attendance percentages. Peaks in the histogram can indicate common attendance
rates, while gaps might suggest less common attendance behaviors.
2. Weight Distribution : The box plot for weight distribution probably displays the spread of student
weights. Outliers may appear as individual points, indicating students with weights significantly higher
or lower than the rest.
3. Height Distribution : Similar to the weight distribution, the box plot for height helps identify the
median, quartiles, and potential outliers in student heights. This can help pinpoint if any students are
unusually tall or short, which might be outliers needing verification.
4. Height vs Weight Scatter P lo t: This scatter plot likely illustrates the relationship between students'
heights and weights. A clear trend, like an upward slope, would suggest a positive correlation, where
taller students tend to be heavier.
5. Course Enrollment: A bar chart showing the number of students enrolled in different courses would
indicate the popularity or selection frequency of each course, like BCA, BBA, or BCom.
 relationship between variables, like age and semester correlatmg with progression

program.
Each c h a rt helps in identifying different data discrepancies: t+pm.= thatreauire
.Thehistogr^mcan reveal Ifattendancedata is [Link]

further investigation.
. The box plots for weight and height can highlight outliers that might be due to data entry er ^
. The scatter plot can help find any unusual relationship between height and weight which might
conform to medical standards.
. The bar chart can suggest if them’s an imbalance in course enrollments that may affect class soes

resource allocation.

building^_____________ __________
Prepare
irrepaic? the D ata for Machine Learning Algorithms
----------------

m m B m m
m achine learning model d u r in g training and evaluation. „„rW i„w this

S t s k s by leveraging existing tools and functions available in Python libranes.

■ E S S l s data cleansing is the

complete, and consistent.

I— ^ ^ i^ W h y Data Cleaning is Important in ML?

Data cleaning is essential in Machine Uarning for the following reasons:

. Accuracy: Clean d au ensures accurate model training and reliable predictions.
■ ^ „ „ „ .H ,,.-n ,,.li» d a ta le a d sto m o re ro b u stm o d e lsa n d b e tte rd e cisio n -n 'a lo ^
 Normalization: Cleaning data helps in standardizing and prepanng the dataset fo ran i^ is~
Feature Engineering: Data cleaning is crucial for effective feature extraction and selection
Model Perform ance: Clean data enhances model performance, reduces errors anri
overall efficiency in ML tasks. ' ™Proves

Missing values in data refer to the absence of inform ation or data points for certain observations or
attributes in a dataset. Handling missing values is crucial in data preprocessing to ensure the qualitv
and reliability of the m achine learning model.

Example: The Titanic Passengers dataset has m issing values in the Age and Cabin columns. The
passenger information has been extracted from various historical sources. In this case the missinc
values couldn't be found in the sources.

Passengerd Survived Pclass Gender Age SibSp Parch Ticket Fare Cabin Embarked
1 0 3 Male 22 1 0 A/5 21171 7.25 ( ) s
2 1 1 Female 38 1 0 PC 17599 71.2833 C85 c
3 1 3 Female 26 0 0 STON/02.311282 7.925 ( > s
4 1 1 Female 35 1 0 113803 53.1 C123 s
5 0 Male 35 0
3
0 373450 8.05 ,P" N
\ s
6 0 3 Male ( 0 0 330877 8.4583 '
Q

Missii^values
Common M ethods to H and le Missing V a lu e s:
• Deletion: Involves removing entire rows with missing values. While simple, it can lead to loss
of valuable data.

• M ean/M edian/M ode Im putation: Replace missing values with the mean, median, or mode
of the respective feature. This method is sim ple but may distort the original distribution.
. Forward Fill/ B ack w ard Fill: Fill missing values with the most recent non-missing value
(forward fill] or the next non-missing value (backward fill) along the column.
. K-Nearest N eig h b o rs (KNN) Im putation: Predict missing values based on the values of the
nearest neighbors in the feature space.
• Prediction M odels: Use machine learning algorithms to predict m issing values based on
other features in the d ataset This approach can be effective but requires m ore computational
resources.

0^ Example Handling Missing Values

Let's consider an example dataset with missing values in the "Age" and -Income" columns:
ID Gender Age Income Region
1 Male 35 50000 East
2 Female NaN 6 0000 West
3 Male 45 NaN North
4 Female 30 7 0000 South
 ■ -':^L<*'

In this example, we have missing values represented as "NaN" in the ’Age' and "Income- columns. To handle
these missing values:
1 Identify Missing Values: Look for cells in the dataset that contain "NaN" or any other placeholder
indicating missing data. In our example, we have missing values in the "Age" and Income columns.
2. Choose Imputation Method: Decide on the imputation method to fill in the missmg values. For
simplicity, let's use mean imputation in this example.
3. Calculate Mean Values: Calculate the mean age and mean income from the available data in the
respective columns.
4. Replace Missing Values: Replace the missing values in the "Age" column with the mean age and the
missing values in the "Income" column with the mean income.
5. Updated Dataset:

Gender Age Income Region

ID
Male 35 50000 East

Female 237.5 (mean) 60000 West

Male 45 56666.67 (mean) North

Female 30 70000 South

Example I A Python- Code to Handle Missing Values “

import pandas as pd
from [Link] import Simplelmputer

# Example DataFrame with m issing valu es

data = { 'F e a t u r e l ': [10, 20, 39 , None, 50 ],
'F e a tu r e 2 ': [5 , None, 15 , 20, 2 5 ]}

df = [Link](data)

# Define the imputer

imputer = Simplelmputer(strategy="mean") # or median, most_frequent

# Apply th e imputer to the DataFrame

df_filled = pd. DataFrame(imputer.fit_transform(df), columns=[Link])

p r in t ("Original DataFrame with Missing Values: )

print(df)
print("\nDataFrame a ft e r Handling Missing V a lu e s:")
p rin t (d f_ fille d )______________ _______ ____________ -_______________ - —
Original DataFrame with Missing Values:
Featurel Feature2
0 10.0 5.0
1 20.0 NaN
2 30.0 15.0
3 NaN 20.0
4 50.0 25.0

DataFrame after Handling Missing Values:

Featurel Feature2
10.0 5.00
2 0.0 16.25
3 0 .0 15.00
2 7 .5 20.00
5 0 .0 25.00
Explanation
The above Python code snippet demonstrates how to handle missing values in a DataFrame using the
Simplelmputer class from scikit-learn.
• The code snippet uses Simplelmputer from scikit-learn to handle missing values in a pandas DataFrame
efficiently.
• By setting the imputer strategy to "mean," missing values in the DataFrame are replaced with the mean
of each column.
• The code demonstrates a practical approach to data cleaning, ensuring the dataset is prepared for
analysis or machine learning tasks.
• The fit_transformO method of the imputer is applied to fill missing values, resulting in a cleaned
DataFrame ready for further processing.
• The code showcases the seamless integration of pandas and scikit-learn for data preprocessing,
______ emphasizing the importance of handling missing values in machine learning workflows._____________

2. H andling Outliers

Outliers are data points that significantly differ from other observations in a dataset. These data points
can skew statistical analyses and m achine learning models, leading to inaccurate results. Outliers can
occur due to various reasons such as measurement errors, data entry mistakes, or genuine extrem e
values in th e data.
E xam ple : In a dataset containing information about individuals, such as their age, it is com m on to
encounter outliers, such as ages above 100 years. While som e individuals may indeed be over 100
years old, extrem e ages can impact statistical analyses and m achine learning models if not handled
appropriately.
Id en tification of Outliers:
• Visual methods like box plots, scatter plots, and histograms can help identify outliers. Statistical
m ethods such as z-scores, IQR (Interquartile Range), and Tukey's method can be used to detect
outliers.
 neededtoensurethattherem ovalofoutliersdoesnotbiastheanalysis.

r = S r r = = = 2 E S r

. m X G a p in g outliers i n » a separate category o r bin ca„ be a suitable approach

depending on the nature o f the data.
. Advanced M odels: Ualizing robust models that are less sensitive to [Link], such as Random

Forest or Support Vector Machines, can be beneHcial.____________ _________________ _______

Js Example ■ Handling Outliers_________ __________ _________________ _____________ ________________—

----------- — *------------ — ----------- ^ nnn to 3 00 000 with a few entries exceeding 5,00,000
If the dataset contains salaries ranging from 30,000 to i,uu,uuu, wiui
(considered outliers):
. [Link](e.g.,4,00,000).
. . 400.000 iscapped at 400,000 to prevent extreme values from skewing the

Example A Python Code to Handle Outliers

import pandas as pd
import numpy as np

# Example DataFrame with potential outliers

data = { ’Values': [10, 20, 30, 150, 25, 35, 200, 40]}
df = [Link](data)

(dn°v:iues'l - dfrvalues.,..ean(„ / dffValues.],std(ddo.=e,

# Define a threshold for outliers (commonly set to 1 or 2 or 3)

threshold = 1

# Identify and remove outliers using the Z-score

H4: rlp;,npd = d ffn p .a b s (d f[ ’ Z _ s co re ']) <= t h r e s h ^ ------------------------------------------------
I Data PreparaKe

# Remove the Z-score column for the cleaned DataFrame

df_cleaned = df_cleaned.drop(columns='Z_score')

# Output the original and cleaned DataFrame

print("Original DataFrame with Outliers:")
print(df)
print("\nDataFrame after Handling Outliers:")
print (df_cleaned)
Output
Original DataFrame with Outliers:
Values Z score
10 -0.814517
20 -0,662979
30 -0.511441
150 1.307015
■25 -0.587210
35 -0.435672
200 2.064705
40 -0.359903

DataFrame after Handling Outliers:

Values
0 10
1 20
2 30
4 25
5 35
7 40
Explanation:
The code calculates the Z-score for each data point in the 'Values' column by subtracting the mean and
dividing by the standard deviation, providing a standardized measure ofhow far each value is from the
mean.
The above code identifies potential outiiers based on a specified threshold, and creates a cleaned
DataFrame by removing the identified outliers.
The threshold value is set to 1, and any data points with an absolute Z-score greater than the threshold
are considered outliers and removed.
The cleaned DataFrame is then displayed without the Z-score column.
This process helps in detecting and handling outliers in the dataset, ensuring data quality for further
analysis or modeling tasks. _________________________________________________

Data Transformation
Data transformation is a fundamental process in data preprocessing that involves modifying the
original data to make it m ore suitable for analysis or modeling. This transformation can help improve
the quality of the data, address issues like skewness or outliers, and enhance the performance of
m achine learning algorithms.
 ....
, Machine Learning ' '' -’.iip '■ ''■^3^'^':^ ' * I't ■
■‘ ; ‘ -■-^V-?-'’^.'V.^

Why Data Transformation is Important in ML?

Data transformation is a crucial step in the machine learning (ML) pipeline because it ensures that the
input data is in a suitable format for modeling, which helps improve the performance and accuracy of the
models. Some key reasons why data transformation is necessary in ML are given below.
• Feature Scaling: Feature scaling is a method used in data preprocessing for machine learning that
involves adjusting the range of the features in the data. This technique is essential because many
machine learning algorithms perform better or converge faster when features are on a relatively
similar scale and close to normally distributed.
Different features in the dataset might have different units and scales. For example, age might
range from 0 to 100, while salary might range from thousands to crores. Algorithms that rely on
the distance between data points, like k-nearest neighbors (KNN) and support vector machines
(SVM), can be biased towards features with larger scales. Transformations like Min-Max scaling
or Standardization help normalize the data, ensuring that each feature contributes equally to the
model's predictions.
• Handling Skewed Data: Many machine learning algorithms assume data is normally distributed.
If the data is skewed, transformations like logarithmic, square root, or Box-Cox can help reduce
skewness, making the patterns in the data more interpretable and accessible to the model
• Encoding Categorical Variables: Machine learning models generally work with numerical data.
Categorical data, such as gender (Male/Female) or state names, need to be converted to numerical
formats using techniques like one-hot encoding or label encoding. This conversion allows
algorithms to process the data effectively.
• Feature Extraction: Transforming data can help in extracting more meaningful features which
might not be captured directly from raw data.
• Improving Model Performance: Proper data transformation can lead to better model performance.

Common Methods of Data Transformation

1. N orm alization :
• Normalization is a type of data transform ation thatscales the values of numerical features
to a standard range, typically betw een 0 and 1.
• It helps in bringing all features to a similar scale, preventing certain features from
dominating the model due to their larger magnitude.
• Normalization involves scaling num erical features to a standard range, like transforming
house prices from 100,000 to 2 0 0 ,0 0 0 to a range between 0 and 1. By normalizing data,
features with different scales, such as square footage and price, are brought to a common
scale for fair comparison.
2. S tan d ard izatio n :
• Standardization is another data transform ation technique th at centers the data around a
mean of 0 and scales it to have a standard deviation of 1. It is like converting heights and
weights to z-scores.
 • It m akes the data follow a standard normal distribution, which can be beneficial for
algorithm s that assume normally distributed data.
3. Log Tran sform ation :
• Log transformation is applied to skewed data, like converting income values to their
logarithm ic form to handle extrem e values.
• It helps in making the data more symmetrical and reducing the im pact of extreme values,
especially in positively skewed distributions.
4. Encoding Categorical Variables :
• Converting categorical variables into numerical representations through techniques like
one-hot encoding or label encoding is a form of data transform ation.
- One-hot Encoding: Creates a new binary column for each category level.
- Label Encoding: Assigns a unique integer based on the alphabetical ordering of
the categories.
• One-hot encoding transforms categorical variables into binary values, such as converting
"color" categories like red, blue, and green into Os and Is.
• It allows categorical data to be used in machine learning models that require numerical
input.
• E xam p le : Converting categorical columns to numerical columns
Gender Male Female
Male 1 0
Female 0 1
Male 1 0
Male 1 0
Female f 0 1
Male 1 0
Female 0 1
Female 0 1

Categorical feature with Categorical feature

two categories: "Male" and converted to two binary
"Female" features: one per category

ySy Example Data Transformation

Let's consider a dataset containing the following information about students' exam scores in two subjects:
Math and English. The Math Score ranges from 0 to 100, while the English Score ranges from 0 to 50.
• Dataset: Student ID, Math Score, English Score
Student ID: 1,2,3,4, 5
Math Score: 85 ,7 0 ,9 0 ,6 5 ,8 0
English Score: 40,30,45,25, 35
m

Machine Learning^

- o b L cav e: [Link] fte exam scores « . a scale between 0 and 1 for bofl. Matt, and Engllsl, scores.

Data T r a n s f o r m a t i o n Steps: n i
. Feature Scaling: Nonnaltee the Math Score and English Score values to a range bet»aen 1.
To normalize the scores, we can use a simple formula;
N orm alized Value = (Value - Min Value] / (Max Value - Min Value)

• Normalized Math Score:

(85 - 65) / (90 - 65) = 0.75
(70 - 65) / (90 - 65) = 0.25
(9 0 -6 5 )/ (9 0 -6 5 ) = 1.00
(6 5 -6 5 )/ (9 0 -6 5 ) = 0.00
(80 - 65) / (90 - 65) = 0.50
. Normalized English Score:
(4 0 -2 5 )/ (4 5 -2 5 ) = 0.75
(3 0 -2 5 )/ (4 5 -2 5 ) = 0.25
(4 5 -2 5 )/ (4 5 -2 5 ) = 1.00
(2 5 -2 5 )/ (4 5 -2 5 ) = 0.00
(35 - 25) / (45 - 25) = 0.50 . „
. Result: After normalization, the Math and English scores are transformed to a Mie between an ,
ensuring that both scores are on a similar scale for analysis and modehng purposes.----------------------- -

A Pyth o n Code to Transform DategJ;. ‘ ------------- . ____ .

Example
import pandas as pd
fron [Link]»pose import ColumnTransformer
from s k l e a r n . p r e p r o c c s s i n g import standardscaler. OneHotEncoder
from [Link] import Pipeline

# Create a sample DataFrame before transformation

data = {'Age'; [35, 28, 45],
•Gender': ['Male', 'Female', 'Male ],
■Income': [50000, 60000, 55000],
■Region': ['East', 'West', 'North']}

df = [Link](data)
print("DataFrame before transformation: )
print(df)

, Define transformations for numerical and categorical columns

num_pipeline = Pipeline([
('scaler', StandardScalerO)

])
 ■r
vPahM^repflwition

cat_pipeline = Pipeline([
('encoder'j OneHotEncoderO)
])

fulljaipeline = ColumnTransformer([
('num', num_pipeline, ['Age', 'Income']),
('cat', catjjipeline, ['Gender', 'Region'])

])

transformed_data = fullj 3ipeline.fit_transforin(df)

# Create a DataFrame with transformed data for display

transformed_df = [Link](transformed_data, columns=['Scaled_Age', 'Scaled_
income', 'Gender_Female', 'Gender_Male', 'Region_Easf , 'Region_North', 'Region,
West'])
print("\nDataFrame after transformation:")
print(transformed_df) _____

SSIiSIIISSl8
DataFrame before transformation:
Age Gender Income Region
0 35 Male 50000 East
1 28 Female 60000 West
2 45 Male 55000 North

DataFrame after transformation:

Scaled_Inconie Gender_Female Gender_Male Region_East Region_North Region_West
Scaled Age
0.0 1.0 1.0 0.0 0.0
0 -0.143346 -1.224745
1.0 0.0 0.0 0.0 1.0
1 -1.146764 1.224745
0.0 1.0 0.0 1.0 0.0
2 1.290110 0.000000

Explanation
In the above code, a sample DataFrame is created with columns for 'Age', 'Gender', 'Income', and 'Region'. The
data is then transformed using ColumnTransformer with separate pipelines for numerical and categorical
columns.
. Numerical Pipeline (StandardScaler): The num_pipeline uses StandardScaler to standardize the
numerical features 'Age' and 'Income'. Standardization involves centering the data around the mean
and scaling to unit variance.
. Categorical Pipeline (O neHotEncoder): The cat_pipeline uses OneHotEncoder to encode the
categorical features 'Gender' and 'Region' into binary vectors. This process converts categorical
variables into a format suitable for machine learning algorithms.
. ColumnTransformer (Full Pipeline): The fulLpipeline combines the numerical and categorical
_______ pipelines to apply the transformations to the respective columns in the DataFrame.----------------------
 m m

Transformed Data Display: The transformed data is stored in transformed_data and then converted
into a new DataFrame [Link] with columns for standardized 'Age' and 'Income' as well as one-
hot encoded 'Gender' and 'Region'.

Output Interpretation:
• After appl}ang the transformation process using ColumnTransformer with StandardScaler for numerical
features and OneHotEncoder for categorical features, the data is transformed as follows:
- [Link]: Represents the standardized (scaled) values of the 'Age' column.
- Scaledjncome: Represents tiie standardized (scaled) values of the 'Income' column.
- [Link] and [Link]: One-hot encoded representation of the 'Gender' column
where 'Female' and 'Male' are encoded as binary values.
- [Link], [Link], and [Link]: One-hot encoded representation of the 'Region'
column where 'East', 'North', and 'West' are encoded as binary values.

• Individual 1:
- Scaled_Age: -0.143346
- Scaledjncome: -1.224745
- Gender. Male (encoded as 0.0 for Female and 1.0 for Male)
- Region; East (encoded as 1.0 for East, 0.0 for North, and 0.0 for West)

• Individual 2:
- Scaled_Age: -1.146764
- Scaledjncome: 1.224745
- Gender: Female (encoded as 1.0 for Female and 0.0 for Male)
- Region: West (encoded as 0.0 for East, 0.0 for North, and 1.0 for West)

• Individuals:
- Scaled_Age: 1.290110
- Scaledjncome: 0.000000
- Gender: Male (encoded as 0.0 for Female and 1.0 for Male)
- Region: North (encoded as 0.0 for East, 1.0 for North, and 0.0 for West)
This transformation process standardizes numerical features and converts categorical features into a format
suitable for machine learning algorithms, ensuring that the data is appropriately prepared for model training
and evaluation

m m p b a t a Reduction
Data reduction is a critical step in preparing data for efficient analysis, especially in contexts involving
large datasets or complex models. The process of data reduction involves diminishing the amount of
data that needs to be processed and analyzed without significantly sacrificing valuable information.
 ’ 3^^;''^'. • Data Prepar

Why Dafa R ed u ction is Important in ML?

The main reasons why data reduction is important in data processing and machine learning are listed
below;
• Improves Efficiency: Reducing the size of the data set can significantiy decrease the computational
resources required for processing. This leads to faster training times for machine learning models
and quicker execution of data analysis tasks.
• Reduces Storage Requirements: By minimizing the data volume, data reduction techniques
help in lowering storage space requirements. This is particularly important for businesses or
applications where data storage costs are a concern.
• Enhances Model Performance: In machine learning, reducing the number of input features
(dimensionality reduction) helps in removing irrelevant or redundant features, which can improve
the model's accuracy and performance. Techniques such as Principal Component Analysis (PCA)
and feature selection are commonly used to achieve this.
• Mitigates Overfitting: Overfitting occurs when a model learns not only the valid patterns but also
the noise in the training data. By reducing the number of features or the complexity of the data, the
risk of overfitting is reduced, making the model more generalizable to new, unseen data.
• Simplifies Data Visualization: Reducing the number of dimensions or the volume of data can
simplify data visualization, making it easier to identify patterns and trends. Visualizing fewer
variables or data points can help in drawing more meaningful conclusions without the distraction
of noise.
• Improves Data Quality: Data reduction can help in improving the quality of data by focusing
on the most relevant attributes. This can be particularly fmportant in scenarios where the data
includes irrelevant or extraneous information that could lead to poor decision-making.
• Cost-effective Data Management Managing large volumes of data can be costly, not just in terms
of storage, but also in terms of the computational cost required for data processing and analysis.
Data reduction helps in managing these costs more effectively._________________

Com mon Methods o f D ata Reduction:

1. Feature S e le ctio n : Feature selection involves choosing a subset of relevant features from
the original d ataset while discarding irrelevant or redundant features. Feature selection
simplifies the model, enhances interpretability, and reduces overfitting by focusing on the
most informative features.
2. Feature E x tra ctio n : Feature extraction transform s the original features into a lower­
dimensional space to capture essential information.: Feature extraction reduces dimensionality
while preserving key variance and discriminative information.
3. Instance S e le c tio n : Instance selection involves choosing a subset of representative instances
from the dataset w hile maintaining overall data characteristics. Instance selection speeds up
training, enhances model generalization,^and-reduces-Storage-requu:ements.
4. D im ensionality R ed u ctio n : Dimensionality reduction aims to reduce the number of input
variables or features in the dataset Dimensionality reduction simplifies the model, improves
 computational efficiency, and enhances model interpretability and generalization. Techniques
like PGA, LDA, t-SNE, and autoencoders are commbhly used for dimensionality reduction in
machine learning.
r Example Data Reduction

A dataset contains information about students, including name, age, date of birth, exam scores, study hours,
extracurricular activities, and academic performance. The goal is to predict student grades based on these
features.
Select exam scores, study hours, and extracurricular activities as the most influential features for predicting
student grades. By selecting key features, such as exam scores and study hours, the dataset is reduced to
essential predictors. The simplified dataset improves model performance, interpretability, and efficiency in
predicting student grades. The streamlined dataset accelerates model training and enhances decision-making
for academic performance analysis.______________ _________________________________ ______________ ______

Example I A Py4;hon Code to Demonstrate Data Reduction

import numpy as np
import pandas as pd
from [Link] import make_classification
from sklearn.feature_selection import SelectKBest, f_classif
from [Link] import PCA
from [Link] import StandardScaler

# Create synthetic student data

[Link](42)
data = [Link]({
'StudyHours': [Link](5, 2, 100),
'Attendance': [Link](8, 2 , 100),
'Participation': [Link](100) * 10,
'ProjectScore': [Link](100) * 100,
'TestScores': [Link](100) * 100,
'Assignments': [Link](100) * 100

})
# Binary target based on some condition
data['Pass'] = (data['TestScores'] + data['Assignments'] + data['ProjectScore'] / 3
> I50).astype(int)

# Feature scaling
scaler = StandardScaler()
fe a tu re s_ sc a le d = scaler.fit_transform([Link]', axis=l))

# Feature selection using SelectKBest with ANOVA F-test

selector = SelectKBest(score_func=f_classif, k=3) # Choosing the top 3 features
selected_features = selector.fit_transform(features_scaled, data['Pass ])___________
# Identify selected feature names
feature_names = [Link][;-l] # Excluding the target variable 'Pass'
selected_feature_names = feature_names[selector.get_support()]

# PCA for dimensionality reduction to reduce to 2 principal components

pea = PCA(n_components=2)
features_pca = pca.fit_transforro(features_scaled)

# Print results
printC'Original Data Shape:", [Link]( 'Pass', axis=l).shape)
print("Data Shape After Feature Selection:", selected_features.shape)
print("Selected Features:", selected_feature_names.tolist())
print("Data Shape After PCA:", features_pca.shape)

Original Data Shape: (100, 6)

Data Shape After Feature Selection: (100, 3)
Selected Features: ['ProjectScore', 'TestScores', 'Assignments']
Data Shape After PCA: (100, 2)_________ _______________________ —
‘Explanation
This Python script is designed to demonstrate data reduction techniques applied to a synthetic dataset
representing student performance metrics. It covers feature scaling, feature selection, and principal component

analysis (PCA).
1. imports an d Data Creation : The script starts by importing necessary libraries: numpy, pandas,
and several modules from scikit-learn. It then generates a synthetic dataset with 100 observations of
student data, including various performance metrics like study hours, attendance, and scores. A binary
target (Pass) is computed based on a custom formula to simulate pass/fail outcomes based on test
scores, assignments, and project scores.
2. Feature Scaling : All features are scaled using StandardScaler, which normalizes the data to have a
mean of zero and a standard deviation of one. This is an important preprocessing step, especially for
PCA and many machine learning algorithms that are sensitive to the scale of the input data.
3. Feature Selection : SelectKBest with ANOVA F-test ([Link] is used to select the top 3 features that
have the highest statistical significance in relation to the target (Pass). This method evaluates each
feature's influence on the target variable and picks the most influential ones.
4. PCA for Dimensionality Reduction : PCA is applied to the scaled data to reduce its dimensionality
’ to 2 principal components. This step transforms the data into a new coordinate system, reducing the
number of features while attempting to keep the most significant variance in the data.

Output I n te r p F e te itio n :
. Original D ata Shape: (1 0 0 ,6 ): The original dataset consists of 100 samples, each witii 6 features. These
features include StudyHours, Attendance, Participation, ProjectScore, TestScores, and Assignments
This is the full dataset before any transformations or reductions are applied..-----------------------------
 Machine learning . -V ‘

. Data Shape After Feature Selection: ( 1 0 0 ,3 ) : After applying feature selection using the SelectKBest
method with an ANOVA F-test, the number of features in the dataset has been reduced to 3. The dataset
still contains 100 samples, indicating that no data points were removed—only features were reduced
This reduction focuses on retaining only the most statistically significant features in relation to the
target variable (Pass). This helps in simplifying the model, potentially improving model performance
by reducing overfitting, and decreasing the computational load for further processing.
• Selected Features: ['ProjectScore', 'TestScores', ’A ssignm ents']: The three features selected as most
relevant are ProjectScore, TestScores, and Assignments. These were determined to have the strongest
statistical relationship with the student's ability to pass (as defined by the target variable).
This output provides insight into which factors are most influential for student success in this sjmthetic
dataset. For instance, how well a student performs in projects, tests, and assignments are key indicators
of their likelihood to pass, according to the model.
. Data Shape After PCA: (1 0 0 , 2 ) : After applying PCA, the dimensionality of the dataset is further
reduced to just 2 principal components from the originally scaled 6 features. This transformation
results in a new dataset that still has 100 samples, but now each sample is represented by only 2
derived features.
PCA helps in reducing the dimensionality while attempting to preserve as much of the data's variability
as possible. These 2 principal components capture the essence of the dataset's information, reducing
the complexity and enhancing computational efficiency. This is particularly useful for visualization,
further analysis, or as input into machine learning algorithms that may perform better with lower­
dimensional data.

F e a tu r e E n g in e e rin g
Feature engineering is a fundamental process in the field of m achine learning where raw data is
transform ed into formatted datasets th at machine learning algorithm s can work with more effectively.
This process involves creating new features from existing data, transforming data into m ore useful
formats, or enhancing the quality of data to improve the accuracy and efficiency of predictive models.

Key Components of Feature Engineering

1. F eatu re Creation: This involves creating new variables from existing data to provide
additional insight to the m odels. This might involve com bining features, deriving new metrics
from existing data, or aggregating data over time or space.
Exam ple : For a dataset containing student attendance and grades, creating a feature that
represents the average grade over the past three tests m ight predict future perform ance better
than individual test scores.
2. F eatu re T ran sfo rm atio n : Transforming features to enhance their predictive power or making
them more suitable for models. Common transform ations include normalization, scaling,
appljang mathematical functions like logarithms or exponentials, and more.
Exam ple: In a student dataset, transforming the 'StudyHours' feature from raw hours to
categories such as 'Low', 'Medium', and 'High' based on defined thresholds (e.g., 0-3, 4-6, 7+
hours) can sometimes provide clearer signals for predicting student performance.
 Data Preparafi

{ m

Why Feature Engineering is Important in ML?

The main reasons why feature engineering is important in data processing and machine learning are listed
below:
• Improves Model Perform an ce: Well-engineered features provide a better representation of
patterns in the data, improving model accuracy and performance.
• Reduces Model Complexity: By effectively capturing the underl3dng signals in the data, simpler
models can be used, or models can converge faster on the optimal solution.
• Enhances Data Interpretability: Good features can help to understand the influence and relation
of variables to the prediction outcomes, providing insights into the process.
• Adaptability Across Various Models: Effective feature engineering can make a dataset more
adaptable across different t)T3es of machine learning models, potentially leading to better
performance without changing the underlying algorithms.______________________________________

Example 1^ Python Code to Demonstrate Featiire'Creation and Feature Transformation

import pandas as pd
import numpy as np

# Create a synthetic dataset

data = [Link]({
'StudentID': range(l, 181),
'StudyHours': [Link](5j 2 , 100), # Average 5 heurs, std deviation 2
■SleepHours’: [Link](7, 1.5, 100), # Average 7 hours, std deviation 1.5
■ExerciseHours': [Link](3, 1, 100), # Average 3 hours, std deviation 1
'GPA': [Link](3, 0.5, 100), # GPA out of 4, mean 3, std deviation 0.5
’FinalExamScore’: [Link](60, 100, 100) # Final exam scores

})

# Normalize the lifestyle data to a common scale

data['NormalizedStudyHours'] = [Link](data['StudyHours'], bins=3, labels=[l, 2, 3])
data['NormalizedSleepHours'3 = [Link](data['SleepHours'], bins=[0,6,8,[Link]], labels=[l, 2, 3])
data['NormalizedExerciseHours'] = [Link](data['ExerciseHours'], bins=3, labels=[l, 2, 3])

# Create Health Index from normalized sleep and exercise hours

data [' Healthlndex']= (data [' NormalizedSleepHours']. astype(int) + data [' NormalizedExerciseHours'].
astype(int)) / 2

# Adjust GPA based on study hours; assuming more study hours slightly improves GPA
data['AdjustedGPA'] = data['GPA'] + (data['NormalizedStudyHours'] .astype(int) - 2) * 0.1

# Display the enhanced dataset

print(data[['StudentID', 'StudyHours', 'SleepHours', 'ExerciseHours', 'GPA', 'AdjustedGPA',
'FinalExamScore', 'Healthlndex']].head())
 StudentID StudyHours SleepHours ExerciseHours GPA AdjustedGPA FinalExamScore Healthlndex
3.048015 2.857966 2.957966 87 1-5
1 8.699350 5.004007
3.454703 3.454703 71 2.5
2 3.906547 7.281035 3.685460
3.126315 63 2.0
; 3 4.824894 7.384568 3.464725 3.126315
2.511550 81 2.5
1 4 5.893448 7.064281 4.293207 2.511550
3.612583 89 1.5
t 5 5.072490 5.258506 3.431551 3.612583
ipiitiiutiuu
Explanation ____________ ______ _______ _____________ -—---------------
1. Data Generation: The script begins by c a S n g a syi,the«c dataset of 100 students with features like
Study hours, sleep hours, exercise hours, GPA, and final exam scores.
2. Normalization: The StudyHours, SleepHours, and ExerciseHours are normalized to a scale of 1 to 3 to
standardize these features for better comparison and integration.
3. Feature Creation: The Healthlndex is computed as the average of the normalized scores of SleepHours
and ExerciseHours, encapsulating overall health and wellness.
4. Feature Transformation: The AdjustedGPA is calculated by adjusting the original GPA based on the
amount of study hours, hypothesizing that more hours of study could reflect a slight improvement m
academic performance.
5. Output The script prints the first few rows of the transformed dataset to provide an overview of the
newly created and transformed features alongside the original data.---------------------------- -----------

Data Spitting
Data splitting in machine learning is the process of dividing the data into separate subsets to be used
at different stages of model building and evaluation. The prim ary goal of data splitting is to ensure
that the model trained on one set o f data can generalize well to new, unseen data. This helps avoid
problems like overfitting, w here a model performs well on the training data but poorly on new data.

1. Training Data: This is the largest portion of the d ataset and is used to train the model. The
model learns to identify patterns and make decisions based on this data. Typically, about 70-
80% of the entire dataset is allocated to the training set.
2. Validation D ata: This subset is used to tune the model's hyperparameters and make decisions
’ about which models o r configurations to use. It acts as a check to avoid overfitting on the
training dataset Typically, about 10-15% of the dataset is reserved for validation.
3. T est Data: This is used to evaluate the final model’s performance after it has been trained and
validated. T h e te st set should be a completely independent dataset that the model has not seen
during training or validation. This helps provide an unbiased evaluation of how w ell the model
is expected to perform in the real world. Typically, about 10-15% of the d ataset is used as the
test set.
 M e t h o d s o f D ata Sp littin g

1. Random Splitting: This is the most common method where data points are randomly
assigned to the training, validation, and test sets. This method assumes that all data points are
independent and identically distributed.
2. Stratified Splitting: In scenarios where the dataset includes categories that are unevenly
distributed, such as in classification problems with imbalanced classes, stratified splitting
ensures that each class is proportionately represented in the training, validation, and te st sets.
This helps in building a model that is fair and has learned adequately from all classes.
3 . T im e-based Splitting: For tim e-series data, where tem poral patterns and dependencies are
important, data is split based on time. For example, the model may be trained on data from the
past year, validated on the following month, and tested on the month after th a t

Why Data Spitting is important in ML?

The main reasons why data spitting is important in data processing and machine learning are listed below:
• Avoiding Oveifitting: When a model is trained extensively on a particular set of data, there is a
risk that it learns the noise and specific details of the training data to an extent that it negatively
impacts the performance on new data. By using separate training and testing datasets, it is possible
to minimize the risk of overfitting.
• Model Validation: Data splitting allows for a validation set that can be used to fine-tune model
parameters (hyperparameters). This process Is essential f jr identifying the best model settings
because it prevents tweaking the model based on the test set, which could lead to biased assessments
of its effectiveness.
• Assessing Model Performance: A test set, separate from the training data, provides an unbiased
evaluation of a final model’s performance. This is critical for understanding how a model is likely to
perform in practical scenarios, ensuring that the evaluations reflect true predictive performance on
unseen data.
• Improving Model Robustness: By training a model on a diverse training set and validating it on
different subsets of the data, the robustness and reliability of the model are enhanced. Data splitting
ensures that the model can handle variations in the data and accurately predict outcomes across a
range of scenarios.
• Effective Tuning and Comparison: Data splitting facilitates rigorous comparisons between
different models and configurations under consistent conditions. Each model is given the same
opportunity to learn from specific data and validated and tested on identical sets, making
comparisons feir and decisions more informed.

Exam ple A Python Code to Spit the Data

import numpy as np
from sklearn.model_selection import train_test_split

# Example data: features and labels

# Let's assume these features could be study hours and grades, and labels are pass/fail indicators
X . [Link]([tie, 80^ ^ 7 3 1 . [2. 401. [IS. 9^, [10, ee], “ 1-
70], [13, 76]])
y . „[Link],y([l, 1, e, 1, 0, 1, e. 1, 1, H ) • 1 f " !■’«■ »

# split the dataset Into training (60%), validation (20%), and test (20%)
, First, split into training and test
[Link], [Link], [Link], y_test ■ train_test_split(x, y. [Link]

. ....................
state=42) # 0.25 x 0.8 = 0.2

# Printing the sizes of each dataset

printC'Training set size: ", len(X_train))
print("Validation set size; ", len(X_val))
print("Test set size; ", len(X_test))

# Displaying the actual sets

print("Training Features and Labels;")
print(X_train, y_train)
print("Validation Features and Labels;")
print(X_val, y_val)
print("Test Features and Labels;")
print(X test, y_test)^______________ _
............ .............................V. • ^

Training set size: 6

Validation set size: 2
Test set size: 2
Training Features and Labels;
[[16 85]
[ 9 55]
[20 85]
[13 76]
[ 2 40]
[15 90]] [ 1 0 1 1 0 1]
Validation Features and Labels:
[[10 80]
[10 60]] [1 0]
Test Features and Labels;
[[15 70]
[ 9 70]] [1 1]
 Data Preparation ^ 2.49

Explanation
1. Data P re p a ra tio n : The array X con tain s h)q)othetical pairs o f features Qike study hours and
grades), w hile the array y contains b inary labels indicating pass ( 1 ] or fail ( 0 ).
2. Data S p iittin g :
• The data is initially divided into training (80% of the original data] and test sets (2 0 %
o f the original data) using train_test_split.
• The training data is further sp lit to carve out a validation set, taking 25% of the training
data (which constitutes 2 0 % o f the original dataset). The proportions are managed
to ensure the final split percentages are maintained as intended (60% training, 2 0 %
validation, 20 % test).
3. Output:
• The code prints the sizes of each dataset to confirm the splitting proportions.
• It also prints the actual training, validation, and test sets with their respective features
and labels.

Select and Train a Model

The process of selecting and training a m achine learning model is a critical phase in a project, w here
the prepared data is utilized to create a m odel capable of making predictions based on new inputs. This
phase involves experimenting with various algorithms, evaluating th eir performance, and optimizing
the final model for deplo)onent. The objective is to ensure that the model not only performs well on
the training data but also generalizes effectively to unseen data, m aking it applicable in real-world
scenarios.

Process of Selecting and Training a M achine Learning Model

Step 1: M odel S e le c tio n

The choice o f machine learning m odel is crucial and depends on the nature of the problem
at hand. Understanding the problem type, whether it involves regression, classification,
clustering, o r other tasks, is essential for selecting the m ost appropriate model that can
effectively address the specific requirem ents and characteristics o f the data. For a task like
predicting student performance (nu m eric score prediction), regression models are typically
suitable.
E x a m p le : Predicting student perform ance, a regression task, could start with simpler models
like linear regression but may require more complex models like Random Forest Regressors
if the relationships between features and target are non-linear. Given the initial analysis
suggesting non-linear patterns, we o p t for a Random Forest Regressor due to its ability to
handle com plex data structures and provide robustness against overfitting.
Step 2: M odel T ra in in g
Model training is a critical step in the machine learning pipeline where the selected model is
exposed to the prepared dataset to learn patterns and relationships between the input features
and the targ et variable. During this phase, the model adjusts its internal parameters based on
the training data to minimize the prediction error and improve its performance.
 E x a m p le : The Random Forest model is trained using features su d i as study
records and liistorical grades. T his model does not require setting many h w e r p a r a m ^
initially but does involve decisions about the number of trees and depth, which we initially set
to defaults for a baseline model.

Step 3 : Model Evaluation ' , . . j

The model's performance is assessed using a validation set, a subset o e ^
th at the model has not seen before. This evaluation helps in evaluating the model learning
capability and its ability to generalize to new data.
Example : Evaluate the initial Random Forest model by calculating its RMSE on a validation
“ e perfonnance is u n satisfacto^, it suggests the need for mning hyperpaiameters or
possibly revisiting the feature engineering step.
Step 4 : H yperparam eter Tuning
Hvoeroarameters are param eters that ate set before the learning process begins. They control
r h ? r e « ^ c e s s and model behavior but are not learned from the data. Examples ■nclude
learning rate, regularization strength, number of hidden layers in a neural network, and kernel
type in Support Vector Machines. .
F iL -tu n in g the model's hyperparam eters is crucial to enhance its
like grid search or random search can be employed to systematically explore different

like the number o f trees or tiee depth in a random

S w e l t t r o u g h ^ d s L c h can help minimize RMSE on the validadon set, improving

model accuracy.

^ ^ “ l u r e s the m bders stabiiity across various data sutaets. B y ^ re p e a t^ y

splitting the data into training and validation sets, training on each subset, and averaging
results, the model's robustness is assessed. j

E x a m p le : Implementing 10-fold cross-validation on a random forest

the training data into 1 0 subsets, using each as a validation set once, and M in in g on
remaining 9 subsets. The average RMSE across all validations provides a reliable perform an

estimate.
Step 6 : Final Model Training
After Identifying the b e st model and hyperpararaeters, the final model is trained on the en
training dataset to leverage all available data for optimal learning.
E x a m p le : The optimized random forest model, with fine-tuned
validation, is trained on the com plete student dataset to maximize its predictive capabilities.

Step ja a s e t not used during training or validation to

.nvolves predicting exam scores for

" t o ^ L e d on th eir study h .bi.s'ai.d past p erfonnance. The model's predictions are
compared against actual scores to caWlaee'the final RMSE, assessing its effectiveness.
 Data Preparation

Example 1 A Pji;hoii Code to Select and Train the Model -;Linear Regression-Model
Objective : The objective of this program is to demonstrate the use of a Linear Regression model to
predict students' performance based on the number of study hours. By training the model on dummy data
representing study hours and corresponding scores, the program aims to provide users with a tool to input
study hours and receive a predicted score within the valid range of 0 to 100. This program serves as a simple
educational example to showcase the prediction capabilities of a machine learning model in the context of
student performance prediction.

# Im port n ecessary l i b r a r i e s
im port numpy as np
from sk le a rn .lin ear_m o d el im port L inearR egression

# Dummy d a ta fo r dem onstration

stu d y _h o u rs = n p .a r r a y ( [ l , 2 , 3 , 4 , 5 ] ) .r e s h a p e (- l , 1 ) # Input f e a tu r e : study hours
s c o r e s = n p .a r r a y ([5 0 , 6 0 , 7 0 , 8 0 , 9 0 ] ) # Output: co rresp on d in g sco res
# S e l e c t and t r a i n a L in ear R eg ressio n model
model = L in earR egression O
m o d [Link](stu d y_h o u rs, s c o r e s )

# A cce p t input from th e u se r and p re d ic t the outcome

w hile T ru e:
try :
user_inp ut = f lo a t(in p u t(" E n te r the number o f stu d y hours: " ) )
i f user_input < 0 :
p rin t("S tu d y hours can not be n e g a tiv e . P le a s e e n te r a v a lid num ber.")
e ls e :
p red ietfid_score = model. p r e d i c t ( [ [ u s e r _ i n p u t ] ] )
# Ensure sco re i s between 0 and 100
p red icted _sco re = m ax(0, m in (p re d ic te d _ s c o re [0 ], 1 0 0 ))
p r i n t ( f “P re d icte d s c o r e f o r {u s e r_ in p u t} stud y h ou rs: { p r e d i c t e d _ s c o r e } " )
e x c e p t V alueError:
p r in t(" P le a s e e n te r a v a li d number o f study h o u r s ." )
e x c e p t K eyboardlnterrupt:
print("\nProgram t e r m in a te d .”)
break
Output
E n te r t h e number o f study h o u rs: 4
P r e d ic te d sco re f o r 4 .0 study h o u rs: 8 0 .0
E n te r t h e number o f study h o u rs: 3 .5
P r e d ic te d sco re fo r 3 .5 stu d y -h o u rs : 7 5 .0
E n te r th e number o f study h o u rs: 5
P r e d ic te d sco re fo r 5 .0 study h o u rs: 9 0 .0
E n te r t h e number o f study h o u rs: 6
P r e d ic te d sco re fo r 6 .0 stud y h o u rs: 1 0 0 .0
 ■ ~ ■
. • ■- 'I.

Enter the number of study hours: 1.5

Predicted score for 1.5 study hours: 55.0
Enter the number of study hours; -1
study hour s cannot be negative. Please enter a valid number.

model to predict students' performance based on the number of study hours.

1 Dummy DaU: Dummy data is created for demonstration purposes.
' feature (study hours), and'stores'represent the correspondmg output (student SCO ).

2 Select and Train Model: A Unear Regression model Is selected and trained on the dummy data [Link]
' the 'fitO' method. The model learns the relationship between study hours and scores.
3 Accept U s e r I n p u t a n d Predict O u t c o m e :
. The code enters a loop «here It prompts the user to Input the number of study hours.
. lftheuserlnputlsnegative,am [Link],ve.

. The model then predlctsthe student's score based on the input study hours using the [Link]

. TheTedlcted score Is constrained to be between 0 and 100 using the 'm a.0' and 'n-inO'
functions.
Finally, the program displays the predicted score for the input study houj;s.

■ » n i $ n i ® * A Jrytnon ****** -------------- _________ — ---------- -------------

By
specifically a Random Forest R e a s o n p of bedrooms, bathrooms, square
training the model on a dataset that inclu es e specific housing details and

rc ra ^ tr— :^^^

Bedrooms,Bathrooms,SquareFootage,Location,Pnce

3,2,1500,1,3000000
4,3,2000,2,4500000
2,1,800,3,2250000
3,2,1600,1,3200000
4,3,2500,2,4750000
5,4,3000,3,5000000
3.2,1800,1,3500000 _______________ _
2,1,1000,3,2100000
3,2,1700,2,3300000
4,3,2400,1,4100000
2,1,850,3,2000000
5,4,2900,2,4800000
3,2,1400,1,3100000
4,2,2200,3,4300000
2,1,900,1,2150000
5,4,2800,2,4600000
3,2,1900,3,3400000
4,3,2100,1,4000000
5,3,2750,2,4950000
2,1,1100,3,2300000
3,2,1600,1,3150000
4,3,2300,2,4200000
5,3,2650,3,4850000
3,1,1300,1,2750000
4,2,2200,2,4150000
2,1,950,3,2050000
5,4,3100,1,5000000
3,2,1450,2,3250000
4,3,2350,3,4450000
2,1,1200,1,2400000

Step 2: Write a Python code to read the CSV file, select the model and train the model to predict the house
price. Save this file as "[Link]"

import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from [Link] import RandomForestRegressor
from [Link] import mean_squared_error

# Load the data from the CSV file

csv_file = 'housing_data.csv'
data = pd.read_csv(csv_file)

# Split the data into features and target

X = data[['Bedrooms', 'Bathrooms', 'SquareFootage', 'Location']]
y = data['Price'] __________________________
# Split the data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random.
state=42)

# Initialize and train the RandomForestRegressor

model = RandomForestRegressor(n_estimators=100, random_state=42)
[Link](X_train, y_train)

# Evaluate the model

predictions = model. predict(X_test)
mse = mean_squared_error(y_test, predictions)
rmse = [Link](mse)
print(f"Model trained and evaluated. RMSE on test set: {rmse:.2f}")

# User Input and Prediction

# Prompt user for property details and predict the price
print("\nEnter property details to predict the price.")

bedrooms = int(input("Enter the number of bedrooms: "))

bathrooms = int(input("Enter the number of bathrooms: "))
square_footage = int( input ("Enter the square footage of the house: "))
location = int(input("Enter the location code (1, 2, or 3): "))

# Create a DataFrame for the input features with appropriate column names
user_features = [Link]({
'Bedrooms': [bedrooms],
'Bathrooms': [bathrooms],
'SquareFootage': [square_footage],
'Location': [location]
})

# Predict using the model

predicted_price = [Link](user_features)[0]
print(f"Predicted price for the house: Rs.{predicted_price:, .2f}")

C:\ML_Projects>python [Link]
Model trained and evaluated. RMSE on test set: 252114.48

Enter property details to predict the price.

Enter the number of bedrooms: 3
Enter the number of bathrooms: 2
Enter the square footage of the house: 1500
Enter the location code (1, 2 , or 3): 1
Predicted price for the house: Rs.3,034,458.33__________
w

T^^^Da^Praparafjon
1 0
C:\ML_Projects>python [Link]
Model trained and evaluated. RMSE on test set: 252114.48

Enter property details to predict the price.

Enter the number of bedrooms: 5
Enter the number of bathrooms: 4
Enter the square footage of the house: 3000
Enter the location code (1, 2, or 3): 2
Predicted price for the house: Rs.4^932,500.00
Explanation
1. Data Preparation: The dataset comprising of 30 records is manually prepared with attributes that
typically influence house prices. These attributes include the number of bedrooms, bathrooms, the
square footage of the house, and its location (categorized as 1,2, or 3). This data is saved into a CSV file
named "housing_data.csv".
2. Model Training and Evaluation:
• Data Loading: The program starts by loading tiie data fi-om "housing_data.csv" using pandas
library.
• Feature-Tai^et Split: The data is split into features (X) and the target variable (y), where the
features include the bedrooms, bathrooms, square footage, and location, and the target is the
house price.
• Train-Test Split: The data is then divided into training and testing sets, with 80% of the data
used for training the model and 20% reserved for testing its performance. This split helps in
evaluating the model on unseen data, thus testing its ability to generalize.
• Random Forest Model: A RandomForestRegressor is initialized and trained on the training data.
This type of model is chosen for its efficacy in handling non-linear data and providing robust
predictions by averaging multiple decision trees trained on various sub-samples of the dataset
• Model Evaluation: After training, the model's performance is assessed on the test set using the
Root Mean Square Error (RMSE), which provides a measure of the average magnitude of the
model's prediction errors.
3. User Interaction for Real-Time Predictions:
• Input Prompt: Users are prompted to enter details about a property (bedrooms, bathrooms,
square footage, and location), mimicking a real-worid scenario where a potential buyer or seller
wants to get an estimate of a property's market value.
• Prediction: The entered details are transformed into a DataFrame with appropriate feature
names, ensuring consistency with the training data's structure. The model then uses this
information to predict the house price, which is displayed to the user.
4. Output and Utili^
• The program not only trains the model to predict house prices with a reasonable degree of
accuracy (as indicated by the RMSE) but also offers a direct application by allowing users to get
price estimates for specific properties based on their features.
• This makes the program a practical tool in real estate, providing insights into how various features
_________might impact the valuation of a property___________________________________
 R e v ie w Q u e s tio n s

1. W hat is the Meaning of Data in Machine Learning?

2. W hat is Labeled Data? Give an example.

3. What is Unlabeled Data? Give an example.
4. How do w e split data in Machine Learning?
5. W hat is Data Spitting?
6 . W rite the Common Types of Data Splits.

7. W hat is Training Data? Give an example.

8 . W hat is Validation Data? Give an example.

9. W hat is Testing Data? Give an example.

10. W hat is Data Preparation ?

11. What is Data Cleaning?
12. W hat is Data Transformation?
13. W hat is Data Reduction?
14. W hat isFeature Engineering?
15. W hat is Dimensionality Reduction?
16. W hat is Normalization and Standardization in Data Preparation?

17. W hat is Feature Transformation and Feature Selection?

18. W rite the Different Techniques for Test Set Creation.

19. Why Data Cleaning is Important in ML?

20. How to Encode Categorical Variables?

1. Explain the Categories of Data in Machine Learning.

2. W hat is the Importance and Benefits of Data Preparation?
3. Discuss the Data Preparation Issues in Machine Learning.

4. Explain Data Cleaning with Examples.

5. Explain Data Transformation with Examples.

6 . Explain Data Reduction with Examples.

7. Explain Feature Engineering with Examples.

8 . Discuss the Sources of Real-World Data.
 9. How to Load the Data and Explore the Data in ML?
10. How to C reate a Test Set?
11. Why Visualizing the Data is Needed During Data Preparation?
12. How to Handle Missing Values? Explain with an example.

13. How to Handle Outliers? Explain with an example.

14. Why Data Transformation is Im portant in ML?
15. Explain th e Common Methods of Data Transformation.
16. Why D ata Reduction is Important in ML?
17. W hat is Feature Engineering? Explain the Key Components o f Feature Engineering.
18. Why Data Spitting is Important in ML?
19. W hat is Data Spitting? Explain Common Types and Methods o f Data Splits.

Eight Marks Questions

1. Explain th e Steps in Data Preparation Process.

2. W hat is Data Collection ? Explain the Key Steps Involved in Data Collection.
3. W hat a re the advantages of Working with Real Data in Machine Learning Projects.
4. Explain th e Structured Approach in Implementing Machine Learning.
5. Explain th e Process of Getting the Data.
6 . W rite a Python Code to Create a Test S et in Python.

7. Explain Data Visualization Techniques.

8 . Explain th e process of handling m issing values and outliers in data preparation.

9. W rite a P)^hon Code to Transform Data.

10. Why Data Reduction is Important ? W rite the Common M ethods o f Data Reduction.
11. W rite a Python Code to Demonstrate Data Reduction.
12. W rite a Pj^hon Code to Demonstrate Feature Creation and Feature Transformation.

13. W rite a P 5^hon Code to Spit the Data.

14. Explain th e Process of Selecting and Training a Machine Learning Model.
15. W rite a Pj^hon Code to Select and Train the Model.
 .... ■ . . s:'‘.^<?y;,.

P M « P T T iP in a g

Introduction
Types o f Supervised Machine Learning
Some Sample Datasets
K-Nearest Neighbors (K-NN) Algorithm
Linear Models
Naive Bayes Classifiers
Decision Trees
Review Questions
M B Introduction
Supervised machine learning is .earn from S e s X t "s
learning. Supervised maclime leammg computer with labeled examples

r : . g ?.- r . = :r r ,r .

Supervised learning is the types of m a c h i n e learning in Tta

■labeled" training data, and on basis of that <Uta, ™ *m es pred.a

z ;r r d :2 T h f m “a c " S ^ ^ ^ ^
o t e Z l t applies the same concept as a .tudent learns in the superv,s,on of the teacher,
r m i t c h a p " we will describe supervised learning in more detail and explam several popular
supervised learning algorithms. __ _________
B E T Types r>f Supervised Machine Learning
“ e^sed machine learning aasslflcation and Regression are two h-ndamental types of tasks
_ ^.1 __ K oin cr n rp d irte d :

" Classification
\f Supervised | 1 1
1 Learning J■
Regression
I __
rlAc<; labels while regression tocuses on picuicuiiiB
Classification is about predicting dis •« -,tHnn and regression depends on the nature

X T —
H H || Classitlcation

A classification problem input

goal is to predict the c a t e g o n c a I l a b e ls ( c l a s s ^ o g P ^_^^^

i“re^rrfrors"Tnrd“ ^^^

based on data. >.

 ■ '' Sopervisedlearning

In classification problems, the goal is to p red ict the categorical class labels of new or unseen input
data based on past observations. The outpu t variable is a category, such as "spam" or "not spam" for
email classification, "male" or "female" for gen der classification or "cat," "dog," or "horse" for image
classification.

What is Classification in ML?

f t
Classification is a type of supervised learning where the goal is to categorize or classify input data
into predefined classes or categories based on their features. The algorithm learns from labeled
training data and predicts the class labels of new or unseen instances.
Example : The output variable is a category, such as "spam" or "not spam" for email classification,
"male" or "female" for gender classification or "cat," "dog," or "horse" for image classification.

Examples Classification Tasks

1. Classifying emails as either Spam o r Not Spam: Predicting whether an email is spam (class 1) or not
spam (class 0). The output variable is discrete, taking on two distinct values (0 or 1) representing the
two classes.
2. Classifying Images of Fruits : Classifying images of fruits into categories such as apple (class 0),
banana (class 1), or orange (class 2). The output variable is discrete, with multiple distinct values (0
or 1 or 2 ) representing the three classes (apple, banana, orange).
3. Sentiment Analysis: Analyzing text data to determine the sentiment of a review (positive, negative,
neutral). The sentiment labels (positive, negative, neutral) are discrete categories assigned to the input
text
4. Medical Diagnosis: Predicting the presence of a disease based on patient symptoms and test results.
The diagnosis categories (e.g., disease present, no disease) are discrete labels assigned to the patient
data.
5. Customer Retention: Predicting whether a customer will renew a subscription or not (churn
prediction).

Key C h aracteristics of a Classification

1. D iscrete Output Variables: The term "discrete" in the context o f machine learning refers to
a type of variable that has specific and separate values, as opposed to continuous variables,
which can take any value within a range. Discrete variables are countable and have distinct
categories or values, which cannot be subdivided meaningfully.
Exam ples o f Discrete Outputs :
• G ender Classification: Male, Fem ale
• Loan Approval: Approved, Rejected
• Movie Genre Classification; Action, Romance, Thriller, Comedy, Drama
2. Supervised Learning: Classification is a supervised learning approach, meaning it relies on
labeled training data to learn the relationship between input features and the target classes.
3-4

3 . Binary or Multi-Class distii^^ishing between

Example of Binary C lassification :

. Task; Determine w hether an emaU is spam.

. Data Input o r F eatu res: Text content of emails.

. Classes: Two classes - Spam and Not Spam.
Example of Multi-Class Classification: . u .onf
.TasteC lassilynew s articles into disdnct categories based on the.r content.

. natalnout or Features: Text content of news articles.

. aass« = Multiple news categoriessuch as PoUtlcs, Sportsjechnology, and Enter^m^^^^^^^

metrics like accuracy, precision, recall. F I score,

predictive power. _____ ^

tinds, and relationships within the data for in fo rm e d

r^ d T ffe rr^ b T ^ ^ ^ ^ r^ Z ^ ^ atu res. h heips in identi^ng

.^ ila ritie s a n d d iff e re n c e s in d a .^ —

L— n T ':“e T r e " on historical data. By leanu^g labeled [Link],

h” eU cTn predict the class labels of new or unseen data .nstances.
. . K .K A ssessm en t: In various industries ^ T ^ I^ in ; da^“

. P e ln a iu a tio n and R eco„[Link]«o„

^ e r r a r b e ^ r T l :^ ” —

e. r :f :d sp ee. - o ^ - =
 7. Healthcare and B iom ed ical R esearch: Classification algorithms are used in healthcare for
disease diagnosis, patient risk stratification, and medical image anafysis. By classifying medical
data, healthcare professionals can make accurate diagnoses and treatment decisions.
8 . Customer Segm en tation: Businesses use classification to segment customers into different
groups based on demographics, behavior, or p references. This segmentation helps in tailoring
marketing campaigns, improving custom er satisfaction, and increasing retention rates.
9. Quality Control an d Anomaly D etectio n : Classification is employed in quality control
processes to classify products as defective o r non-defective. It is also used for anomaly
detection to identify unusual patterns or outliers in data, signaling potential issues.
10. Automated Decision-M aking: With the advancem ent of artificial intelligence and machine
learning, classification algorithms are integrated into automated decision-making systems.
These systems can classify data in real-tim e and make autonomous decisions based on
predefined rules and models.

Classification Algorithms in Machine Learning

These classification algorithms offer a diverse s e t o f tools for solving a wide range of classification
tasks in machine learning. Each algorithm has its strengths and is suitable for different types of data
and problem domains. Experimenting with these algorithm s and understanding their characteristics
can help in selecting the m ost appropriate approach for a given classification problem.
1. Logistic Regression:
• Description: Logistic regression is a lin ear classification algorithm used for binary
classification tasks. It estimates the probability that a given input belongs to a particular
class.
• Advantages: Simple, interpretable, w orks w ell for linearly separable data.
• Application: Spam detection, custom er churn prediction.
2. Support Vector Machines (SVM):
• Description: SVM is a versatile classification algorithm that finds the optimal hyperplane
to separate classes in the feature space. It can handle linear and non-linear classification
tasks.
• Advantages: Effective in high-dimensional spaces, works well with clear margin of
separation.
• Application: Text categorization, image recognition.
3. Decision Trees:
• Description: Decision trees are non-linear classifiers that recursively split the data based
on feature values to make predictions. They create a tree-like structure of decisions.
Advantages: Easy to interpret, can handle b oth numerical and categorical data.
• Application: Customer segmentation, m edical diagnosis.
 Machine Lrarhing'

4. R andom F o re s t; con sists of multiple decision

. D escrip tion : Random Forest is an ensem . overfitting by aggregating the
trees. It improves prediction accuracy and reduces overtitting y
predictions of individual trees.
. A d v antag es: Robust to overfitting, handles [Link] d ata well.
. A pplication: Credit risk analysis, image classification.

" " " r a T i n ^ ^

m ajority class among their k nearest neighbors i n the feature spac .

. A dvantages: Simple, non-parametric, easy to implement.

. A pplication: Pattern recognition, recommendation systems.

6. Neural N etw orks: ip^mine classifiers th a t consist of interconnected

■ “ ;r r r o r :o d " tlr n " » ^ ^ ^ ^ ^ ^ pattems in the d ata through training with

/ A ^ l ^ a b l e o f l e a r n i n g i n t r i c a t e patterns,suitahleforlargedatasets.

. A pplication: Image recognition, speech recognition.

Measuringperf— indassificationin;ivesevalua^ng^^^^^^^

positives CFP), tru e negatives (TN), and false nega ives performance of a
Confusion M atrix : A confusion matrix is a . ^e of an algorithm by displaying the

____________ _Predicted Positive

Predicted Negative
False Positive (FP)
True Negative (TN)
Actual Negative
True Positive (TP)
False Negative [FN)
is # ,: ^ -■

1. A ccu racy : Accuracy is the most com monly used metric for evaluating classification models.
It m easu res the proportion of correct predictions made by the m od el out of the total number
of predictions. It is calculated as the ratio of the number o f c o r re c t predictions to the total
num ber o f predictions.
Accuracy = (TP + TN) / (TP + FP + TN + FN)
A high accuracy score indicates that the model is making co rre ct predictions most of the time.
However, accuracy can be misleading when the class distribution is imbalanced.
2. P re c isio n : Precision measures the proportion of true positives am ong the instances that the
model predicted as positive. It is calculated as the ratio o f the n u m b er of true positives to the
total nu m b er of instances predicted as positive.
P recision = TP / (TP + FP)
A high precision score indicates that the model is making few er false positive predictions. It is
useful w hen the cost of false positives is high.
3. R e c a ll: Recall measures the proportion of true positives am ong th e instances that are actually
positive. It is calculated as the ratio o f the number of true positives to the total number of
actual positive instances.
Recall = TP / (TP + FN)
A high recall score indicates that the model is capturing a m ajority of the actual positive
instances. It is useful when the cost o f false negatives is high.
4. F I s c o r e : F I score is the harmonic mean o f precision and recall. It provides a balance between
the two m etrics and is particularly useful when the class (jistribu tio n is imbalanced.
F I score = 2 * (p recisio n * recall) / (p re c is io n + recall)
A high F I score indicates that the model has both good precision and recall. It is useful when
both false positives and false negatives are equally important

Example
Suppose we have a binary classification problem where we are predicting whether an email is spam (positive
class) or not spam (negative class). After appljnng a classification algorithm to a set of emails, we can construct
a confusion matrix to evaluate its performance.
Actual/Predicted Predicted Not Spam Predicted Spam
Actual Not Spam True Negative (TN) False Positive (FP)
Actual Spam False Negative (FN) True Positive (TP)
In this example:
• True Negative (TN) : Emails correctly predicted as not spam.
• False Positive (FP): Emails incorrectly predicted as spam (actually not spam).
• False Negative (FN) : Emails incorrectly predicted as not spam (actually spam).
• True Positive (TP): Emails correctly predicted as spam.
Suppose we have 100 emails, out of which 30 are spam and 70 are not spam. After applying a classification
algorithm, we obtain the following confusion matrix:
 T

Actual/Predicted Predicted Not Spam Predicted Spam

Actual Not Spam 65 5
Actual Spam 10 20

In this example:
• True Negative (TN) = 65 (65 emails were correctly predicted as not spam).
• False Positive (FP) = 5 ( 5 emails were incorrectly predicted as spam).
• False Negative (FN) = 10 (10 emails were incorrectfy predicted as not spam).
• True Positive (TP) = 20 (20 emails were correcdy predicted as spam).
By anal3^ing these values, we can calculate various performance metrics such as accuracy, precision, recall,
and FI score to assess the effectiveness of the classification algorithm in distinguishing between spam and not
spam emails.
1. Accuracy:
Accuracy = (TP + TN) / (TP + TN + FP + FN)
Accuracy = (20 + 65) / (20 + 65 + 5 + 10) = 85 /100 = 0.85 or 85%
The accuracy of 85% indicates that the model is making correct predictions for 85% of the total
instances. This means that the model is performing well in terms of overall classification accuracy
2. Precision :
Precision = TP / (TP + FP)
Precision = 20 / (20 + 5) = 20 / 25 = 0.8 or 80%
The precision of 80% indicates that out of all the instances predicted as positive, 80% of them are
actually positive. This means that the model is making fewer false positive predictions.
3. Recall (Sensitivity):
Recall = TP/(TP+ FN)
Recall = 2 0 / (20 + 10) = 20 / 30 = 0.67 or 67%
The recall of 67% indicates that out of all the actual positive instances, the model is able to capture
67% of them. This means that the model is missing some of the actual positive instances.
4. F I S co re:
F I Score = 2 * (Precision * Recall) / (Precision + Recall)
F I Score = 2 * (0.8 * 0.67) / (0.8 + 0.67)
F I Score = 2 * (0.536) / (1.47) = 1.072 / 1.47 = 0.73 or 73%
The F I score of 73% indicates that the model has a good balance between precision and recall. This
means that the model is making accurate positive predictions while capturing a majority of the actual
positive instances. However, it is important to note that the FI score is lower than the accuracy score,
which suggests that the class distribution may be imbalanced_______________________________

Regression
Regression is a type of supervised learning technique in machine learning where the goal is to predict
continuous or quantitative outputs based on input features. Unlike classification, where the output is
categorical, regression models predict a continuous value.
 . ; -'-V -

Regression is a process of finding the correlations between dependent and independent variables.
It helps in predicting the continuous variables such as prediction o f M arket Trends, prediction of
House prices, etc. The task of the Regression algorithm is to find the mapping function to map the
input variable(x] to the continuous output variable(y).
Regression algorithms are used if there is a relationship betw een the input variable and the output
variable. It is used for the prediction o f continuous variables. The goal of regression tasks is to predict
a continuous numbe or a real number. If there is continuity betw een possible outcomes, then the
problem is a regression problem.

What is Regression in ML?

f i
Regression in machine learning is a type of supervised learning problem where the goal is to
predict continuous numerical values based on input features. Unlike classification, which predicts
discrete class labels, regression models estimate a continuous output variable. The objective of
regression is to establish a relationship between the input variables and the continuous target
variable and allowing the model to make predictions on new or unseen data points.
Example: Predicting house prices, forecasting stock prices, estimating the temperatiire based on
weather variables.

0^ Examples Regression Tasks

1. Real Estate Valuation:

• Task: Predict the market value of properties.
• Features Features like area, number of bedrooms, age of the property, amenities, etc.
• Target Variable: Estimated price of the property.
• Purpose: Helps buyers and sellers get a fair idea of property prices and assists investors and real
estate companies in making informed decisions.
2. Stock Market Prediction:
• Task: Predict the stock price or return value
• Features Historical stock prices, trading volume, market indices, economic indicators.
• Target Variable: Estimated Future stock price or return.
• Purpose: Using regression analysis, historical stock data, and relevant market indicators, a
model can predict the fiitijre price or return of a stock. It helps investors in making informed
trading decisions.
3. Credit Risk Assessment:
• Task: Predict the probability of default or credit risk
• Features: Credit score, income, debt-to-income ratio, loan amount
• Tai^et Variable: Probability of default or credit risk.
• Purpose: Regression models can be employed to assess credit risk by analyzing borrower
information and financial metrics, predicting the likelihood of default or the level of credit risk
_________associated with a loan applicant It helps financial institutions in making lending decisions.
 learning
M

~1 ConHnuous O u tp u t V ariables; The term "continuous" refers to output v ariab les that can e
■ any value v,ithin a range. These are quantiflable and can he subdivided into fin e r increments,
which are not restricted to separate categories.
Ejcamples of Continuous Outputs:
. House P ric e P rediction: Predicting property prices based on location, size, and other
features The predicted house price is a continuous variable that re p rese n ts the monetary
value of a house. The output is a real number that can vaiyacross a wide ran ge, depending
on the input features.
. sto c k P ric e Forecasting; Estimating future stock prices based on h isto rical data and
market indicators. Predicted stock prices are continuous values that can fluctuate minute
by minute. Each predicted value is a specific numeric figure that re p rese n ts the stock
price at a future time.
2 Supervised L ea rn in g ; Regression also relies on labeled training data w here e a ch input feature
set i s " “ ed with a continuous output value. This data is used to train the m odel to understand
and predict the relationship between input variables and the continuous ou tcom e.
3. L in e a r vs. N o n -lin ear; The relationship can be linear (simple linear reg re ssio n l
(polynomial regression, logistic regression for binao- outcomes m a
L \ e l and choosing the correct type of regression model depends on th e nature of the
relationship betw een variables.
4 D ecision B o u n d a rie s; in regression, the decision boundary can be con sid ered as the line
or c u ™ that b e st fits the data points in the feature space. This concept is nacre nuanced m
regression as the - f i f direcUy predicts a value rather than categonzing.
5 P erfo rm an ce E valuation M etrics; Performance in regression (asks is a s s e s s ^
focusing on how close the predicted values are to the actual values. Com m on M etrics nclude
Mean Squared E rro r [MSE), Root Mean Squared Error (RMSE). Mean A bsolute Er [ ),
R-squared (Coefficient of Determination).

l’ s im p le L iiiia r R egressio n ; Simple linear regression predicts a resp o nse variable usmg a
l g ! e f e l r e . It assumes a linear relationship between the independent variable and the

target variable. .. j u
Exam ple; Predicting a student's exam score based on the number of hours th e y studied Here,
the number of hours studied is the single feature used to predict the exam sco re.
2 M ultiple L in e a r Regression; Multiple linear regression involves using multiple features
To X t a re s p o n s f variable. It extends simple linear regression to incorporate several
independent variables.
Exam ple; Predicting house prices based on features like square footage, n u m b er of bedroom ,
l o c a J n , and age o f the house. In this case, multiple features are con sid ered to estimate the
selling price o f a property.
 ' •- ' Supervised teaming^ 3.11

3. P o l^ o m ia l R e g re ss io n : Polynomial regression models the relationship between the

independent v a ria b le and dependent variable as an n* degree polynom ial, allowing for more
complex curve fitting.
Example: P red ictin g the growth of a plant based on time spent in sunlight. By using polynomial
regression, th e m od el can capture non-linear relationships, such as accelerated growth with
increased su n lig h t exposure.
4. Logistic R e g r e s s io n : Logistic regression, despite its name, is used for binary classification
problems w h e re th e target variable has two classes. It estimates the probability of an instance
belonging to a p articu lar class.
Example: P red ictin g whether a customer will churn or not based on factors like usage patterns
and custom er demographics. Logistic regression can be employed to classify customers into
chumers and non-churners based on historical data.

Regression A lgorithm s in Machine Learning

Regression algorithm s a re used to predict continuous values based on input features. Some common
regression algorithm s w ith their descriptions, advantages, and applications:
1. Linear R e g re s sio n :
• D e scrip tio n : Linear regression models the relationship b etw een the independent
variables an d the continuous target variable by fitting a linear equation to the data.
• Advantages: Simple, interpretable, computationally efficient.
• A p p licatio n : Predicting house prices, estim ating sales revenue.
2. Polynomial R eg re ssio n :
• D escrip tio n : Extends linear regression by adding polynomial term s to the model,
allowing it to capture non-linear relationships.
• Advantages: Can model a broader range o f data shapes than lin ear regression.
• A p p licatio n : Situations where the relationship between variables is curved, such as
growth ra te s, trajectories, and other natural phenomena.
3. Ridge R e g re ssio n :
• D escrip tio n : Ridge regression is a regularized form of linear regression that adds a
penalty te r m to the cost function to prevent overfitting by shrinking th e coefficients.
• Advantages: Handles multicollinearity, reduces model complexity.
• A p p licatio n : Stock price prediction, risk analysis.
4. Lasso R e g re ssio n :
• D e scrip tio n : Lasso regression is another regularized linear regression technique that
uses the L I norm penalty for feature selection by shrinking som e coefficients to zero.
• Advantages: Feature selection, interpretable models.
• A p p licatio n : Marketing spend optimization, medical cost prediction.
3.11 l^ ^ to ^ iiy ;^ '«

5. Random Forest Regression:

• Description: Random Forest regression is an ensemble m ethod th a t combmes multiple
decision trees to improve prediction accuracy and handle non -lin ear relationships.
• A dvantages: Robust to overfitting, handles large datasets.
• A p p licatio n : Demand forecasting, stock market analysis.
6. Support Vector Regression (SVR):
• D e scrip tio n : SVR is a regression technique based on support v e cto r machmes that finds
the hjqjerplane with the maximum margin of error to make predictions.
. A dvantages: Effective in high-dimensional spaces, handles non-linear relationships.
• A p p licatio n : Real estate price prediction, time series forecasting.
7. Decision Tree Regression:
. D e scrip tio n : Decision tree regression builds a tree structure to make predictions by
partitioning the feature space into regions and assigning a co n stan t value to each region.
. A dvantages: Easy to interpret, handles both numerical and categorical data.
• A pp lication : Sales forecasting, risk assessm ent
8 . G rad ien t B o ostin g Regression:
. D e scrip tio n : Gradient Boosting regression builds an ensem ble of weak learners
(typically decision trees) sequentially to minimize the loss function.
. A dvantages: High predictive accuracy, handles complex relationships.
• A p p licatio n : Credit scoring, anomaly detection.

Im portance and Benefit? of Regression in Machine Learning

Regression analysis plays a crucial role in machine learning and data science fo r predicting continuous
outcomes based on input variables. Some key importance and benefits o f regression in machine
learning are:
1 P re d ictiv e Modeling: Regression models are essential for m aking predictions and forecasting
' fixture trend s based on historical data. They help in understanding th e relationship between
input features and tiie target variable.
2. In te rp re ta b ility : Regression models are often easy to interpret, especially linear regression,
as they provide coefficients that indicate the impact of each feature o n the target vanable. This
interpretability is valuable for decision-making and understanding th e driving factors behind
predictions.
3 F e a tu re Selectio n : Regression analysis can help in identifying th e m ost important features
’ tiiat influence the target variable. Techniques like Lasso reg ression can perform featiire
selection by shrinking coefficients to zero, leading to a more con cise and relevant model.
4 . M odel Evaluation: Regression models provide mettles such as RMSE (Root Mean Square
Error) and R-squared to evaluate the performance o f the model. T h e se metrics help in assessing
the accuracy and reliability of predictions.
 5. Handling Non-linear Relationships: Techniques like polynom ial regression, decision tree
reg ression , and support vector regression can capture non-linear relationships between
variables, allowing for more flexible modeling of complex data patterns.
6 . R isk A sse ssm e n t: Regression models are widely used in risk assessment and financial
analysis to predict outcomes such as credit risk, stock prices, and insurance claims. They help
in quantifying and managing risks effectively.
7. Optimization and Decision Making: Regression models can b e used for optimization tasks,
such as determining the optimal pricing strategy, resource allocation, or process improvement
based on predictive insights.
8. Scalability and Efficiency: Regression algorithms can handle large datasets efficiently making
them su itab le for real-world applications with high-dimensional data and a large number of
observations.
9. Generalization: Well-constructed regression models can generalize well to unseen data,
m aking th e m reliable for making predictions on new instances o r in production environments.
10. Versatilily: Regression techniques are versatile and can be applied to various domains such
as h ealth care, marketing finance, and engineering, making them a fundamental tool in data
analysis and decision support systems.

When evaluating the performance o f regression models, various m etrics are used to assess how well
tiie model p re d icts continuous outcomes. Here are some common perform ance evaluation metrics in
regression:
1. Mean Squared Error (MSE):
• D e scrip tio n : MSE calculates the average of the squared differences between predicted
v alu es and actual values.
• Formula: MSE = E(yi - yO^ / n
• Advantages: Penalizes large errors, provides a m easure o f model accuracy.
• Disadvantages: Sensitive to outliers.
2. Root Mean Squared Error (RMSE):
• D e scrip tio n : RMSE is the square root of the MSE, providing a measure of the standard
d eviation of the residuals.
• Formula: RMSE = V(I(yi - yO^ / n)
• Advantages: Interpretable in the same units as the targ et variable.
• Disadvantages: Same as MSE, sensitive to outliers.
3. M ean A b solu te Error (MAE):
• D e scrip tio n : MAE calculates the average of the absolute differences between predicted
valu es and actual values.
• Formula: MAE = EJyi - ^i| / n
 Sis:
>:MachrneiMming -‘r/.. - f e ? ■-

• Advantages: Less sensitive to outliers compared to MSE and RMSE.

• D isadvantages: Does n o t penalize large errors as much as MSE.
4 . R -sq u ared (R^):
• D escription: R-squared m easures the proportion o f the variance in the d ep endent
variable that is predictable from the independent variables.
• Form ula: = 1 - (E(yi - yO^ / ECYi -
• Range: 0 (worst) to 1 (b e st).
• In terp retatio n : Higher values indicate a better fit o f the model to the data.
5. A dju sted R-squared:
• D escription; Adjusted R-squared penalizes the addition o f unnecessary predictors in the
model, adjusting for the nu m ber of predictors.
• Form ula: Adjusted R^ = 1 - (1 - R^) * (n - 1 ) / (n - p - 1 )
• Advantages: Helps p rev ent overfitting by considering the number of predictors.
6 . M ean Squared L ogarithm ic E r r o r (MSLE):
. D escription: MSLE calcu lates the mean of the squared differences between the natural
logarithm of predicted valu es and the natural logarithm of actual values.
• Form ula: MSLE = E (lo g (l + yO - log(l + yO)^ / n j
• A pplication: Useful w hen th e target variable has exponential growth patterns.
These perform ance evaluation m etrics provide insights into the accuracy, precision, and generalization
capabilities of regression models, h elping data scientists and analysts assess the quality o f pred ictions
and optimize model performance.

Difference between Regression and Classification

Regression and Classification algorithm s are Supervised Learning algorithms. Both the algorithm s
are used for prediction in Machine learning and work with the labeled datasets. But the difference
between both is how they are used fo r different machine learning problems.
The main difference between R egression and Classification algorithm s that Regression algorithm s
are used to p red ict the con tin u o u s values such as price, salary, age, etc. and Classification algorithm s
are used to predict/Classify th e d is c r e te values such as Male or Female, True or False, Spam o r Not
Spam, etc.
 •■’■i L«mt>ngyTl5

' I'w^Regression Algoridm i <• - Classificatibn'A^rithm

In Regression, the output variable must be of In Classification, the output variable must b e a discrete
continuous nature or real value. value.
The task of the regression algoridim is to map the The task of the classification algorithm is to map the
input value (x) with tiie continuous output variable[y). input value(x) with the discrete output variable(y).
Regression Algorithms are used with continuous data. Classification Algorithms are used with discrete data.

In Regression, we try to find the b est fit line, which can In Classification, we tiy to find the decision boundary,
predict the output more accurately. which can divide the dataset into different classes.
Regression algorithms can be used to solve the Classification Algorithms can be used to solve
regression problems such as Weather Prediction, classification problems such as Identification of spam
House price prediction, etc. emails. Speech Recognition, Identification of cancer
cells, etc.
Regression algorithms can be further categorized into Classification algorithms can be divided into binary
linear regression (example, Ordinary Least Squares) classifiers (example. Logistic Regression, Support
and non-linear regression (example. Decision Trees, Vector Machines) and multi-class classifiers (example,
Random Forest). Random Forest, K-Nearest Neighbors).
Common evaluation metrics for regression include Evaluation metrics for classification include Accuracy,
Mean Squared Error (MSE), Root Mean Squared Error Precision, Recall, FI Score, and Area Under the ROC
(RMSE), and R-squared. Curve (AUC-ROC).
Regression models are often more interpretable as Classification models may focus more on decision
they provide coefficients indicating the impact of input boundaries and class predictions rather than feature
features on the output importance.
Some regression algorithms may be computationally Certain classification algorithms can handle large
intensive for large datasets due to the complexity of datasets efficlentiy, making them suitable for scalable
fitting curves. applications.

Some Sample D a tasets

Creating sample datasets m anually is a simple and easy way to understand supervised learning
algorithm s and gain insights in to model training processes. T hese datasets serve as a fundam ental tool
for grasping algorithmic c o n ce p ts and understanding m achine learning skills. Moreover, integrating
real-world data from extern al repositories enhances the learning experience and facilitates the
development of predictive m o d els customized to specific dom ains and scenarios.
For practical applications, real d ata can be obtained from various reputable sources such as
1. UC Irvine M achine L e a r n in g Repository: [Link]
2 . Amazon's AWS d a t a s e ts : [Link]
3 . K ag g le: [Link]
4 . Google D ataset S e a r c h : [Link]
Let's create sample datasets m anually to explore and apply various supervised learning algorithms.
1. Students P e rfo rm a n ce D a ta set: This dataset captures the academic perform ance o f students
in educational institu tio n s. It includes attributes such as student ID, marks obtained in subjects
like Mathematics, S cien ce, Languages, attendance percentage, participation in sports or
 cultural events, and overall grade. It can be used for practicing supervised learning algorithms
in the context of student performance analysis
Type the below data and Save the file as students_data.csv._______ __________ __________ _
student_id,m ath_score,science_score.language_score,attendance_percentage,sports_

participation,culturaLevents,overall_grade
1,85,78,92,95,Yes,No^
2,72,65,80,88 ,No,Yes,B
3,90,85,88,92,Yes,Yes,A
4,78,70,75.85,No,No,C
5,95,88,94,98,Yes,Yes,A
6,68,72,70,80,No,No,D
7,82,75,85,90,Yes,Yes,B
8,88,82,90,94,Yes,No,A
9,75,68,72,82,No,Yes,C
10,93,90,87,96,Yes,Yes,A
11,70,62,78,86,No,No,C
12,84,80,86,9l,Yes,Yes,B
13,77,72,74,84,No,No,C
14,91,86,89,93,Yes,YesA
15,73,68,70,81,No,Yes,C
2 Stock M arket T ren d s Dataset; The dataset focuses on the stock m arket trends of Indian
companies. This dummy dataset includes information on various com panies such as
stock prices, trading volumes, market indices, P /E ratios, dividend yields, and sector-wise
performance. It can be used for analyzing stock m arket trends, exploring co rre la o m s widi
L n o m i c indicators, and practicing predictive modeling for future trend predictions. TVpe the
below data and Save the file as stockm [Link]._________________ ___________ ________
company_name,stock_price,trading_volume,market_index,pe_ratio,dividend_yield,sector.

performance
Company A,1200,50000,15000,25,2.5,0utperforming

Company 6,800,35000,1 2 0 0 0 ,18 ,1 .8 ,Underperforming

Company C,9 5 0 ,42000,13500,20,2.0,Neutral
Company D,1100,48000,14500,22,2.2,0utperforming

Company £,700,30000,11000,15,1.5,Underperforming
Company F,850,40000,13000,19,1.9,Neutral
Company G,1050,45000.14000,21,2.1,0utperforming

Company H,750,32000,10500,16,1.6,Underperforming______________ ___________ ___________

 Supervis^ iMrnlng^ 3.17

Company 1,1000.43000,12500,20,2.0,Neutral
Company 1,800.38000.11500,17,1.7,Underperforming
Company K,950,44000,13000,19,1.9,Neutral
Company L,1150.47000,14000,23,2.3,0utperforming
Company M,7 2 0 3 1 0 0 0 ,10500 ,16 ,1 .6,Undeq3erfomiing
Company N,880.39 000,12000,18,1.8,Neutral
Company 0,1020.46000,13s00,21,2.1,0utperforming
3. W eadier P a tte r n s D ataset: This dummy dataset includes information on w eather conditions
such as tem p eratu re, humidity, wind speed, and the weather condition for each day. It can be
used for anal3^ in g w eather patterns, studying the impact of weather on various activities, and
building pred ictive models for weather [Link] the below data and Save the file as
w ead ier_d [Link].

The K-Nearest N eighbors (K-NN) algorithm is a fundamental machine learning technique that operates
on the principle o f sim ilarity. It is a versatile and intuitive method used for b oth classification and
regression tasks. In K-NN, the classification of a new data point is determined by th e majority class of
its nearest neighbors in th e training dataset Similarly, for regression tasks, the algorithm predicts the
value of a new data p o in t based on the average o f the target values of its clo sest neighbors. K-NN is
known for its sim plicity and effectiveness in scenarios where the underlying d ata distribution is not
well-defined or w hen lin e a r separation is not feasible.
 Vjlhachine learning

Why do we need K-NN Algorithm?

Suppose there are two categories, i.e., C ategory A and Category B, and we have a new data point
x l, so this data point will lie in which o f these categories. To solve this type of problem, we
need a K-NN algorithm. With the help o f K-NN, we can easily identify the category or class o f a
particular dataset. Consider the below d iagram ;

\
9 O
Category B Category B

New Data point

* New data point

0 * * * assigned to
^ • Category A
CategoryA
©

Example
In the scenario where we have an image of a creature that exhibits similarities to both cats and dogs, but we
need to determine whether it belongs to the cat or dog category, the K-Nearest Neighbors (K-NN] algorithm can
be employed. By leveraging its similarity-based approach, the K-NN model will analyze the features of the new
image and compare them to existing images of cats and dogs in the dataset. Based on the closest resemblance
or similarity to features of known cat and dog images, the algorithm will classify the new image into either the
cat or dog category. This process of identifying the category of the creature in the image showcases how K-NN
utilizes the concept of similarity to make accurate classifications in machine learning tasks.

KNN Classifier

Input value Predicted Output

H H Characteristics of K -N earest Neighbors (K-NN) Algorithm:

1. N on -P aram etric Nature:
K-NN is a non-parametric algorithm th a t does not make assum ptions about the underlying
data distribution. It relies on the sim ila rity of data points for classification or regression.
2. Lazy L earn ing Approach:
K-NN is considered a "lazy learn er" a s it postpones the learning process until the classification
phase. It stores the training d ata and performs classification only when a prediction is
requested.
 - Supervised Learning

3. S torage o f Training Data:

During the training phase, K-NN stores the entire training dataset. It does not actively learn
from the data but retains it for future classification tasks.
4. N earest Neighbor Classification:
The algorithm classifies new data points based on the sim ilarity to the k-nearest neighbors
in the training dataset. The m ajority class among these neighbors is assigned to the new data
point for classification.
5. R egression and Classification:
K-NN can be applied to both regression and classification problems. In regression tasks, it
predicts the value of a new data point based on the average o f the target values of its nearest
neighbors.
6. Sim ilarity-Based Prediction:
K-NN operates on the assum ption that similar data points belong to the same class or have
sim ilar target values. It categorizes new data points based on their proximity to existing data
points.
7. V ersatility and Ease of Im plem en tation :
K-NN is known for its sim plicity and ease of implementation. It is a versatile algorithm suitable
for various machine learning tasks, especially in scenarios w here the data distribution is not
explicitly defined.

jm How K-Nearest Neighbors (K-NN) Works ?

1. S tore Training Data:
During the training phase, the K-NN algorithm stores all the training data points in m em ory
w ithout performing any com putation on the data.
2. C alculate Distance:
W hen a new data point is presented for prediction, the algorithm calculates the distance
betw een this new data point and all the data points in the training set.
The distance metric used is typically the Euclidean distance, although other distance m etrics
like Manhattan distance or Minkowski distance can also be utilized based on the problem
requirem ents.
3. S elect N earest Neighbors:
After calculating the distances, the algorithm identifies the K nearest data points (neighbors)
to the new data point based on the calculated distances.
The value o f K is a hyperparam eter th at needs to be predefined by the user. It determ ines the
num ber o f neighbors considered for classification or regression.
4. For Classification:
In the classification task, once the K nearest neighbors are identified, the algorithm assigns the
m ajority class label among these neighbors to the new data point.
 M a e h i n e ''1: - ’

The class with the highest frequ en cy among the K neighbors is selected as the predicted class
for the new data point.
5. For Regression:
In regression tasks, the algorithm calculates the average (o r weighted average) of the targ et
values o f the K nearest neighbors.
This average value serves as th e predicted value for the new data point, providing a continuous
output rather than discrete cla sse s as in classification.

K -N N A lg o r ith m
The K-NN working can be explained on th e basis of the below algorithm :

Algorithm K-NN Algorithm

Step-1: Choose the Number of Neig^ibors (K):
The first step in the K-NN algorithm is to select the number of neighbors (K) that will be considered
when making predictions for a new data point The value of K is a hyperparameter that needs to
be specified before running the algorithm.
Step-2: Calculate Distance:
Compute the distance between the new data point and all the data points in the training s e t The
distance metric, commonly Euclidean distance, measures the similarity or proximity between
data points in the feature space.
Step-3: Sort and Select Nearest N e i g h b o r s :
After calculating the distances, sort the distances in ascending order and selects the K data points
with the smallest distances to the new data point These K data points are the nearest neighbors
to the new data point in the feature space.
Step-4: For Classification:
In the classification task, assign a class label to the new data point based on the majority class
among the K nearest neighbors. The class with the highest ft-equency among the K neighbors is
chosen as the predicted class for the new data point
Step-5: For Regression: For Regression:
In regression tasks,
In regression tas compute the average (or weighted average) of the target values of the K
nearest neighboi
nearest neighbors. This average value serves ^ the predicted target value for the new data point
in regression analysis

How to select the v alu e o f K in the K-NN Algorithm?

Selecting the value of K in the K -N earest Neighbors (K-NN) algorithm is a crucial step th a t can
significantly impact the model's perform ance. Below are some points to remember while selecting
the value o f K in the K-NN algorithm :
. For binary classification task s. It is recommended to choose an odd value of K to avoid ties m
m ajority voting. This helps in m aking a clear decision when selecting the class label.
. Experiment with different v alu es o f K and observe the model's performance on a validation
s e t By Iteratively testing d ifferen t K valuM and analyzing the results, you can fine-tune the K
value to achieve the b est pred ictive accuracy.
 ' '.l/'Supervised Learning
- ‘ ® -' ■ --I*' . L'.r-^-'-.^-"'' . -;..i , ■■_ .......... ...... ,

• As a general guideline, start with small values of K (e.g., K=3 or K=5] and gradually increase the
value while monitoring the model's performance. This iterative approach can help in finding
an optimal K value that balances bias and variance in the model.

m y jm How to Calculate Euclidean Distance ?

• F o r Two Points (2D ) : The Euclidean distance formula for calculating the distance between
two points in a two-dimensional space (2D) is given by:
d = S Q R T ((v x ,)^ +(y,-yx)^)
• Fo r Three Points (3D ): The Euclidean distance formula for calculating the distance between
three points in a three-dimensional space (3D) is given by:
d=SQRT( (x ,-x j^ + (y ,-y j^ + (z 3 -z ,)2 )
• For Four Points (4D ): In a four-dimensional space (4D), the Euclidean distance formula for
calculating the distance between four points is given by:
d= SQRT( (X^-Xj)^ + (y 2 -y j' +(z2"Zi)' + (w^-Wj)^)
In general, the Euclidean distance formula can be extended to higher dimensions by adding the
squared differences of each coordinate and taking the square root of the sum of these squared
differences.
Where:
• d is the Euclidean distance between the points.
• (X j,yJ and (x^,y2) are the coordinates of the two points in a 2D space.
• (x^,yj,Zj) and (x2,y2,Z2) are the coordinates of the three points in a 3D space.
• (x^,yj,Zj,wJ and (x 2 ,y 2 ,Z2 ,w^) are the coordinates of the four points in a 4D space.
These formulas help calculate the Euclidean distance between points in different dimensions, which is
a fundamental metric used in the K-Nearest Neighbors (K-NN) algorithm to determine the proximity
or similarity between data points in the feature space.

0^ Example 1 KNN Algorithm for Classification Task

Let's consider an example with a dataset for classifying fruits based on two features: sweetness and acidity. We
will use the K-Nearest Neighbors (KNN) algorithm to classify a new fruit based on its sweetness and acidity
values.
• Example Training Data:
Fruit 1: Sweetness 8, Acidity 3 - Type: Apple
Fruit 2: Sweetness 6, Acidity 2 - Type: Apple
Fruit 3: Sweetness 3, Acidity 7 - 13^)6: Lemon
Fruit 4: Sweetness 2, Acidity 8 - Type: Lemon
• New Data Point: Sweetness 5, Acidity 4
Find the type of fmit using KNN algorithm.
Solution: . c j-
. For the new data point with Sweetness 5 and Acidity 4. the KNN algorithm would classify it by finding
its nearest neighbors based on Euclidean distance in the 2D feature space of Sweetness and Aadity, and
then determining the majority class among those neighbors to assign the type of fhnt.
. Let's choose K = 3 for this example.
. Calculate the Euclidean distance between the new data point and all data points in the training set
Distance from (5,4) to Fruit 1 (8,3) : sqrtCC8-5)> * (3-4)>) = sqrt[9 1 1 ) = sqrt(lO) » 3.16
DisUnce from (5,4) to Fnilt 2 (6,2) : sqrtC(6-5)= * (2-4)’) = sqrt(l t 4) = sqrt(5) = 2.24
Distance Irom (5,4) to [Link] 3 (3,7) : sqrtCC3-S)' + = *<lrt(4 + 9) = sqrt(13) » 3.61
Distance from (5,4) to Fruit 4 (2,8) : sqrt((2-5)» t (8-4)>) = sqrt(9 * 16 ) = sqrt(25) = S

. Sort the distances in ascending order:

2.24 (Fruit 2), 3.16 (Fruit 1), 3.61 (Fruit 3), 5 (Fruit 4)
. Select the 3 nearest neighbors (we have chosen k=3) based on the calculated distances.

Fruit 2, Fruit 1, Fruit 3

• Select Majority Class:
Among these nearest neighbors: Fruit 2 and Fruit 1 are both Apples. Fruit 3 is a Lemon.
Since there are 2 Apples (Fruit 2 and Fruit 1) and 1 Lemon (Fruit 3) among the 3 nearest neighbors, the
majority class is Apple.
Therefore, based on the majority class rule, the new fruit with sweetness 5 and acidity 4 will be classified as an
Apple using the KNN algorithm with K = 3._______________

KNN Algorithm for Regression Task

with a dataset for predicting prices based on two features: sweetness and an^lty.
We will use the K-Nearest Neighbors (KNN) algorithm to predict the price of a new fruit based on its sweetnes
and acidity values.
• Example Training Data:
Fruit 1: Sweetness 8, Acidity 3 - Price: ^100
Fruit 2: Sweetness 6 , Acidity 2 - Price: ^80
Fruit 3: Sweetness 3, Acidity 7 - Price: ^50
Fruit 4: Sweetness 2, Acidity 8 - Price: ^40
• New Data Point: Sweetness 5, Acidity 4
Find die price of fruit using KNN algorithm.
Solution:
. For the new data point with Sweetness 5 and Acidity 4, the KNN algorithm would predict the price by
finding its nearest neighbors based on Euclidean distance in the 20 feature space of Sweetness an
Acidity, and then determining the average price among those neighbors.
. Let's choose K = 3 for this example.
. Calculate the Euclidean distance between the new data point and all data points in the training set
Distance from (5,4) to Fmit 1 (8,3) : sqrt((8-5y * (3-4)-) = sqrt[9 . 1) = sqrt(lO) ^ 3.16_
 ••>21
Supervis^lMraiii
m ■

Distance from (5,4) to Fruit 2 (6,2) : sqrtC(6-5)' + ( 2 -4 ) 2) ^ sqrt(l + 4) = sqrt(5) =«224

Distance from (5,4) to Fruit 3 (3,7) : sqrt((3-5)2 + (7-4)^ =sqrt(4 + 9) = sqrt{13) =3.61
Distance from (5,4) to Fruit 4 (2,8) : sqrt((2-5)' +(8-4)2) ^ + i6) = sqrt(25) =5
• Sort the distances in ascending order.
2.24 (Fruit 2), 3.16 (Fruit 1), 3.61 (Fruit 3), 5 (Fruit 4)
• Select the 3 nearest neighbors (we have chosen k=3) based on the calculated distances.
Fruit 2, Fruit 1, Fruit 3
• Predicted Price:
Average Price of Nearest Neighbors; (Fruit 2 Price + Fruit 1 Price + Fruit 2 Price) / 3
= (^80 + ^100 + ^50) / 3 = ^76.67
Therefore, based on the average price of the 3 nearest neighbors, the predicted price for the new fruit with
sweetness 5 and acidity 4 will be ^76.67 using the KNN algorithm with K = 3 in a regression task.___________

g§y Example 3 KNN Algorithm for Classification Task

Consider a dataset with the following points in a two-dimensional space;
• A: (2,3) - Class 1
• B: (3,5) - Class 1
• C: (3,2) - Class 2
• D: (6, 7) - Class 2
Now, we want to classify a new data point E: (4, 4).
Using the Euclidean distance as the distance metric, we calculate the distance between E and each point in the
training set
• Distance(E, A) = sqrt((4-2)^ + (4-3)^) = sqrt(5) 2.24
• Distance(E, B) = sqrt((4-3)2 + (4-5)^) = sqrt(2) == 1.41
• Distance(E, C) = sqrt((4-3)^ + (4-2)^) = sqrt(2) = 1.41
• Distance(E, D) = sqrt((4-6)2 + (4-7)^) = sqrt(lO) =: 3.16
Next, we select the K closest points to E based on the calculated distances. Let's say K = 3. The three closest
points to E are B, C, and A.
Finally, we assign the class to the new data point E based on the most common class among its K nearest
neighbors. In this case, two of the three closest points belong to Class 1, so we classify E as Class 1.
Therefore, using the KNN algorithm with K = 3 and the Euclidean distance metric, the new data point E: (4,4)
is classified as Class 1.

Example 4 KNN Algorithm for Classification Task in 3D

Let's consider an example with a dataset for classifying students based on three features; Math, Computer
Science (CS), and English scores. We will use the K-Nearest Neighbors (KNN) algorithm to classify a new
student based on their Math, CS, and English scores.
• Example Training Data:
11: Math 4, CS 3, English 2 - Output: FAIL
___________ 12; Math 6 , CS 7, English 5 - Output: PASS__________________________________________
 13; Math 7, CS 8, English 6 •Output: PASS
14: Math 5, CS 5, English 4 - Output: FAIL
15: Math 8 , CS 8, English 7 - Output; PASS
New Data Point: Math 6. CS 8 , English 6
Find the class label (PASS/FAIL) using KNN algorithm.
Solution:
. Fortha new test instance with Math 6, CS 8, and English 6, the KNN algorithm will classify it l^ findmg
Its threenear^stneighbors based onEuclidean distance in the3DfeaturespaceofMath,KimdEn^ish,
and then determining the majority class among those neighbors to assign the class label (PASS/FAIL).

. Let's choose K = 3 for this example.

. Calculate the Euclidean distance between the new data point and all data points in the training set
Distance Iron. (6 ,8 ,6 ) to II (4 .3 ,2 ): sqrt((4-6)> * C3-8V+ C2-6)>) = sqrt[4 1 2 5 1 16) =sqtt(45) .6 .7 1
Distance from (6 ,8 ,6 ) to 12 ( 6 ,7 ,5 ) : sqrt(C6-6)> * (7-8V * C5-60 = sqrtCO + 1 * 1) = sqrt(2) .1 .4 1
Distance tron. (6 ,8 ,6 ) to 13 ( 7 .8 ,6 ) : sqrt(C7-6)‘ * (8-8)’ + (6-6)-) = sqrt(l . 0 1 0) = sqrt(l) = 1
Distance from (6 ,8 ,6 ) to 14 ( 5 ,5 ,4 ): sqrt((5-6)‘ ♦ (5-8)> * C4-6)‘) = sqrt(l + 9 . 4 ) = sqrt(14) .3 .7 4
Distance from (6 ,8 ,6 ) to IS ( 8 ,8 , 7 ) ; sqrtCC8-6)> * (8-8)' t (7-6)=) = sqrt(4 + 0 +1) = sqrt(5) . 2.24

• Sort the distances in ascending order:

1 (13), 1.41CI2), 2.24(15), 3.74(14), 6.71(11)
. Select the 3 nearest neighbors (we have chosen k=3) based on the calculated distances.

13,12,15
. Select Majority Class:
Among these nearest neighbors; 12,13, and 15 are PASS.
Since all 3 nearest neighbors are classified as PASS, the majority class is PASS.
Therefore, based on the majority class rule, the new student with Hath 6 , CS 8 , and English 6 will be con-ectly
classified as PASS using the KNN algorithm with K = 3

Example I A Python Code for Classification Task Using KNN Classifier

import nutnpy as np
from sk learn .n eig h b o rs import KN eighborsClassifier

# Example Training Data

X_train = n p .a r r a y ([[4 , 3, 2 ] , [ 6 , 7 , 5 ] , [7 , 8 , 6 ] , [5 , 5, 4 ] , [8, 8, 7 ]])
y_train = n p .a r r a y (['F A IL 'j 'P A SS', 'P A S S ', ' FAI L, PASS ] )

# Take input from the user f o r th e new student s scores

math_score = flo at(in p u t("E n te r Math s c o r e fo r the new stu d en t: " ) )
cs_score = flo a t(in p u t("E n te r Computer S cien ce score fo r th e new student: - ) )
e n g lish .s c o re = flo a t(in p u t(-E n ter E n g lish score fo r the new stu d en t: " ) )

new student = n p .a rra y ([[math_score, cs_score, english_score3])__________ __

 ' 'Supervised Learning^ 3.25

# KNN Classifier with K=3

knn = KNeighborsClassifier(n_neighbors=3)
[Link](X_train, y_train)

# Predict the class label for the new student

prediction = knn. predict (new_student)

print("Predicted Class Label:", prediction[0])

Enter Math score for the new student: 6

Enter Computer Science score for the new student: 8
Enter English score for the new student: 6
Predicted Class Label: PASS

Enter Math score for the new student: 4

Enter Computer Science score for the new student: 3
Enter English score for the new student: 5
Predicted Class Label: FAIL
Explanation
T rain in g Data: The example training dataset X_train contains the Math, CS, and English
scores o f 5 students, while y_train holds the corresponding PASS or FAIL labels, enabling the
model to learn from this labeled data.
U ser Inp u t: The code prom pts the user to input the Math, CS, and English scores for a new
student, storing these values in the [Link] array for prediction using the trained KNN
model.
M odel Initialization: The code initializes a KNeighborsClassifier object with n_neighbors=3,
setting up the KNN algorithm to consider the 3 nearest neighbors when making predictions
for the new student
M odel Training: The KNN classifier is trained on the training data X_train and y_train using
the fitO method, allowing the model to understand the relationships between input features
and output classes.
P re d ictio n and Output: The trained KNN model predicts the class label for the new stud ent
based on their input scores, applying the majority class rule of the 3 nearest neighbors.
The predicted class label (PASS o r FAIL) is then displayed as the final output to indicate the
model's classification decision for the new student _________________

Example A Pytlibn Code for Regression .Task [Link]'Regressor

from [Link] import KNeighborsRegressor
import numpy as np

# Example Training Data

X_train = [Link]([[8, 3], [6, 2 ], [3, 7], [2, 8]]) # Sweetness, Acidity
y_train = [Link]([100, 80, 50, 40]) # Price______
 ; '¥ -.-
m i

# User input for sweetness and acidity values of the new fruit
sweetness = float(input ("Enter sweetness value (1-10): "))
acidity = float(input ("Enter acidity value (1-10): "))
X_new = [Link]([[sweetness, acidity]])

# Choose K value for KNN

K = 3

# Create and fit the KNeighborsRegressor model

knn_reg = KNeighborsRegressor(n_neighbors=K)
knn_reg.fit(X_train, y_train)

# Predict the price of the new fruit

predicted_price = knn_reg.predict(X_new)

print(f"The predicted price for the fruit with sweetness {sweetness} and acidity
{acidity} is: Rs.{predicted_price[0]:.2f}")
Output
Enter sweetness value (1-10): 5
Enter acidity value (1-10): 4
The predicted price for the fruit with sweetness 5.0 and acidity 4.0 is: Rs.76.67
Explanation
T raining Data: The example training data consists o f arrays representing sw eetness and
acidity values of fruits (X_train) and their corresponding prices (y_train). This data is used to
train the KNeighborsRegressor model.
U ser Input: The code prom pts the user to input sw eetness and acidity values for a new fruit.
These values are stored in an array X_new for predicting the price of the new fruit based on
the trained model.
M odel Initialization: The code initializes a KNeighborsRegressor object with n_neighbors=3,
setting up the KNN algorithm to consider the 3 nearest neighbors when making predictions
for the new student.
M odel Training: The model is then fitted with the training data (X_train, y_train) to learn the
relationships between fruit characteristics and prices.
P rediction and Output: The model predicts the price o f the new fruit (X_new) using the
predictQ method. The predicted price is displayed._______________________________________

Applications of KNN Algorithm

The real-world applications of the K-Nearest Neighbors (KNN) algorithm:
1. H ealthcare: KNN is used in healthcare for tasks such as disease diagnosis and patient outcome
prediction. By analyzing the medical history and sym ptom s of patients, KNN can assist in
identifying similar cases and recommending appropriate treatments or interventions.
 2 . Finance: In the financial sector, KNN is applied fo r credit scoring, fraud detection, and stock
market analysis. By comparing the financial behavior of customers or detecting unusual
patterns in transactions, KNN helps financial institutions make informed decisions and
mitigate risks.

3 . Retail. KNN is utilized in retail for custom er segm entation, personalized recommendations,
and market basket analysis. By identifying sim ilar customer profiles or recommending
products based on past purchases, KNN enhances th e shopping experience and boosts sales.
4 . Social Media: Social media platforms leverage KNN for friend recommendations, content
filtering, and sentim ent analysis. By analyzing u ser interactions and preferences, KNN suggests
connections, filters news feeds, and categorizes u se r sentiments to enhance user engagement
5. Environmental S cie n c e : In environmental science, KNN is employed for tasks such as species
classification, pollution monitoring, and clim ate modeling. By analyzing environmental data
and patterns, KNN helps researchers predict sp ecies distribution, detect pollution hotspots,
and model climate changes.

Advantages and Disadvantages o f KNN Algorithm

I Advanteges of KNN Algorithm , :

O It is simple to implement.
O Very easy to understand, and often gives reasonable performance without a lot of tuning. >
O It is robust to the noisy training data
O It can be more effective if the training data is large.
O Building the nearest neighbors model is typically fast, although prediction speed may decrease with
very large training sets.

Disadv^tages of KNN Algorithm :•

C The need to determine the value of K can sometimes be challenging.

C The computation cost is high because of calculating the distance between the datapoints for all the
training samples.
C It does not perform well on datasets with many features (hundreds or more).
C The algorithm faces difficulty in proper classification when dealing with high-dimensional data due to
the curse of dimensionality.

o n Linear Models

Linear models are a fundamental class of algorithm s in supervised machine learning that make
predictions by computing a linear combination of the input features. In a linear model, the relationship
betw een the input features and the target variable is represented as a linear function.
T h e general form of a linear model can be expressed as: Y = C„ + C X + _______ + C X
O i l * n n

In the formula, Y and X^,X^..... represent the variables in th e dataset C^, Cj.™C„ are the regression
coefficients that we estimate from the dataset
 1. Y (D ep end entV ariable):
• Y is the dependent variable, also known as the response variable or targ et variable. It's
what you are trying to predict or explain.
• In practical term s, Y could be som ething like the price of a house, the weight of an
individual, the mileage of a car, or any o th er variable that depends on other factors.
2 . X (Independent V ariab les);
• Xj, Xj.....X^ are the independent variables, also known as predictors o r explanatory
variables. These are the variables we use to predict Y.
• Each X, represents a different feature or characteristic. For example, in a model predicting
house prices, X^ might represent the size o f the house, X^ might represent the number of
bedrooms, X3 might represent the age o f the house, and so on.
3. C (Coefficients):
• Cg, Cj are the coefficients or param eters o f the model. They quantify the relationship
between each independent variable and the dependent variable.
• C„ is a special coefficient known as the in te rcep t It represents the expected value of Y
when all the X variables are equal to zero.
• CjC^....Cj^are the slopes for the respective X variables. They represent how much Y is
ej^ected to change with a one-unit change in the corresponding X variable, holding all
< other variables constant.
Linear models are characterized by their simplicity and interpretability, making them widely used in
various machine learning tasks. These models are efficient, easy to implement, and provide insights
into the importance of different features in making predictions.

Classification and Regression Tasks with Linear Models

Linear models in supervised machine learning are versatile algorithms used for both regression and
classification tasks. By understanding the below concepts and examples, we can understand how
linear models are applied in real-world scenarios for regression and classification tasks in supervised
machine learning.
1. Regression w ith L in ea r Models: In regression tasks, linear models predict a continuous
target variable based on input features by fitting a linear relationship betw een the features
and the target. _______________________ _

Jig Examples Regression Tasks with L in ear Models

• Example 1 : Consider a housing price prediction task where the goal is to predict the price of a
house based on features like area, number of bedrooms, and location. Alinear regression model
can be trained to estimate the house price by learning the coefficients for each feature and an
intercept term.
• Example 2 : Predicting student scores based on study hours. Gwen the number of hours a
student studies, a linear regression model can predict the exam score.___________ ^
__________
Algorithm: Linear Regression is a common algorithm for regression tasks th at fits a linear
relationship between features and the target variable.
»1 * ■ ■
■
■
"
''■
’'•9
4
PI'’’'■
•'''
.................../- ' v' - ' - Su p ew s^ ^

2. C lassification w itli L inear M o d e ls: In classification tasks, linear models separate classes by
defining a lin ear decision boundary in the feature space to classify data points into different
categories.

Examples Classifications Tasks with Linear Models

Example 1: For binary classification, consider a spam email detection system where emails
are classified as spam or non-spam. Logistic Regression is a linear model that can be used to
model the probability of an email being spam based on features like ke3rwords and sender
information.
Example 2; In a multi-class classification scenario such as classifying different types of fhiits
(example, apples, oranges, and bananas) based on their features like color, size, and texture.
Example 3 : Classifying whether a transaction is fraudulent based on transaction features like
amount, location, and time. Logistic regression can output the probability of fraud for each
transaction.
A lgorith m : Logistic Regression is com m only used for classification tasks,

Characteristics of Linear Models

Linear models are a fundamental class of algorithm s in supervised machine learning that make
predictions by com puting a linear combination o f the input features. Some key characteristics of
linear models are listed below:
1. Linear R e la tio n sh ip : Linear models assum e a linear relationship betw een the input features
and the target variable. The predicted output is a linear combination of the input features.
Example: In lin ear regression, the relationship between a house's price (target variable) and
features like area, number of bedrooms, and location can be modeled linearly as:
Y = C„ + C j* A rea + * Bedroom s + C3 * L o catio n
2. In terp retab ility : Linear models provide interpretable coefficients that indicate the impact of
each feature on the target variable. A positive coefficient implies a positive relationship, while
a negative coefficient implies a negative relationship.
Example: In logistic regression for spam email detection, a positive coefficient for the
"keyword" feature indicates that the presen ce o f that keyword increases the likelihood of an
email being classified as spam.
3. Scalability: Linear models are com putationally efficient and can handle large datasets with
many features. Training and making predictions with linear models are generally f ^ e r
compared to m ore complex models.
Example: In a sentim ent analysis task vdth a large text dataset, linear models like Linear
Support Vector Machines (SVM) can efficiently classify text data into positive or negative
sentiments.
4. Regularization: Linear models can be regularized to prevent overfitting by penalizing large
coefficients. Regularization techniques like Lasso (LI) and Ridge (L2] regression help in
improving the m odel's generalization.
Example: In Ridge regression, the model penalizes the sum of squared coefficients, encouraging
smaller coefficients and reducing the m odel's complexity.
 5. B in a ry and Multi-Class C lassification: Linear models can be u sed for both binary and multi­
class classification tasks. For binary classification, logistic regression is commonly used,
while strategies like One-vs-Rest or One-vs-One can extend lin e a r models for multi-class
classification.
E xam p le: In a medical diagnosis system , linear models can classify patients into multiple
disease categories based on various medical test results.

Linear Regression
Linear Regression is a supervised machine learning algorithm used fo r predicting a continuous
numerical output based on one or more input features. The algorithm aim s to find the best-fitting
linear relationship between the input features and the target variable.

How Linear Regression Works ?

1. M odel Representation: In linear regression, the relationship b etw een the input features (X)
and the target variable (Y) is represented by a linear equation o f th e form:
Y = C„ + C,X, + ............ + C„X
W here
• Y is the predicted output,
• Cp ,Cj ,C 2 are the coefficients (weights) to be learned,
• Xj ,X^ r- ,\ are the input features.
2. O b jectiv e: The goal of linear regression is to find the values o f coefficients ,C^ ,C^ ,Cj that
minimize the difference between the predicted values and the actu al target values.
3. T ra in in g : The algorithm learns the optimal values of coefficients by minimizing a cost function,
t 3Apically the Mean Squared Error (MSE), which measures th e average squared difference
betw een predicted and actual values.
4. P re d ic tio n : Once trained, the model can make predictions on n ew data by plugging in the
input features into the learned equation.

JSy E xam ple Car Price Prediction Using Linear Regression

Consider a simple linear regression model predicting the price of a car based on its mileage and age:
• Y: Car price (in dollars)
• XjZ Car mileage (in thousands of miles)
• X^: Age of the car (in years)
• C^, Cj, C^: Coefficients to be estimated from data
The model might look something like this:
• Car Price=Cj + x Mileage + x Age
Here, Cj tells us how much the car price decreases for every additional thousand miles driven, and C^ tells us
how much the car price decreases for every additional year of age, with C^ indicating the base price of the car.
The objective is to create a linear regression model that predicts the price of a car, in lakhs of INR, based on
two main factors: its mileage and age. ____________________
 Supervised Learning,|T 3.31

Given D ata Sample:

MUeage (xiOOO kin) / , ^Age (y ear^ ' Price (lakhs INR)

80 5 5
50 2 7
90 4 4.5
30 1 8
70 3 6
Model Training and Coefficients:
• The linear regression model has been trained using the given data, which consists of car mileage in
thousands of kilometers, age in years, and the corresponding prices in lakhs of INR.
• In tercep t (C^^: Approximately 9.75 lakhs. This represents the estimated base price of a car with 0
mileage and 0 age.
• C j: Approximately -0.05. This suggests that for each additional 1000 km, the price decreases by 0.05
lakhs (or Rs. 5000).
• : Approximately -0.15. This indicates that for each additional year, the price decreases by 0.15 lakhs
(o r Rs.15,000).
Using the derived coefficients from the linear regression model and appl)dng them to the formula, the predicted
price o f a car with 45,000 km mileage and 3 years old is calculated as follows:
• Predicted Car Price=Cj + C^ x Mileage + C^ x Age
Predicted Car Price = 9.75+ (-0.05x45)+ (-0.15x3) = 7.05 lak h s
This calculation suggests that, under the model derived from the given data, a car with these specifications
(45,000 km mileage and 3 years of age) is predicted to have a price -of approximately 7.05 lakhs. This price
reflects the combined effect of the car's mileage and age on its overall value in the market

gSjj 'Exam ple House Price Prediction Using Linear R egression

Consider a simple linear regression model predicting the price of a house based on its size (in square feet)
and age:
• Y: House price (in lakhs)
• X^: House size (in square feet)
• X^: Age of the house (in years)
• Cg, Cj, C^: Coefficients to be estimated from data
The model might look something like this:
• House Price = C^ + C^* Size + C^ * Age
Given D ata Sample:

Size (sq.'ft) ■ Age (yean) "Price (In lakhs)

1500 5 200
1200 2 180
1800 4 220
1000 1 160
1400 3 190
Q i

Let's calculate the coefficients CO, Cl, and C2 for the linear regression model predicting the house price based
on size and age using the given data sample step by ^ p :
Step 1: Calculate the Mean Values:
Mean Size = (1500 + 1200 + 1800 + 1000 +1400) / 5 = 1380 sq. ft
MeanAge = (5 + 2 + 4 + l + 3 ) / 5 = 3years
Mean Price = (200 + 180 + 220 + 160 + 190) / 5 = 190 lakhs
Step 2: Calculate the Covariance and Variance:
Covariance(Size. Price) = Z((Size - Mean Size) * (Price - Mean Price)) / (n-1)
Covariance(Size, Price) = [(1500-1380)(200-190) + (1200-1380)(180-190) + (1800-1380)(220-190)
+ (1000-1380)(160-190) + (1400-1380)(190-190)]/4 = 6750
Covariance(Age, Price) = I((Age - Mean Age) * (Price - Mean Price)) / (n-1)
Covariance(Age, Price) = [(5-3)(200-190) + (2-3)(180-190) + (4-3)(220-190) + (1-3)(160-190) +
(3-3) (190-190)] /4 = 30
Variance(Size) = Z((Size - Mean Size)^) / (n-1)
Variance(Size) = [(1500-1380)^ + (1200-1380)^ + (1800-1380)^ + (1000-1380)^ + (1400-1380)^] / 4
= 92000
Variance(Age) = Z((Age - Mean Age)^) / (n-1)
Variance(Age) = [(5-3)^ + (2-3)^ + (4-3)^ + (1-3)^ + (3-3)^] / 4 = 2.5
Step 3: Calculate the Coefficients:
Cl = Covariance(Size, Price) / Variance(Size) = 6750 / 92000 = 0.073
C2 = Covariance(Age, Price) / Variance(Age) = 30 / 2.5 = 12
CO = Mean Price - Cl * Mean Size - C2 * Mean Age
= 190 - 0.073 * 1380 - 1 2 * 3 = 190 - 69 - 30 = 53.26 lakhs
Model Training and Coefficients:
. The linear regression model has been trained using the given data of house size, age, and corresponding
prices.
. Intercept (CJ = 53.26 lakhs: CO represents the intercept of the linear regression model. In this case,
it is 91 lakhs. When both the size and age of the house are zero, the predicted house price is 53.26
lakhs. However, in real-world scenarios, this interpretation may not be meaningful as houses cannot
have zero size or age.
• Cj = 0.073: Cl is the coefficient associated with the size (sq. ft) feature in the linear regression model.
For every one square feet increase in the size of the house, the predicted house price is expected to
increase by 0.073 lakhs (Rs.7300), assuming the age remains constant
. =12: C2 is the coefficient associated with the age (years) feature in the linear regression model. For
every one year increase in the age of the house, the predicted house price is expected to increase by 12
lakhs, assuming the size remains constant
Predicted House Price Calculation: Using the derived coefficients from the linear regression model, the
predicted price of a house with 1300 sq. ft size and 2 years old is calculated as follows:________________
 • House Price = C, + * Size + * Age
House Price = 53.26 + 0.073 * 1300 +12*2=53.26 + 94.9 +24=172.16 lakhs
This calculation suggests that, under the model derived from the given data, a house with these specifications
(1300 sq. ft size and 2 years old) is predicted to have a price of approximately 172.16 lakhs. This predicted
price reflects the combined effect of the house’s size and age on its overall value in the real estate market

Example A Pythbff Code to Prediirt House^nce Using L in e ^ ^gre^Ion ‘

# Import necessary libraries
import numpy as np
from sklearn.linear_model import LinearRegression

# Given data sample

data = [Link]([[1500, 5, 200],
[1200, 2, 180],
[1800, 4, 220],
[1000, 1, 160],
[1400, 3, 190]])

# Separate features (size and age) and target (price)

X = data[:, :2] # Features: size and age
y = data[:, 2] # Target: price

# Create and train the linear regression model

model = LinearRegressionO
[Link](X, y)

# Take user inputs for size and age

new_size = float(input ("Enter the size of the house: "))
new_age = float(input ("Enter the age of the house: "))

# Predict the price for the new data point

predictedjjrice = [Link]([[new_size, new_age]])

# Print the predicted price

print ("Predicted Price for a house with size {} sq. ft and age {} years: {:.2f} Lakhs",
format(new_size, new_age, predicted_price[0])) ______________________
Output

Enter the size of the house: 1300

Enter the age of the house: 2
Predicted Price for a house with size 1300.0 sq. ft and age 2.0 years: 183.57 Lakhs

Note: The differences in predicted prices between the manual calculation and the scikit-learn LinearRegression
model can be attributed to various factors such as the model complexity, feature scaling, model assumptions,
data variability, and the handling of the intercept term.
To align the manual calculation with the model predictions, one would need to adjust the manual calculation
methodology to match the assumptions and processes used by the scildt-leam LinearRegression model.
3,34 X ‘’-‘i •tachine Learning

.Explanation
1. Data Preparation : The given data sample consists of house features (size and age) and the target
variable (price). The features (size and age) are stored in the variable X, while the target prices are
stored in the variable y after separating them from the data sample.
2. Model T rain in g: A Linear Regression model is created using scikit-learn's LinearRegression class and
trained on the features (size and age) and target prices from the given data sample. The model learns
the relationship between the features and the target variable during the training process.
3. User Input: The program prompts the user to enter the size and age of a new house for which they want
to predict the price. The user inputs are stored in the variables new_size and new_age after converting
them to floating-point numbers.
4. Prediction : The trained Linear Regression model is used to predict the price for the new house based
on the user-provided size and age. The model's predictQ method is called with the new feature values
to obtain the predicted price for the new data point
5. Output: Finally, the program prints the predicted price for the new house with the given size and age
in a formatted string, displaying the input values and the predicted price in Lakhs. This allows users
to quickly get an estimate of the house price based on the provided features using the trained linear
regression model.

Logistic Regression
One common method for using regression for classification is logistic reg ression . A logistic regression
is actually a classification algorithm that predicts the probability of an ob servation belonging to a
certain class. The logistic regression model uses a logistic function to map th e output to a probability
value between 0 and 1, making it suitable for binary classification tasks.
For example, in a b inary classification problem w here the goal is to predict w h e th e r an email is spam
or not spam, logistic regression can be used to model the probability o f an em ail being spam based on
features such as the presence of certain keywords, email length, or sen d er information. The output
of the logistic regression model can then be interpreted as the probability o f th e email belonging to
the spam class.
In multi-class classification tasks, multinomial logistic regression can be used to predict the probability
of an observation belonging to each class within the dataset This allow s fo r the classification of
observations into multiple categories based on the highest predicted probability.
It's important to note that while regression for classification can be a u seful technique, there are
also dedicated classification algorithms, such as decision ft-ees, su p p ort vector machines, and
neural networks, th at are specifically designed for handling classification ta sk s and may outperform
regression-based approaches in certain scenarios.
Jig Example Binary Classification using Logistic Regression:
Consider a binary classification problem where the task is to predict whether a student will pass (class 1) or
fail (class 0) an exam based on the number of hours studied. The dataset contains the number of hours studied
by each student and whether they passed or failed.
Data:
• Independent Variable (X): Number of hours studied
Dependent Variable (Y): Pass (1) or Fail (0) ____________________________ _
Supervised le a m in g ^ ^ 3 35
3. 3 6 V M ach in e Leqrning' H f ^ ^ W ^ ^ f e »

Explanation, ______________ __ ______________________________ ______

1. The code defines the dataset for binary classification witii hours studied and pass/Cail labels.
2. It trains a Logistic Regression model using LogisticRegression.
3. User input is taken for the number of hours studied.
4. The model predicts the probability of passing the exam based on the input hours studied.
5. The program outputs whether the predicted class is PASS or FAIL based on the predicted probability
threshold of 0.5. ___________ __ ______________________________ _______________

Example ’ Multi-Class Classification using Multinomial Logistic Regression

In a multi-class classification scenario, multinomial logistic regression can be used to predict the probability
of an observation belonging to each class within the dataset Let's consider a scenario where the task is to
classify images of fhiits into three categories: apples, oranges, and bananas based on their size and color.
• Data:
. Independent Variable (X): S iz e and color features of the fruits
. Dependent Variable (Y): Categories (apples, oranges, bananas)
• Multinomial Logistic Regression M odel:
• The multinomial logistic regression model predicts the probabilities of the fhiit being an apple,
orange, or banana based on its features.
• The model's output will include the predicted probabilities for each class.
• Prediction:
. Given a new fruit with specific size and color features, the model predicts the probabilities of it
being an apple, orange, and banana
• Classification: '
• The fruit is classified into the categoty with the highest predicted probability_______________

Example A Python Code to Demonstrate Multi-Class Classification using Logistic Egression

import numpy as np
from sklearn.linear_model import LogisticRegression

# Data for multi-class classification (fruit size and color)

X_multi = [Link]([[l, 0], [0, 1], [1, 1], [0, 0]]) # Size and color features
y_multi = [Link]([0, 1, 2, 0]) # Categories: 0=apples, l=oranges, 2=bananas

# Train a Multinomial Logistic Regression model for multi-class classification

model_multi = L o g i s t i c R e g r e s s i o n (multi_class='multinomial', solver='lbfgs')
m o d e l_ m u lt i.f it ( X _ m u lt i, y _ m u lti)

# User input for fruit features

size = float (input ("Enter the size of the fruit (0-1): "))
color = float(input("Enter the color of the fruit (0-1): "))
fruit_features = [Link]([[size, color]]) _______________________ ______
 7 \ '•.- '", ‘ Suj^rins^ Leqrning ^^3^
m
# Predictions
■ulti_prediction = model_multi.predict_proba(fruit_features)

# Output
print ("Predicted probabilities for each class:")
for i, prob in enuraerate(multi_prediction[0]):
print(fProbability of class {i}: {prob}")
Output
Enter the size of the fruit (0-1): 1
Enter the color of the fruit (0-1): 1
Predicted probabilities for each class:
Probability of class 0: 0.3660619419625619
Probability of class 1: 0.2542368791495831
Probability of class 2: 0.37970117888785493

Enter the size of the fruit (0-1): 1

Enter the color of the fruit (0-1): 1
Predicted probabilities for each class:
Probability of class 0: 0.3660619419625619
Probability of class 1: 0.2542368791495831
Probability of class 2: 0.37970117888785493
E x p ira tio n .;
1. The code defines the dataset for multi-class classification with fruit size and color features.
2. It trains a Multinomial Logistic Regression model using LogisticRegression with the multi_
class='multinomiar parameter.
3. User input is taken for the size and color of a new fhiit
4. The model predicts the probabilities for the fruit being an apple, orange, or banana based on the input
features.
5. The program classifies the fhiit into the category with the highest predicted probability._____________

Applications of Linear Models

Linear m od els such as Linear Regression and Logistic Regression are widely used in various
applications acro ss different fields due to their simplicity, interpretability, and efficiency. Some
common applications of linear models:
1. Predictive Modeling:
• L in e a r Regression: Used for predicting continuous outcomes, such as house prices,
sto c k prices, sales forecasts, etc.
• L o g istic Regression; Applied in binaiy-classification tasksrsuch as predicting whether
an email is spam or not, custom er churn prediction, disease diagnosis, etc.
2. Marketing and Business:
• C u sto m er Segm entation: Linear models can help identify customer segments based on
demographic or behavioral data.
3.38 v :

• Market Basket Analysis: Predicting which products are likely to be purchased together
in retail settings.
3. Finance:
. Risk Assessment: Predicting credit risk, loan default probabilities, insuran ce claim
likelihood, etc.
• Stock Market Analysis: Forecasting stock prices o r identifying trading opportunities.
4. Healthcare:
• Disease Prediction: Using logistic regression to predict the likelihood of a p atien t having
a particular disease based on symptoms and medical history
• Drug Response Prediction: Predicting how patients will respond to different treatm ents
based on their characteristics.
5. Recommendation Systems:
• Collaborative Filtering: Linear models can be used in recommendation system s to
predict user preferences based on historical data.
6. Natural Language Processing (NLP):
• Text Classification: Logistic Regression is commonly used for sentim ent analysis, spam
detection, and text categorization tasks.

Advantages and Disadvantages of Linear Models

Adv^ntages'ofJ^inear Mpdels , f/ >i ^=-• -. •

3 Linear models are easy to Interpret and understand. The coefficients provide insights into the
relationship between input features and the target variable.
O Training and making predictions with linear models is computationally efficient, especially for large
datasets.
O Linear models can handle large datasets with high-dimensional feature spaces.
O Linear models can help identify the most important features influencing the target variable.
O Linear models serve as a good baseline model for more complex algorithms, helping to compare
performance. _________________________________ _____ ______________

Disadvantages of Linear Models

C Linear models assume a linear relationship between features and the target variable, which may not
hold true for complex datasets.
C Linear models may not capture intricate relationships in the data compared to non-hnear models like
decision trees or neural networks.
C Outliers can significantly impact the coefficients in linear models, affecting model performance.
C Handling categorical variables in linear models may require encoding techniques like one-hot encoding,
which can increase dimensionality._______ _________ _______________________________________
 ’ • Supervised learni

Naive Bayes Classifiers

Naive Bayes classification is a probabilistic method for categorizing data points based on Bayes'
theorem, which establishes a connection between the present data and existing assumptions or
beliefs. It involves updating p rior beliefs (prior probabilities of class labels) based on the observed
evidence (features of the input data) to make predictions.
It is commonly used for binary and multi-class classification problems. Naive Bayes classificationis
particularly popular in natural language processing tasks like spam filtering and document
categorization.

Naive Bayes classification depends on the principle of Bayes’ Theorem. Before moving to the Naive
Bayes, it is important to know about Bayes’ theorem.

Bayes'Theorem

Bayes' theorem describes th e probability of an event, based on prior knowledge o f conditions that
might be related to the event. In the context of classification, Bayes' theorem is used to calculate
the probability of a class label given the observed features. In the context of classification, it can be
expressed as:
Bayes' Theorem Formula: P(A|B) = (P(B|A) x P(A))/ P(B)
where:

o P (A|B) : The probability o f hypothesis A given the evidence B. (Posterior Probability)

o P (B|A) : The Probability of evidence B given the hypothesis A. (Likelihood Probability)
o P (A) : Prior probability o f hypothesis A
o P (B): Prior Probability o f evidences.

qSq Example 1 Medical Diagnosis

Scenario:
• Hypothesis (A): A patient has a particular disease.
• Evidence (B): The results of a diagnostic test for the disease.
Probabilities:
• Prior Probability (P(A)): Prior Probability that the patient has the disease = 0.01 (1% of the
population has the disease).
• Likelihood (P(B|A)): 0.95 (95% chance of a positive test if the patient has the disease).
• Evidence Probability (P (B )): 0.02 (2% false positive rate).
Calculation:
P(A|B) =(P(B|A) x P(A))/p (B)
= (0.95 X Q.01)/0.02
= (0.0095/0.02)
= 0.475
The calculated probability P(A|B) is 0.475, or 47.5%. This means that there is a 47.5% chance that the patient
actually has the disease given that they have tested positive on the diagnostic test
 £\anipl(‘ 2 Email SPAM Probability
............ . ___
Problem: Consider an email system where:
• 30% of all emails are spam.
• If an email is spam, there's a 40% chance it contains the word "free".
• If an email is not spam, there's a 10% chance it contains the word "free".
Calculate the probability that an emaU is spam given that it contains the word "free".
Scenario:
• H)T>othesis (A): An email is spam. (Here A means Spam)
• Evidence (B): The email contains the word "free". (Here B means free)
Probabilities:
• Prior Probability (P(A) = P(Spam)) : Probability that an email is spam = 0.30 (30% of all emails are
spam).
. Likelihood (P(B/A) = P(Free|Spam)): Probability that an email contains the word "free" given it is
spam = 0.40.
• Evidence Probability (P(B) = P(Free)): Probability that an email contains the word "free" = P(Free).
Calculation to Determine P(B) i.e P(Free):
• The probability of an email containing the word "free" (P(Free)) can be calculated using the total
probability rule, which considers both the likelihood of "free" appearing in spam and non-spam emails,
weighted by the overall probability of any email being spam or not:
P(Free) = P(Free/Spam)xP(Spam)+P(Free/NotSpam)xP(NotSpam)
= 0.40x0.30+0.10x0.70
= 0.12+0.07
= 0.19
Calculate P(A|B) Using Bayes' Theorem:
• Now, apply Bayes' Theorem to calculate the probability that an email is spam given that it contains the
word "free";
P(Spam/Free) = (P(Free|Spam)xP(Spam))/ P(Free)
= (0.40 X0.30)/0.19
= 0.12/0.19
= 0.6315
The probability that an email is spam given that it contains the word ’free" is approximately 63.15%.______

N a iv e B a y e s C la s s ifie r
Naive Bayes is a sim ple and powerful classification algorithm based on Bayes' Theorem with an
assumption o f independence between features. The assumption of independence between features
means that the presence of a particular feature in a class is independent of the presence of any other
feature. This assum ption simplifies the calculation o f probabilities by assum ing th at the effect of one
feature on the class is independent of the presence o f other features.
 '5;'jr: V -. - '
Supervised Learn il

Example: Consider a text classification task w here we want to classify em ails as spam or not spam
based on the p resen ce of two features: the words "discount" and "offer". The independence assumption
implies that the occurrence of the word "discount" in an email does not affect the occurrence of the
word "offer" in th e sam e email when determining if the email is spam or not.

What is Naive Bayes Classifier ?

The Naive Bayes Classifier is a probabilistic machine learning model that's used for classification tasks. It
is based on Bayes' Theorem with the assumption that means that the presence of a particular feature in a
class is independent of the presence of any other feature.
The Naive Bayes Classifier algorithm is comprised of two words Naive and Bayes:
• Naive: It is called Naive because it assumes that the occurrence of a certain feature is independent
of the occurrence of other features. Such as if the fruit is identified on the bases of color, shape, and
taste, then red, spherical, and sweet fruit is recognized as an apple. Hence each feature individually
contributes to identify that it is an apple without depending on each other.
• Bayes: It is called Bayes because it depends on the principle of Bayes’ Theorem.
It is widely used in text classification, spam filtering, and recommendation systems due to its efficiency
and effectiveness in handling high-dimensional data.

^ -Examples Naive Bayes Classifier

1. Example of Naive Bayes Classifier for Fruit Classification:
Features: Color (Red, Yellow) and Shape (Round, Oval).
Training Data: Red, Round -> Apple
Yellow, Oval -> Orange
Prediction: Fruit is Red and Round, classify as Apple or Orange using Naive Bayes.
2. Example of Naive Bayes Classifier for Holiday Plan:
Features: Temperature (Hot, Cold) and Weather (Sunny, Rainy).
Training Data: Hot, Sunny -> Beach
Cold, Rainy -> Resort
Prediction: Given Hot and Sunny weather, classify as Beach or Resort using Naive Bayes.
3. Example of Naive Bayes Classifier for Email Spam Detection:
Features: Words in the email.
Training Data: Spam Email: "Get rich quick money", "huge discount offer", "free iphone"
Non-Spam Email: "Meeting scheduled for tomorrow at 10 AM"
Prediction: iven an email with the content "Get rich quick money" or "huge discount offer" or "free
iphone", classify as Spam or Non-Spam using Naive Bayes.

The Naive Bayes Classifier works by calculating the posterior probability of each class label given the
input features using Bayes' Theorem. It assumes feature independence, simplifying the calculation
by considering each feature's contribution to the class probability Independently. By multlpl3 dng the
likelihood of each feature given the class label with the prior probability o f the class, the classifier
 orninq>-*v..'--,!'.Sfr^ ■ -. ~, . * --. o'--vs>?i ■■</*)»■.?v

determines the m ost probable class for th e input data. This approach enables efficient and effective
classification, making Naive Bayes a popu lar choice for text classification, spam filtering, and other
machine learning tasks.
Working Principle:
1. Bayes' Theorem: Bayes' Theorem calculates the probability o f a hypothesis (class label) given
the data (features]. Mathematically, it is represented as:
P(A|B) = (P(B|A )xP(A ))/P(B)
where;
o P (A|B) : The probability o f class A given the data B. (Posterior Probability)
o P (BIA) : The Probability o f d ata B given the class A. (Likelihood Probability)
o P (A) : Prior probability o f cla ss A.
o P (B): Prior Probability o f class B.
2. Naive Bayes Assumption:
• Naive Bayes simplifies the computation of P(BIA) by assuming that all features in B (such
as words in an email) are independent of each other given the class A. This assum ption
allows the model to tre a t each feature separately, which simplifies the calculations
drastically.
3. Classification Process:
. Given a set of features X = {x^, x , , ..., x j and a set of class labels C = {c,, c^.....c J , the Naive
Bayes classifier predicts th e m ost probable class label for the input features.
• The classifier calculates th e posterior probability for each class label and selects the class
with the highest probability.
4. Model Training - Calculating Probabilities
• Calculate Prior Probabilities: This is the probability o f each class in the training dataset
like P(A), P(B) etc..
• Calculate Likelihoods P(B,|A): This involves calculating the probability of each feature
B. given each class A.
5. Calculating Likelihood Product
. Given a set of features X = {x^ , x^ ,...x^}, the likelihood of the features given a class is
calculated by multiplying th e probabilities of each independent feature:
The likelihood product P(X|C J is calculated by assuming feature independence:
P(X1CJ = P (x JC J X P (x,| C J X...............X P(xJC,)
• Each term P (xJC J is the probability of feature x, given class C^.
6. Calculating Probability of the features P(X)
• The generic formula for th e total probability of a feature set X in the context o f Naive
Bayes classification is given by:
P(X) = P (X I Cj ) x P ( c J + PC X I c, )xP(c,) + ............ + P( X I c, )x P (c J
 .'S u p e r v ise d Learnin g

Where,
o P(X|Cj) is the probability of observing the feature set X given class c ..
o P(Cj) is the p rio r probability of class c..
o k is the total n u m b e r of classes.
7. Calculating Posterior Probabilities for Classification
• For each class C^, calcu late the posterior probability that a given set of features X belongs
to class using the form ula derived from Bayes' theorem :
P(CJX) = (P (X | C JX P (C J3/ P (X )
where:
o P(C^|X) is the p o sterio r probability of class given the features X.
o P(X|CJ is the likelihood of the features given class C^.
o P(C J is the p rio r probability of class
o P(X) is the probability of the features.
8. D ecision Rule:
• The Naive Bayes cla ssifier selects the class label C^ that maximizes the posterior
probability P(CJX).
• Select the class label w ith the highest posterior probability as the predicted class.

'Example Problem: Spam Email Detection

Problem:
Suppose we want to classify emails as either spam or not spam based on two features: the presence of the word
"free" and the presence of the word "money". Let's use the following hypothetical data to train our Naive Bayes
Classifier:
• Spam Emails (Total = 100)
o "free": 40 occurrences
o "money": 30 occurrences
• Non-Spam Emails (Total = 100)
o "free": 10 occurrences
o "money": 20 occurrences
• Prior Probabilities
o 50% of the emails are spam,
o 50% of the emails are not spam.
We receive a new email that contains both "free" and "money". We want to classify this email as either spam or
not spam using a Naive Bayes Classifier.
Solu tion:
• Given Data:
o Class C: {C j.C J => {S p a m , Not Spam}
o Features X: X = { x ,, x ,) => ( "free", "money"}____________________________________________
 o Total Spam Emails (Spam): 100
o Total Non-Spam Emails (Not Spam): 100
o Occurrences of "free" in Spam Emails: 40
o Occurrences of "money" in Spam Emails: 30
o Occurrences of "free" in Non-Spam Emails: 10
o Occurrences of "money" in Non-Spam Emails: 20
o Prior Probability of Spam (P(Spam)): 0.5
o Prior Probability of Not Spam (P(NotSpam)): 0.5
Step 1: Calculate Likelihoods:
Likelihood of "free" given Spam (P(free|spam)): = 40/100 = 0.4
Likelihood of "money" given Spam (P(money|spam)): = 30/100 = 0.3
Likelihood of "free" given Non-Spam (P(free|not spam)): = 10/100 = 0.1
Likelihood of "money" given Non-Spam (P(money|not spam)): = 20/100 = 0.2
Step 2: Calculating Likelihood Product
Given the new email features "free" and "money", we need to calculate the posterior probability
for each class (spam and not spam).
o Class C: {C j, C^) => {Spam, Not Spam}
o Features X: X = { Xj,x^) => { "free", "money"}
The likelihood product P(X|CJ is calculated by assuming feature independence:
P(X|C,) = P (xJC J X P (xJC J X.............X P(xJC J
P(X|spam) = P(free|spam) x P(money|spam) = 0.4 x 0.3 = 0.12
P(X|not spam) = P(free|not spam) x P(money|not spam) = 0.1 x 0.2 = 0.02
Step 3: Calculating Probability ofthe features P(X)
Total Probability of all features (P(X))
P(X) = P(X|spam) X P(spam) + P(X|not spam) * P(not spam)
= (0.12 X 0.5) + (0.02 X 0.5)
= 0.06 + 0.01
= 0.07
Step 4: Calculating Posterior Probabilities for Classification
• For each class calculate the posterior probability that a given set of features X belongs to class
using the formula derived from Bayes' theorem:
P(CJX) = (P(X|CJxP(CJ)/P(X)
• P(spam|X) = (P(X|spam) x P(spam))/ P(X)
= (0.12 X 0.5)/0.07
= 0.06/0.07
= 0.8571
 • P(not spam|X) = (P(X|not spam) x P(not spam))/ P(X)
= (0.02 X0.5)/0.07
= 0.01/0.07
= 0.142
Step 5: Decision Rule
• Choose the class with the highest posterior probability.
» Given P[spam|X) = 0.8571 and P(not spam|X) = 0.142, the email is classified as spam.

Example A Python Code fpy Spam Eniail Qet^gtion using thfiN^ve Bayes classification algorithm
# Import necessary libraries ,,
from sklearn.naive_bayes import MultinomialNB
from sklearn.feature_extraction.text import CountVectorizer
import numpy as np

# Given Data
X_train = [Link](["free money", "click here for free", "lottery", "buy now", "amazing
offer"])
y_train = [Link]([1, 1, 1, 0, 0]) # i for spam, 0 for non-spam

# Transform text data into numerical features using CountVectorizer

vectorizer = CountVectorizer ()
X_train_counts = vectorizer.fit_transform(X_train)

# Train a Multinomial Naive Bayes classifier

clf = MultinomialNB()
[Link](X_train_counts, y_train)

# Take input from the user for a new email

new_email_text = input ("Enter the text of the new email: ")
new_email = [new_email_text]

# Transform the new email using the same vectorizer

new_email_counts = [Link](new_email)

# Predict the class of the new email

predicted_class = clf .predict(new_email_counts)

# Output the classification result

if predicted_class[0] == 1:
classification = "Spam"
else:
classification = "Not Spam"

print("The email is classified as:", classification)

 Enter the text of the new email: you won a lottery
The email is classified as: Spam

Enter the text of the new email: Buy iphone at amazing offer
The email is classified as: Not Spam
Explanation
1. Data Preparation: The code initializes training data X_train containing email text and y_train
containing corresponding labels (1 for spam, 0 for non-spam). It uses CountVectorizer to transform the
text data into numerical features. This step converts the text data into a matrix of token counts.
2. Model Training: A Multinomial Naive Bayes classifier (MultinomialNB) is instantiated and trained
on the transformed training data (X_train_counts] and labels (y_train). Naive Bayes classifiers are
commonly used for text classification tasks like spam detection due to their simplicity and effectiveness
with text data.

3. User Input and Prediction: The code prompts the user to enter the text of a new email for classification.
The new email text is transformed using the same CountVectorizer instance to convert it into numerical
features (new_email_counts).

4. Classification: The trained classifier predicts the class of the new email by calling predict on the
transformed new email data (new_emaiI_counts). If the predicted class is 1, the email is classified as
"Spam"; otherwise, it is classified as "Not Spam".

Types of Naive Bayes Classifiers

There are several types of Naive Bayes classifiers commonly used in machine learning. The choice of
which Naive Bayes classifier to use depends on the nature of the data and the assumptions that can be
made about the features. Here are some of the popular types of Naive Bayes classifiers:
1 . Gaussian Naive Bayes:
Assumes that continuous features follow a Gaussian distribution.
• Suitable for continuous features.
2. Multinomial Naive Bayes:
• Assumes that features follow^ a niuitinomial distribution.
• Commonly used for text classification tasks where features represent word counts or
frequencies.
3. Bernoulli Naive Bayes:
• Assumes that features are binary (Bernoulli distributed).
• Suitable for binaiy feature vectors, such as presence or absence o f a feature.
4. ComplementJyiaive-Bayes:
• A variation of Multinomial Naive Bayes that is particularly suited for imbalanced datasets.
• It adjusts the probabilities for each class based on the class frequencies.
 Supervised L e a r n in g ^ 3 .4 7

5. Categorical Naive Bayes:

• Suitable for categorical featu res th at do not have a natural ordering.
• It can handle features with m o re than two categories.
6. Mixed Naive Bayes:
• Allows for a combination o f d ifferen t types of features, such as continuous, binary, and
categorical features.
• Each feature type is modeled using the appropriate distribution.
Each type of Naive Bayes classifier makes d ifferen t assumptions about the distribution of features and
is suitable for different types of data. It is im portant to choose the appropriate Naive Bayes classifier
based on the characteristics of the d ataset to achieve optimal perform ance in classification tasks.

Applications of Naive B ayes Classifiers

Naive Bayes classifiers are popular in various applications due to th eir simplicity, efficiency, and
effectiveness in many scenarios. Some com m on applications of Naive Bayes classifiers are listed
below:
1. Text Classification: Naive Bayes classifiers are widely used in text classification tasks such as
spam email detection, sentim ent analysis, document categorization, and language detection.
2. Spam Filtering: Naive Bayes classifiers are particularly effective in spam filtering applications
where em ails or messages are classified as spam or non-spam based on the presence of certain
keywords or features.
3. Medical Diagnosis: Naive Bayes classifiers can be used in medical diagnosis systems to
predict the likelihood of a patient having a particular disease based on symptoms and medical
test results.
4. Recommendation Systems: Naive Bayes classifiers can be employed in recommendation
systems to predict user preferences and provide personalized recommendations for products,
movies, o r content
5. Fraud Detection: Naive Bayes classifiers are utilized in fraud detection systems to identify
potentially fraudulent transactions or activities based on historical patterns and features.
6. Document Classification: Naive B ayes classifiers are used in docum ent classification tasks to
autom atically categorize docum ents into predefined classes or topics.
7. Sentiment Analysis: Naive Bayes classifiers are applied in sentim ent analysis to determ ine
the sentim ent (positive, negative, n eu tral) of text data such as reviews, social media posts, or
custom er feedback.
8. Customer Segmentation: Naive B ayes classifiers can be used for customer segmentation in
marketing to group customers b ased on their behavior, preferences, or demographics.
9. Fault Diagnosis: Naive Bayes classifiers are employed in fault diagnosis systems to identify
faults o r anomalies in machinery, equipm ent, or systems based on sen sor data and operational
param eters.
10. Biometric Authentication: Naive B ayes classifiers can be utilized in biometric authentication
systems for tasks such as fingerp rint recognition, face recognition, or iris recognition.
 M achine

Advantages and Disadvantages of Naive Bayes Classifiers

Advantages of Naive Bayes C^ssifi^rs. -;.'‘'ii:;.-

O Naive Bayes classifiers are simple and easy to implement. It is suitable for quick prototj^ing and
baseline classification tasks.
O They are computationally efficient and can handle large datasets with high-dimensional feature spaces.
O Naive Bayes classifiers scale well with the size of the dataset and are particularly useful for text
classification and other high-dimensional data
O The probabilistic nature of Naive Bayes classifiers provides a clear interpretation of the classification
decisions based on probabilities.
O Naive Bayes classifiers can handle missing values in the dataset without the need for imputation
techniques.

Disadvantages of Naive Bayes Classifiers' ^ v

C The assumption of feature independence may not hold true in real-world datasets, leading to suboptimal
performance in some cases.
C Due to the simplicity of the model. Naive Bayes classifiers may not capture complex relationships
between features.
C Outliers or extreme values in the data can affect the performance of Naive Bayes classifiers
C If a categorical variable in the test data has a category that was not observed in the training data, the
model assigns a zero probability, leading to incorrect predictions.
C Naive Bayes classifiers require a relatively large amount of training data to estimate the probabilities
accurately, especially for rare classes or features.____________________________________________

D e c is io n T r e e s
Decision tree-based algorithms use a tree-like model to make decisions based on input data. The
tree-like model consists of a series o f nodes that represent decisions or tests on the input data, and
branches that represent the possible outcomes of those decisions or tests. The leaves o f th e tree
represent the final decision or prediction.
The process of building a decision tree-based algorithm involves selecting the best attrib u te to split
the data at each node, based on a m easure of information gain or impurity reduction. T he goal is
to create a tree that is as small as possible while still accurately classifying or predicting th e target
variable.
There are several popular decision tree-based algorithms, including IDS, C4.5, and CART. Each
algorithm has its own strengths and weaknesses, and the choice of algorithm depends on th e specific
problem and data set.
Decision tree-based algorithms are widely used in a variety of applications, including classification,
regression, and feature selection. T hey are particularly useful for problems with a large num ber
of features or complex decision boundaries, as they can capture non-linear relationships and
interactions between features.
One o f the main advantages o f d ecisio n tree-based algorithms is their interpretability. The resulting
tree can be easily visualized and understood, making it useful for explaining the reasoning behind
the model's predictions. However, decision tree-based algorithm s can also be prone to overfitting,
especially when the tree is too large or the data set is noisy. Regularization techniques, such as
pruning or ensemble methods, ca n help to mitigate this issue.

mmm The Decision T ree Algorithm

The decision tree algorithm is a popular method for predictive modeling and classification tasks
It involves the construction o f a tree-like structure to make decisions based on the features o f the
dataset. The process of building a decision tree can be explained using the following algorithm:
Algorithm Decision Tree Algorithm
1. Begin the tree with the root node, denoted as S, which contains the complete dataset.
2. Find the best attribute in the dataset using an Attribute Selection Measure (ASM). The ASM is a
criterion used to select the attribute that provides the best split for the dataset Common measures
include information gain, Gini impurity, and gain ratio.
3. Divide the dataset S into subsets that contain possible values for the best attribute found in step 2. Each
subset represents a branch of the decision tree based on the values of the selected attribute.
4. Generate a decision tree node that contains the best attribute. This node becomes an internal node in
the decision tree and represents a decision point based on the selected attribute.
5. Recursively make new decision trees using the subsets of the dataset created in step 3. Continue
this process until a stage is reached where we cannot further classify the nodes, and these nodes are
called leaf nodes. The leaf nodes represent the final outcomes or decisions based on the features of the
dataset
6. The process continues until the entire dataset is classified into their respective categories at the leaf
nodes, and the decision tree is fully constructed.
The decision tree algorithm follow s a recursive process of selecting the best attribute, dividing the
d ataset into subsets, and generating decision tree nodes until the leaf nodes are reached, representing
the final outcomes. This algorithm is used to construct a decision tree that can be used for classification
and prediction tasks.

Attribute Selection Measure (ASM)

The a ttrib u te selection m e a s u r e (ASM) is a criterion used in decision tree algorithms to select the
b est attribute for splitting the d ata a t each node. The ASM assigns a score to each attribute based on
its ability to divide the data into su b sets that are more hom ogeneous in terms of the target variable.
The attribute with the highest sco re is selected as the splitting attribute for that node.
The goal of the ASM is to find the attribute that provides the m ost information gain or the b e st split
for the data. The Gini Index, Gain Ratio, and Information Gain are the most widely used selection
m etrics.

1. Information Gain

In physics and m athem atics, entropy is referred to as the randomness or the im purity in a
system. In information theory, it refers to the impurity in a group of examples.
 ^ c ii in e le a r n i n g '■ ' ^

An entropy is a m easure of the impurity or random ness of a dataset and it is used in decision
tree algorithms to evaluate the effectiveness o f attributes in partitioning the data into more
homogeneous subsets with respect to the ta rg e t variable. A lower entropy indicates a more
homogeneous subset, while a higher entropy in d icates a more heterogeneous subset.
we follow these steps:
To calculate the information gain for attribute A,
1. Calculate the entropy of the original dataset D.
Entropy (D) = - Zi Pi logj p

Where P. is the probability that an a rb itrary tup le in D belongs to class Cj.

The generic formula for calculating the en tro p y of a dataset D with resp ect to a binary
class label (example, "Yes" or "No") is given by:
Entropy(D) = -p(Yes) log^ pCYes) - p(No) log^ p(No)
where:
• p(Yes) is the proportion of exam ples in D that have a class label o f "Yes,"
• pfNo) is the proportion of exam ples in D that have a class label o f "No,"
• log^ is the base-2 logarithm.
This formula represents the measure o f im pu rity or randomness in the dataset D with
respect to the binary class label. A low er entro p y value indicates a m ore homogeneous
or pure dataset, while a higher entropy valu e indicates a more heterogeneous or impure
dataset.
2. Calculate the average entropy after partitioning the dataset based on the values of
attribute A.
Entropy^(D) = £ ^ i ! j ^ X Entropy(Dj) (or)

Entropy^(D) = (|DJ/|D|) * Entropy(D,) + (|DJ/|D|) * Entropy(D^) +... + (|DJ/|D|) * Entropy(D,)

where D^, D^,..., Di^are the subsets of D th a t correspond to each value o f attribute A, and
|Dj|, ID jI,..., |D^| are the sizes of those su b sets.
3. Compute the information gain using the formula:
Gain(A) = E ntropy(D ) - Entropy^(D)
The attribute A with the highest inform ation gain, Gain(A), is chosen as the splitting
attribute at a particular node in the d ecisio n tree. This means that the attribute that
provides the m ost reduction in entropy o r th e most effective partitioning of the data is
selected for splitting at that node.

tAs Example Understanding Entropy

Entropy is a measure of the impurity or randomness of a dataset. In the context of decision tree algorithms,
entropy is used to evaluate the effectiveness of attributes in partitioning the data into more homogeneous
subsets with respect to the target variable.
The formula for entropy is: Entropy(D) = -p(Yes) lcg2 p(Yes) - p(No) log2 p(No)
where D is the dataset, p(Yes) is the proportion of examples in D that have a class label of "Yes," and p(No) is
the proportion of examples in D that have a class label of "No." _______________
I Supervised I

Problem 1: Suppose we have a dataset of 10 examples, each with a binary class label ("Yes" or "No"].
There are 6 examples with a class label of "Yes" and 4 examples with a class label of "No." The entropy
of the dataset is:
Entropy(D) =-6/10 log2(6/10) - 4/10 log2(4/10) = 0.971
This indicates that the dataset is relatively impure or random, with a high degree of uncertainty about
the class labels.
Problem 2: Suppose we have a dataset of 10 examples, each with a binary class label ("Yes" or "No").
There are 10 examples with a class label of "Yes" and 0 examples with a class label of "No." The entropy
of the dataset is:
Entropy(D) = -10/10 log2(10/10) - 0/10 log2(0/10) = 0
This indicates that the dataset is completely pure o r homogeneous, with no uncertainty about the dass
labels.
Problem 3: Suppose we have a dataset of 10 examples, each with a binary class label ("Yes" or "No").
There are 5 examples with a class label of "Yes" and 5 examples with a class label of "No." The entropy
of the dataset is:
Entropy(D) = -5/10 log2(5/10) - 5/10 log2(5/10) = 1
This indicates that the dataset is relatively impure or random, with a high degree of uncertainty about
the class labels.

1^ Example Information Gain

Suppose we have a dataset of 10 examples with two attributes: "Oudook" and "Temperature", and a binaiy
class label "PlayTennis". We want to decide the best attribute to split the data on in order to create a decision
tree.
•Example Temperature PlayTemiis
1 Sunny Hot No
2 Sunny Hot No
3 Overcast Hot Yes
4 Rainy Mild Yes
5 Rainy Cool Yes
6 Rainy Cool No
7 Overcast Cool Yes
8 Sunny Mild No
9 Sunny Cool Yes
10 Rainy Mild Yes
We want to determine which attribute ("Outlook" or "Temperature") is the best to split on based on information
gain.
Step-by-step Calculation:
Step 1: Calculate the entropy of the entire dataset based on the class label "PlayTennis".
Step 2: Calculate the information gain for the attribute "Outiook".
Step 3: Calculate the information gain for the attribute "Temperature".
Step 4: Choose the attribute v«th the highest information gain as the root of the decision tree.
 M achine L earning

Let's calculate the information gain step by step for this problem.
1. Calculate the entropy of the entire dataset based on the class label "PlayTennis :
-f There are 6 examples with TlayTennis=Yes" and 4 examples with "PlayTennis=No".
-f The entropy of the dataset is:
♦ Entropy(D) = -6/10 log2(6/10) - 4/10 log2(4/10) = 0.9709
2. Calculate the information gain for the attribute "Outlook":
There are 4 examples with "Outlook=Sunny", 3 examples with "Outlook=Overcast", and 3 examples
with "Outlook=Rainy".
-f Calculate the entropy of each subset based on the class label PlayTennis :
♦ Subset with "Outlook=Sunny":
^ There are 2 examples with "PlayTennis=No" and 2 examples with "PlayTennis=Yes".
The entropy of this subset is;
A Entropy(Dl) = -2/4 logZ(2/4) - 2/4 log2(2/4) = 1
♦ Subset with "Outlook=Overcast":
^ There are 0 examples with "PlayTennis=No" and 3 examples with "PlayTennis=Yes".
•/ The entropy of this subset is:
Entropy(D2) = -0/3 lcg2C0/3) -3/3 log2(3/3) = 0
♦ Subset with "Outlook=Rainy":
•y There are 2 examples with "Pla5 ^ennis=No" and 1 example with PlayTennis=Yes .
The entropy of this subset is:
^ Entropy(D3) = -2/3 log2(2/3) -1/3 log2(l/3) = 0.9183
Calculate the weighted average entropy of the subsets:
Weighted Average Entropy=(4/10 * Entropy(Dl)) + (3/10 * Entropy(D2)) + (3/10 * Entropy(D3))
= (4/10 * 1) + (3/10 * 0) + (3/10 * 0.9183)
= 0.5509
-f Calculate the information gain of the attribute "Outlook":
Information Gain(Outiook) = Entropy(D) - Weighted Average Entropy
= 0.9709 - 0.5509
= 0.4200
3. Calculate the information gain for the attribute "Temperature":
There are 4 examples with "Temperature=Hot", 2 examples with "Temperature=Mild", and 4 examples
with "Temperature=Coor'.
-f Calculate the entropy of each su b set based on the class label "PlayTennis :

♦ Subset with "Tem perature=H ot":

^ There are 1 example with "PlayTennis=No” and 3 examples with "PlayTennis=Yes".

The entropy of this subset is:

___________ A Entropy(Pl] = -1/4 log2(l/4) - 3/4 log2(3/4) = 0.8113___________________
 ♦ Subset with "Temperature=Mild":
'/' There are 1 example with "PIayTennis=No" and 1 example with "PlayTennis=Yes".
The entropy of this subset is:
^ Entropy(D2) = -1/2 log2(l/2) -1/2 log2(l/2) = 1
♦ Subset witii"Temperature=Coor:
There are 2 examples with "Pla3 ^ennis=No" and 2 examples with "Pla3 ^ennis=Yes".
The entropy of this subset is:
^ Entropy(DS) = -2/4 log2(2/4) - 2/4 log2(2/4) = 1
Calculate the weighted average entropy of the subsets:
Weighted Average Entropy = (4/10 * Entropy(Dl)) + (2/10* Entropy(D2)) + (4/10 ♦ Entropy(D3]]
= (4/10 * 0.8113) + (2/10 * 1) + (4/10 * 1)
= 0.9113
Calculate the information gain o f the attribute "Temperature":
Information Gain(Temperature) = Entropy(D) - Weighted Average Entropy
= 0.9709-0.9113
= 0.0596
4. Choose the attribute with the highest information gain as the root of the decision tree:
■f Since "Outlook" has the highest information gain of0.4200, we choose it as the root of the decision
tree.
-f The decision tree would look like this:
♦ If Outlook = Sunny, then:
If Temperature = Hot, then PlayTennis = No
If Temperature = Mild, then PlayTennis = No
If Temperature = Cool, then Plaj^ennis = Yes
♦ If Oudook = Overcast, then PlayTennis = Yes
♦ If Outiook = Rainy, then:
If Temperature = Hot, then PlayTennis = Yes
If Temperature = Mild, then Pla}^ennis = Yes
If Temperature = Cool, then PlayTennis = No

2. Gain Ratio

The Gain Ratio m easures the effectiveness o f a particular attribute in classifying the data. It takes into
account both the information gain and the sp lit information, providing a more balanced assessm ent
of the attributes usefulness in the decision-m aking processof building a^iecision-tFeerThe^Gain Ratio
is a metric used in decision tree algorithms, particularly in the C4.5 algorithm.
By using the Gain Ratio, decision tree algorithm s can make more informed decisions about which
attributes to use for splitting, leading to m ore accurate and generalizable models.
3 .5 4 ^ '.

Gain(A)
The Gain Ratio is defined as: J " Splitlnfo^ (D)
W here:
• GainCA) is the information gain of attribute A on d ataset D.
• Splitlnfo^(D) is the spUt inform ation of attribute A on d a ta se t D.
V |D. I
fl DJl
The Splitlnfo^(D) is calculated as: SplitInfoA(D) = - ^ - j ^ x l o g 2
1D|
W here:
. |Djl/lDl acts as the weight o f the jth partition.
• V is the number of discrete values in attribute A.

Example“
Understanding Gain Ratio
_______ ' -------------
Suppose we have a dataset of students with two attributes: "Study Hours" and "Pass/Fail". We want to build a
decision tree to predict whether a student will pass or fail based on the number of study hours.
1. Information Gain:
^ Information Gain measures how much the "Study Hours" attribute helps in predicting the "Pass/
Fail" outcome.
Higher Information Gain indicates that "Study Hours" is more useful for making decisions in the
decision tree.
2. Split Information:
Split Information measures the uncertainty caused by different splits on the "Study Hours
attribute.
^ It considers the number of study hour ranges and how many students fall into each range.

3. GainRatio:
The GainRatio is the ratio of Information Gain to Split Information.
^ It balances the usehilness of "Study Hours" with the potential uncertainty introduced by its
different splits.
For example, if splitting the data based on "Study Hours" leads to a high Information Gain but also mtroduces
a lot of uncertainty due to many different study hour ranges, the GainRatio will help in evaluating whether the
split is worth it
In this way the GainRatio guides the decision tree algorithm in choosing the most effective attributes for
making decisions, leading to more accurate predictions.______________ ________ _____________________

Example Gain Ratio

Let's go through die calculation of GainRatio for the "Study Hours" attribute step by step with a detailed
explanation for each step.
Step 1: The Dataset
We have a dataset of students with two attributes: "Study Hours" and "Pass/Fail. The goal is to
build a decision tree to predict whetiier a student will pass or fail based on the number of study
hours. _______ _____________________ ____ _____________ ______
 & ^ p e rm iie d Learniii

Here's our dataset:

• "Salient ■^ d y Hours Pass/Fail

Alice 2 Fail
Bob 1 Fail
Carol 3 Pass
Dave 2 Fail
Eve 4 Pass
Step 2: Calculate Entropy (Entropy(D))
Entropy measures the uncertainty or impurity in the dataset D. The formula for entropy is as
follows:
Entropy(D)=-p^^ log^ ) - p^^, -log, {p^^)
Where:
• Pp^ is the proportion of students who passed.
• Pp^,| is the proportion of students who failed.
In our dataset
2
• Ppass = g (2 students passed out of 5)

3
• Ppaii - g (3 students failed out of 5)

Calculate Entropy(D):
'2^ '2 ' '3^ '3 '
Entropy(D) = - •log2 •10g2
.5 , .5 , ,5,
Using base-2 logarithms:
'2^
Entropy(D)=- (-2 .3 2 2 )- (-1.585)
15,
Entropy(D3»0.971
Step 3: Calculate Information Gain (Gain(Study Hours))
Information Gain measures how much the "Study Hours" attribute helps in reducing uncertainty
(Entropy) in predicting "Pass/Fail." The formula for Information Gain is as follows;

|DJ
Gain(Study Hours) = Entropy(D) - ^ . Entropy(D,)
\m )
Where:
• k is the number of possible values of the attribute.
• D|is the subset of data where the attribute has the i'*' value.
In our case, "Study Hours" has four possible values: 1, 2, 3, and 4.
For "Study Hours = 1" (D l):
• Entropy(Dl) = 0 (Since there's only one student, the entropy is 0.)
For "Study Hours = 2" (D2):
» Entropy(D2) = 1______
 For "Study Hours = 3" p 3 ) :
• Entropy[D3) = 0 (Since there's only one student, the entropy is 0.)
For "Study Hours = 4" (D4):
• Entropy(D4)=0 (Since Aere's only one student, the entropy is 0.)
Now, calculate Gain(Study Hours):

Gain(StudyHours)=0.971 - i . 0 + - . l + - . 0 + i .0
5 5 5 5

Gain(StudyHours)= 0 .9 7 1 --
D
Gain(StudyHours) = 0.971-0.4
Gain(StudyHours]=0.571
Step 4: Calculate Split Information (SplitInfo(Study H ours))
Split Information measures the uncertainty introduced by different splits on the "Study Hours
attribute. The formula for Split Information is as follows:

SpIitInfo(StudyHours) = - X t , ^

Where k is the number of possible values of the attribute. In our case, k = 4.

Calculate SplitInfo(Study Hours):
2
Splitlnfo(StudyHours) = - ■logj •log; .log.
•10g2

Splitlnfo(StudyHours) ~ 1.921
Step 5: Calculate GainRatio (GainRatio(Study H ours))
GainRatio is the ratio of Information Gain to Split Information;
, „ ^ Gain(StudyHours)
GainRatloCStudyHours)=j^|.^n,„([Link])
0.571
GainRatio(StudyHours) =
1.921
GainRatio(StudyHours)w0.297
So, the GainRatio for the "Study Hours" attribute is approximately 0.297. This value helps us evaluate the
usefulness of "Study Hours" in making decisions in the decision tree while considering all possible values of
"Study Hours," including "4." Higher GainRatio values indicate that the attribute is more valuable for splitting
the data while considering the potential uncertainty introduced by different splits.______________ .

The Gini index, used in the CART algorithm, is a m easu re o f impurity or uncertainty in a dataset It
is commonly used to evaluate the quality of a particu lar sp lit in a decision tree. The Gini index for a
dataset D is calculated based on the probabilities o f e a ch class in the dataset
Another decision tree algorithm CART (Classification an d Regression Tree) uses the Gini method to
create split points.
Glni(D) = l - X : . P | '
Where p. is the probability th at a tuple in D belongs to d a s s C,.
The Gini Index considers a binary split for each attribute. We can compute a weighted sum of the
impurity of each partition. If a binary split on attribu te A partitions data D into D1 and D2, the Gini
index of D is;

G in i.C D )= !j^ G in iC D .)+ l^ G in i(D ,)

In interacting with a discrete-valued attribute, the splitting attribute is chosen from the subset that
5 aeldsthe lowest gini index for the given value. W hen dealing with continuous-valued characteristics,
the approach is to consider every pair of neighboring values as a potential split point; the splitting
point is determined by selecting the point with the low est gini index.
AGini(A) = Gini(D) - GiniJD)
The splitting attribute is determined by taking the attribute with the lowest Gini index.

GiniIndex
Let's understand a simple example of calculating the Gini Index and selecting the splitting attribute using a
decision tree. We'll start with a basic dataset and explain each step in detail.
Step 1: The Dataset
Imagine we have a dataset of fhiits with two attributes: "Color" and "Class" (whether the fruit is
"Apple" or "Banana"). We want to build a decision tree to classify these fhiits based on their color.
Sample dataset:
C olor ■■■' Class :
Fruiti Red Apple
Fruit2 Yellow Banana
Fruits Red Apple
Fruit4 Yellow Banana
Step 2: Calculate Gini Index (Gini(D))
The Gini Index measures the impurity or uncertainty in the dataset D. The formula for Gini Index
is as follows:

GiniCD) = l-X L (P .)^

Where:
• c is the number of classes.
• p, is the probability of a data point in D belonging to class C^.
In our dataset, there are two classes: "Apple" and "Banana."

• p^pp,^ = —(2 apples out of 4 fruits).

Calculate-Gini(D):

Gini(D) = l -

Simplify:
^ Jsa^ l^ .fflach in e learninjir

1 1^
Gini(D) = l - ---1---
4 4

Gini(D) = l - -

Gini(D] = i

Step 3: Calculate Gini Index for Attribute "Color" (GiniA(D))

Now, we want to calculate the Gini Index for the "Color" attribute, which has two possible values:
"Red" and "Yellow."
For "Color = Red" (Dl):
2
• p^ppi^ = -(2 apples out of red fhiits).

• = ^(0 bananas out o f 2 red fruits)

Calculate Gini(Dl):
(r
Gini(Dl) = l -

Simplify: Gini(Dl) = 1 - (1 + 0)
Gini(Dl) = 0
For "Color = Yellow" (D2):

• = ^(0 apples out of 2 yellow fhiits).

2
• Peanana = bananas out o f 2 yellow fruits).
2
Calculate Gini(D2):
(2 \
Glni(D2) = l - +

Simplify:
Gini(D2) = 1 - (0 + 1)
Gini(D2) = 0
Step 4: Calculate AGini(Color)
Now, calculate the reduction in impurity (AGini) for the "Color" attribute:
f l Di O
AGini(Color)= Gini(D) - Gini(D)
i D| j
Where:
|D||is the size of subset Dj created by the split (example, "Red" and "Yellow" subsets).
|D1 is the size of the original dataset D.
Gini(D^ is the Gini Index for subset D, calculated in Step 3.
 AGini(Color) = ^ - - .0 +
4
Simplify:

AGlni(Color) = " (0 + 0)

AGini(Color) = ^

Step 5: Select the Splitting Attribute

The attribute with the lowest AGini value is chosen as the splitting attribute. In this case, "Color"
has the lowest AGini value of - , indicating that it's the best attribute to split the data.

So, in our decision tree, we would split the data based on the "Color" attribute, specifically into "Red" and
“Yellow" subsets, because it results in the greatest reduction in impurity (Gini Index). This process continues
recursively to build the decision tree.

IDS Algorithm
The IDS algorithm is a classic decision tree algorithm that is used to build a decision tree from a
dataset. T he goal of the algorithm is to create a tree that can predict th e class label of instances based
on the attrib u te values. The algorithm selects the best attribute a t each node of the tree based on
inform ation gain, which measures the effectiveness of an attribute in classifying the training data.
The IDS algorithm works by recursively partitioning the dataset into subsets based on the values
of the attrib u tes. At each node of the tree, the algorithm selects the attribute that provides the m ost
inform ation gain, which is a measure o f how much the attribute red uces the uncertainty about the
class labels. T he attribute with the highest information gain is chosen as the splitting attribute for the
node.
The inform ation gain is calculated using the entropy of the d ataset b efo re and after the split. Entropy
is a m easure o f the impurity of a set of examples, where a set is consid ered pure if all examples belong
to the sam e class.

Algorithm ID3 Algorithm

1. Start with the original dataset as the root node of the tree.
• The original dataset contains all the instances and their corresponding attribute values.
2. If all instances in the current node belong to the same class, then create a leaf node for that class and
stop.
• Check if all instances in the current node have the same class label.
• If true, create a leaf node with the corresponding class label and stop the splitting process for this
branch.
3. If the attribute set is empty, then create a leaf node for the most common class and stop.
• If there are no more attributes to consider for splitting, create a leaf node wnth the class label that
is most common among the instances in the current node and stop.
4. Otherwise, calculate the information gain for each attribute in the attribute set
• For each attribute remaining in the attribute set, calculate the information gain to determine the
attribute that provides the most useful splitting._________________________________________
 5. Select the attribute with the highest information gain as the splitting attribute for the current node.
• Choose the attribute that yields the highest information gain as the attribute for splitting the
current node into child nodes.
6. Split the current node into child nodes based on the values of the selected attribute.
• Create child nodes for each possible value of the selected attribute, dividing the instances based
on tiieir attribute values.
7. Recur on each child node, using the subset of instances corresponding to that attribute value.
• For each child node created in the previous step, repeat the process recursively using the subset
of instances that correspond to the attribute value of that child node.
• This recursive process continues until the stopping criteria (steps 2 and 3) are met for each
branch. ____

Example’;'" ID2 Algorithm

Let's consider another example to illustrate the generic algorithm for building a decision tree. Suppose we
have a dataset of customers who have purchased products from an online store, with the following attributes:
"Age," "Gender," "Location," and "Product Category" (where "Product Category" is the class label with values
"Electronics" and "Clothing").
1. Start with the original dataset as the root node of the tree.
• The original dataset contains instances of customers and their attributes.
2. If all instances in the current node belong to the same class, then create a leaf node for that class and
stop.
• Check if all instances in the current node have the same class label. If so, create a leaf node with
the corresponding class label and stop.
3. If the attribute set is empty, then create a leaf node for the most common class and stop.
• If there are no more attributes to consider for splitting, create a leaf node with the class label that
is most common among the instances in the current node and stop.
4. Otherwise, calculate the information gain for each attribute in the attribute set
• Calculate the information gain for each attribute ("Age," "Gender," and "Location"] to determine
the attribute that provides the most useful splitting.
5. Select the attribute with the highest information gain as the splitting attribute for the current node.
• Suppose "Product Category" has the highest information gain, so we select "Product Category" as
the splitting attribute for the current node.
6. Split the current node into child nodes based on the values of the selected attribute.
• Create child nodes for each possible value of the selected attribute "Product Category" Example,
"Electronics" and "Clothing").
7. Recur on each child node, using the subset of instances corresponding to that-attribute-value.
• For the child node representing "Product Category = Electronics," repeat the process using the
subset of instances of customers who have purchased electronics.
• For the child node representing "Product Category = Clothing," repeat the process using the
subset of instances of customers who have purchased clothing. _______________________
This process continues recursively, considering the remaining attributes and their values, until the stopping
criteria are met for each branch, resulting in the creation of a decision tree that can classify customers based
on their attributes and the products they have purchased.
This example demonstrates how the generic algorithm for building decision trees can be applied to a dataset
of customers and their purchase history to create a decision tree for classification._______________________

m
1. Simplicity: ID3 is relatively simple to understand and implement, making it accessible for beginners
and useful for educational purposes.
2. Handles Categorical Data: IDS is well-suited for handling categorical attributes and class labels,
making it effective for classification tasks involving non-numeric data.
3. Interpretability: The resulting decision tree is easy to interpret and visualize, allowing users to
understand the decision-making process and the rules used for classification.
4. Feature Selection: 1D3 inherently performs feature selection by choosing the most informative
attributes for splitting, which can help in identifying the most relevant features for classification.

Limitations of n)3 Algorithm

1. Handles only Categorical Data: 1D3 is designed to work with categorical attributes, and it does
not handle continuous or numerical attributes directly. Preprocessing techniques such as binning or
discretization are often required for numerical data.
2. Overfitting: 103 tends to overfit the training data, especially when dealing with noisy data or datasets
with a large number of attributes. This can lead to poor generalization on unseen data.
3. Biased towards Attributes with many Values: 1D3 favors attributes with a large number of distinct
values, which can lead to biased trees and overfitting. This bias can be problematic when dealing with
high-cardinality attributes.
4. Lack of Pruning: ID3 does not include a pruning mechanism to prevent overfitting, which can result in
complex trees that do not generalize well to new data. ___________________________ _____________

C4.5 Algorithm
The C4.5 algorithm is a popular decision tree algorithm developed by Ross Quinlan as an extension
of the earlier IDS algorithm. It addresses some of the limitations of IDS and introduces several
improvements.
C4.5 is a decision tree based algorithm used for constructing decision trees from a dataset Its
primary purpose is to perform classification tasks by creating a decision tree that can be used to
make predictions about the class label of new instances based on their attribute values.
The C4.5 algorithm uses a top-down, greedy approach to construct a decision tree. It employs the
concept of information gain and entropy to determine the best attribute for splitting at each node of
the tree. The algorithm aims to create a tree that maximizes the information gain at each split, leading
to more accurate and efficient classification.
 'M

Kej’ Improvements of C4.5 over ID3

The C4.5 decision tree algorithm, an improvement over IDS, introduces several key enhancements
and strategies to build more accurate decision trees. The key improvements are listed below:
1. Handling Missing Data:
C4.5 handles missing data by simply ignoring it during the calculation of gain ratio. When
building the decision tree, the gain ratio is calculated based only on the records that have
a value for the attribute in question. To classify a record with a missing attribute value, the
algorithm can predict the value for that item based on the known attribute values of other
records.
JSg Example Missing Data
Let's consider a simple example where we have a dataset for predicting whether a person will buy a
product based on their age and income. However, some entries have missing values for the income
attribute.
Suppose we have the following dataset:
Age Income Will Buy
25 30,000 Yes
35 50,000 No
45 Yes
30 40,000 No
In this example, the income value for the third entry is missing. When building the decision tree using
C4.S, the gain ratio for splitting based on income would be calculated based only on the available
records (1", 2"“* and 4 “'’ entries). If a new record with a missing income value needs to be classified, C4.5
can predict the income value for that item based on the known attribute values of other records in the
dataset __________________ _
2. Continuous Data:
C4.5 addresses the handling of continuous data by dividing the data into ranges based
on the attribute values found in the training sample. This allows the algorithm to
effectively work with continuous attributes, unlike IDS, which primarily handles discrete
attributes.
Example Continuous Data
Suppose we have a dataset of housing prices with two attributes: square footage and price. The square
footage attribute is continuous, and we want to build a decision tree to predict the price of a house
based on its square footage.

'< Square Footage Price

1000 100
1500 150
2000 200
2500 250
3000 300
 To handle the continuous square footage attribute, C4.5 divides the data into ranges based on the
attribute values found in the training sample. For example, we could divide the square footage into the
following ranges:
• 0-1500
. 1500-2500
• 2500-3500
Then, C4.5 can use these ranges to build a decision tree that predicts the price of a house based on its
square footage. The resulting decision tree might look like this:
• If square footage <= 1500, then price = 100
• If square footage > 1500 and square footage <= 2500, then price = 150
• If square footage > 2500, then price = 250
This illustrates how C4.5 handles continuous data by dividing it into ranges based on the attribute
values found in the training sample. This allows the algorithm to effectively work with continuous
attributes, unlike ID3, which primarily handles discrete attributes._______________________________
3. Pruning:
C4.5 introduces pruning strategies to reduce overfitting and improve the generalization of the
decision tree. Two primary pruning strategies are proposed:
• Subtree Replacement: This strategy involves replacing a subtree with a leaf node if the
replacement results in an error rate close to that of the original tree. The replacement
process works from the bottom of the tree up to the root.
• Subtree Raising: In this strategy, a subtree is replaced by its most used subtree, raising
it from its current location to a higher node in the tree. The increase in error rate for this
replacement must be determined.

u=n Pruning
Suppose we have a dataset for predicting whether a customer will purchase a product based on their
age and income level. We use C4.5 to build a decision tree, and the resulting tree is as follows:
If age < 30 and income = high, then purchase
If age < 30 and income = low, then no purchase
If age >= 30 and income = high, then purchase
If age >= 30 and income = low, then no purchaseNow, let's say we have a validation dataset
that we use to evaluate the performance of the decision tree. We find that the decision tree has a high
accuracy on the training dataset, but a lower accuracy on the validation dataset. This suggests that the
decision tree is overfitting to the training dataset
To address this issue, we can prune the decision tree by removing subtrees that do not improve its
performance on the validation dataset. For example, we might consider removing the subtree for the
condition "age < 30," since it only applies to a small subset of the data and may be overfitting to noise
in the training dataset
After pruning the decision tree, we might end up with the following simplified tree:
I f income = high, then purchase
If income = low, then no purchase _______________________________________ _________
 • •

This pruned decision tree is simpler and more generalizable than the original decision tree, since it is
less likely to overfit to noise in the training dataset By pruning the decision tree, we have improved its
performance on the validation dataset and made it more suitable for predicting whether a customer
will purchase a product based on their age and income level.
4. Rules:
C4.5 allows for classification via decision trees or rules generated from them. It also offers
techniques to simplify complex rules. For example, it can replace the left-hand side of a rule
with a simpler version if all records in the training set are treated identically. Additionally, an
"otherwise" type of rule can be used to indicate what should be done if no other rules apply.

□
HOg EYainpli' Rules
Suppose we have a dataset for predicting whether a customer will purchase a product based on their
age and income level. We use C4.5 to build a decision tree, and the resulting tree is as follows:
If age < 30 and income = high, then purchase
v' If age < 30 and income = low, then no purchase
^ If age >=30 and income = high, then purchase
^ If age >= 30 and income = low, then no purchase
We can use C4.5 to generate rules from this decision tree. For example, the first rule would be:
^ If age < 30 and income = high, then purchase
Similarly, we can generate rules for the other branches of the decision tree. For example, the second
rule would be:
If age < 30 and income = low, then no purchase
We can also use C4.5 to simplify complex rules. For example, suppose we have the following rule:
If age < 30 and income = high and gender = female and education = college, then purchase
This rule is complex and may be overfitting to noise in the training dataset. To simplify this rule, C4.5
can replace the left-hand side of the rule with a simpler version if all records in the training set are
treated identically In this case, we might simplify the rule to:
^ If age < 30 and income = high, then purchase
Finally, we can use an "otherwise" type of rule to indicate what should be done if no other rules apply
For example, we might have the following rule:
'/ If no other rules apply, then no purchase
These rules generated from the decision tree can be used to classify new customers based on their age
and income level. By generating rules from the decision tree, we can simplify the classification process
and make it more interpretable.___________ _______ __________________________________ _____
5. Splitting:
the is s u e of-overfittingjjy taking into account the cardinality of each division
C 4 .5 - a d d r e s s e s

when selecting the best attribute for splitting. It uses the GainRatio instead of Gain for splitting
purposes. The GainRatio compensates for the skewness of the GainRatio value toward sphts
where the size of one subset is close to that of the starting one, ensuring a larger than average
information gain.
 Example .".\ Splitting
Suppose we have a dataset of students and we want to build a decision tree to predict whether a student
will pass o r fail an exam based on two attributes: study hours per week and attendance percentage. The
dataset has the following distribution:
• Pass: 60 students
• Fail: 40 students
We want to decide which attribute to use for the first split in the decision tree. We calculate the
GainRatio for both attributes (study hours and attendance percentage) to determine the best attribute
for splitting.
For the study hours attribute, the dataset can be split into two subsets:
• Subset 1: Study hours <5
• Subset 2: Study hours >=5
The information gain for this split is calculated using entropy measures. Similarly, we calculate the
information gain for the attendance percentage attribute by splitting the dataset based on different
attendance percentage thresholds.
After calculating the information gain for both attributes, we compute the GainRatio for each attribute.
The GainRatio takes into account the cardinality of each division and compensates for the skewness of
the GainRatio value toward certain splits.
Suppose we find that the GainRatio for the study hours attribute is 0.6, and the GainRatio for the
attendance percentage attribute is 0.8. Based on these values, C4.5 would select the attribute with the
highest GainRatio (in this case, the attendance percentage attribute) for the first split in the decision
tree.
By using GainRatio, C4.S ensures that the attribute selected for splitting takes into account the size of
the subsets and compensates for any skevraess in the information gain values, thereby addressing the
issue of overfitting and improving the accuracy of the decision tree.
This example demonstrates how C4.5 uses GainRatio to make informed decisions about attribute
selection for splitting, ultimately leading to the construction of more effective and generalized decision
trees. __________________________________________________

How C4.5 Algorithm Works?

Algorithm Generic Algorithm; C4.5 Algorithm

The generic algorithm for building a decision tree using the C4.5 algorithm can be summarized as follows:
1. Start with the original dataset as the root node of the tree.
• The original dataset contains instances of data and their attributes.
2. If all instances in tiie current node belong to the same class, then create a leaf node for that class and
stop.
• Check if all instances in the current node have the same class label. If so, create a leaf node with
the corresponding class label and stop.
3. If the attribute set is empty, then create a leaf node for tiie most common class and stop.
• If there are no more attributes to consider for splitting, create a leaf node with tiie class label that
is most common among the instances in the current node and stop.______________________ _
 4. Otherwise, calculate the information gain ratio for each attribute in the attnbute set.
. Calculate the information gain ratio for each attribute to determine the attribute that provides
the most useful splitting. Information gain ratio takes into account the intrinsic information of an
attribute.
5. Select the attribute with the highest information gain ratio as the splitting attribute for the current
node.
• Choose the attribute that maximizes the information gain ratio as the splitting attribute for the
current node.
6. Split the current node into child nodes based on the values of the selected attribute.
• Create child nodes for each possible value of the selected attnbute.
7. Recur on each child node, using the subset of instances corresponding to that attribute value.
. For each child node, repeat the process using the subset of instances correspondmg to the
attribute value of that node.
8. Pruning (optional): After the tree is constructed, pruning techniques can be applied to reduce
overfitting and improve generalization.
The C4.5 algorithm improves upon 1D3 by addressing some of its limitations, such as handling continuous
attributes, handling missing attribute values, and reducing overfitting through pruning. Additionally, C4.5
introduces the ability to handle both categorical and continuous attributes, making it more versatile for real-
world datasets. ________ __________ ______________ _____________

g-.-g Example C4.5 Algorithm

-------°_________________________________ _________________ _^---------------------------- -----
Suppose we have a dataset of weather conditions and corresponding activities, and we want to build a decision
tree to predict the activity based on the weather attributes: outlook, temperature, humidity, and windy.
1. Start with the original dataset as the root node of the tree:
• The original dataset contains instances of weather data and their corresponding activities.
2. Check if all instances in the current node belong to the same class:
• If all instances in the current node have the same activity (e.g., "play" or "don't play"), create a leaf
node with the corresponding activity label and stop.
3. If the attribute set is not empty, calculate the information gain ratio for each attribute:
. Calculate the information gain ratio for each attribute (outlook, temperature, humidity, windy) to
determine the attribute that provides the most useful splitting.
4. Select the attribute with the highest information gain ratio as the splitting attribute:
. Suppose we find that "outlook" has the highest information gain ratio among the attributes, so we
select "outlook" as the splitting attribute for the current node.
5. Split the current node into child nodes based on the values of the selected attribute:
. Create child nodes for each possible value of the "outlook" attribute (e.g., sunny, overcast, rainy).
6. Recur on each child node, using the subset of instances corresponding to that attribute value:
. For each child node (sunny, overcast, rainy), repeat the process using the subset of instances
________corresponding to the "outlook" attribute value of that node._______________ ______________
 S u p erv ised Learning^

7. Continue the process o f calculating information gain ratio, selecting splitting attributes, and creating
child nodes until the tree is fully constructed.
8. Pruning (optional):
•_______ After the tree is constructed, pruning techniques can be applied to reduce overfitting and improve
________ generalization.____________

m
1. Versatility: The C4.5 algorithm can handle both categorical and continuous attributes, making it
suitable for a vvide range o f datasets.
2. HandlingMissing Data: The C4.5 algorithm can handle missing data by ignoring missing values during
attribute selection and prediction.
3. Reduced Overfitting: The C4.5 algorithm uses pruning techniques to reduce overfitting and improve
generalization.
4. Easy to Interpret: The decision tree generated by the C4.5 algorithm is easy to interpret and can
provide Insights into the underlying data._____________________________________________

1^ 1 C4.5 Algorithin ...

1. Computationally Expensive: The C4.5 algorithm can be computationally expensive, especially for
large datasets virith many attributes.
2. Sensitive to Noisy Data: The C4.5 algorithm is sensitive to noisy data, which can lead to overfitting and
inaccurate predictions.
3. Biased towards Attributes with ManyValues: The C4.5 algorithm tends to favor attributes with many
values, which can lead to overfitting and inaccurate predictions.
4. Limited to Binary Classification: The C4.5 algorithm is limited to binary classification problems and
cannot handle multi-class classification problems without modifications. _______________________

CART( C lassification and Regression Tree)

The CART (Classification and Regression Trees) algorithm is a popular decision tree algorithm used
for both classification and regression tasks. CART is a recursive partitioning algorithm that recursively
splits the dataset into subsets based on the values of input variables. It constructs binary trees where
each non-leaf node represents a decision based on a feature, and each leaf node represents the output
(class label or numerical value).
The purpose of the CART algorithm is to create a predictive model that can be used for both
classification and regression tasks. It aims to partition the input space into regions that are as
homogeneous as possible vdth respect to the target variable.
The algorithm uses a top-down greedy approach to recursively split the dataset based on the feature
that provides the best split, as determined by a criterion such as the Gini impurity for classification
tasks or the reduction in variance for regression tasks. The process continues until a stopping criterion
is met, such as reaching a maximum tree depth or having nodes vdth a minimum number of samples.
How it works?

1. Select the B est Split: The algorithm evaluates all possible splits for each feature and selects
the one that maximizes the homogeneity of the resulting subsets.
^ For Classification: Calculate the Gini impurity for each feature and select the split that
maximizes the homogeneity of the resulting subsets.

Gini = l-S ;L i(p i)^

where pi is the probability of an object being classified to a particular class.
2. Recursively Partition the Data: The selected feature and split point are used to partition the
data into two subsets. This process is repeated for each subset until a stopping criterion is met.
3. Create the T ree: The result is a binary tree where each non-leaf node represents a decision
based on a feature, and each leaf node represents the predicted output
4. Handling Categorical and Numerical Features:
For categorical features, the algorithm considers all possible ways to split the data based
on the categories.
^ For numerical features, the algorithm considers all possible split points to find the best
threshold for dividing the data.
5 Stopping Criteria: The algorithm stops growing the tree when a stopping criterion is met,
such as reaching a maximum tree depth, having nodes with a minimum number of samples, or
when no further split will improve the performance.
6. Pruning (optional): After the tree is fully grown, a pruning strategy can be applied to reduce
the size of the tree and prevent overfitting. Pruning involves removing nodes that do not
significantly improve the predictive accuracy of the tree.
7. Prediction: To make predictions for new data points, the algorithm traverses the tree based on
the feature values of the data point until it reaches a leaf node, and then outputs the predicted
class label. ________________ ____

Example CART
Suppose we have a dataset of weather conditions and corresponding activities, and we want to build a decision

tree to
TemperatiuB * Humidity Activity <.
-■ “ Outlook
high false no
sunny hot
hot high true no
sunny
high false yes
overcast hot
high false yes
rainy mild
normal false yes
rainy cool
normal true no
rainy cool
normal true yes
overcast cool
high false no
sunny mild
normal false yes
sunny cool
normal 1 false yes
rainy mild
 S u p erv ised iMrnii

sunny mild normal yes

overcast mild high yes
overcast hot normal false yes
rainy mild high tnie
This dataset includes weather conditions such as outlook (sunny, overcast, rainy), temperature (hot, mild,
cool), humidity (high, normal), and windy (true, false), as well as the corresponding activity (play or don't
play). This dataset can be used to build a decision tree to predict the activity based on the weather conditions.
1. Start with the original dataset as the root node of the tree:
• The original dataset contains instances of weather data and their corresponding activities.
2. Check if all instances in the current node belong to the same class:
• If all instances in the current node have the same activity (example, "play" or "don't play"), create
a leaf node with the corresponding activity label and stop.
3. If the attribute set is not empty, calculate the Gini impurity for each attribute:
• Calculate the Gini impurity for each attribute (outlook, temperature, humidity, windy) to
determine the attribute that provides the most useful splitting.
4. Select the attribute with the lowest Gini impurity as the splitting attribute:
• Suppose we find that "outlook" has the lowest Gini impurity among the attributes, so we select
"outlook” as the splitting attribute for the current node.
5. Split the current node into child nodes based on the values of the selected attribute:
• Create child nodes for each possible value of the "outlook" attribute (example, sunny, overcast,
rainy).
6. Recur on each child node, using the subset of instances corresponding to that attribute value:
• For each child node (sunny, overcast, rainy), repeat the process using the subset of instances
corresponding to the "outlook" attribute value of that node.
7. Continue the process of calculating Gini impurity, selecting splitting attributes, and creating child nodes
until the tree is fully constructed.
8. Pruning (optional):
• After the tree is constructed, pruning techniques can be applied to reduce overfitting and improve
generalization.
9. Prediction : The resulting decision tree might look like this:
Outlook
/ I \
Sunny Overcast Rainy
/ \
Humidity Yes Windy
1 / \
High No Yes
/ \
No Yes
3.70\ .- M achin e l e a r n i n g - y ■_. ;

The root node is "Outlook," and it has three branches for "Sunny," "Overcast," and "Rainy."
Under "Sunny," there's a decision based on "Humidity."
Under "Overcast," the decision is straightforward, resulting in a "Yes" prediction.
Under "Rainy," there's a decision based on "Windy."___________________________

j Advantages of CART_________________ ___________________________ ________ ____________

1. Simple to Understand and Im plem ent: CART produces binary decision trees that are easy to interpret
and understand, making it accessible to non-experts.
2. Handles Both Numerical and Categorical Data: CART can handle both numerical and categorical
input variables, providing flexibility in the types of data it can work with.
3. Automatic Variable Selection: CART automatically selects the most important variables and their
interactions, reducing the need for manual feature selection.
4. Non-parametric: CART does not make assumptions about the distribution of the data, making it
suitable for a wide range of data types and distributions.
5. Handles Missing Values: CART can handle missing values in the input data without the need for
imputation.

Limitations<of CART
1. Tendency to Overfit: CART decision trees can grow very large and complex, leading to overfitting on
the training data. Pruning techniques are often required to address this issue.
2. Sensitive to Small Variations in D ata: Small changes in the input data can lead to significantly different
tree structures, making the model less robust
3. Binary Splits Only: CART creates binary trees, which may not capture more complex relationships
present in the data that require multiway splits.
4. Not Suitable for Unbalanced D ata: CART may produce biased trees when dealing with unbalanced
datasets, where one class is much more prevalent than the others.
5. Greedy Algorithm: fcART uses a greedy algorithm to select the best split at each node, which may not
always lead to the globally optimal tree structure.________________________ _______________ ____

Example A Python C ^ e to D en ion^ ate Classifi^tion Tasks using CART

Create a CSV file with the below data and save the file as weather_data.csv.

OutlookjTemperature,Humidity, Windy, Activity

sunny, hot,high,false, no
sunny,hot,high,true,no
overcast,hot,high,false,yes
rainy, mild, high, false, yes
rainy, cool, normal,false,yes
rainy, cool, normal, true,no
overcast, cool, normal, true, yes
sunny, mild,high,false, no
sunny, cool, normal,false,yes_________ ______ ~______________
 v.' f ;' ' Supervised L e a r n i n g ^ 3.7T )
-'-.'■■T''- ........
rainyjinild, normal,false,yes
sunny,mild, normal,true,yes
overcast, mild, high, true,yes
overcast, hot, normal,false, yes
rainy,mild,high,true,no

import pandas as pd
from [Link] import DecisionTreeClassifier
from [Link] import OneHotEncoder

# Load the data from the CSV file

data = pd. read_csv('weather_data.CSV')

# Perform one-hot encoding for categorical features

data_encoded = pd.get_dummies(data, columns=['Outlook', 'Temperature', 'Humidity',
'Windy'])

# Define features and target variable

X = data_encoded.drop('Activity', axis=l)
y = data_encoded['Activity']

# Instantiate a Decision Tree classifier

clf = DecisionTreeClassifierO
[Link](X, y)

# Take user input for Outlook, Temperature, Humidity, and Windy

outlook = inputC'Enter Outlook (sunny/overcast/rainy): ")
temperature = input("Enter Temperature (hot/mild/cool): ")
humidity = input("Enter Humidity (high/normal): ")
windy = input("Enter Windy (True/False): ")
# Create a DataFrame with user input for prediction
user_input = [Link]( [ [outlook, temperature, humidity, windy, 'Play']],
columns=['Outlook', 'Temperature', 'Humidity', 'Windy', 'Activity'])
user_input_encoded = pd.get_dummies(user_input)

# Reindex user input to match the encoded data columns

user_input_encoded = user_input_encoded.reindex(columns=data_encoded.columns, fill_
value=0)

# Predict the Activity based on user input

prediction = clf .predict(user_input_encoded.drop('Activity', axis=l))
print("Predicted Activity:", prediction[0])
Enter Outlook (sunny/overcast/rainy): sunny
Enter Temperature (hot/mild/cool): hot
Enter Humidity (high/normal): high
Enter Windy (True/False); True
Predicted Activity: no

Enter Outlook (sunny/overcast/rainy): rainy

Enter Temperature (hot/mild/cool): mild
Enter Humidity (high/normal): normal
Enter Windy (True/False): False
Predicted Activity: yes_____________________
Explanation*- ____________________ ______________ ________________________ _______
1. Data Loading and Preprocessing: The code starts by loading weather data froma CSVfile using Pandas
[Link] function. It then performs one-hot encoding on categorical features ('Outlook', 'Temperature',
'Humidity', 'Windy') using get_dummies function to convert categorical variables into numerical format
for machine learning.
2. Defining Features and Target Variable: The features (X) are defined as all columns except the target
variable 'Activity', which is dropped using drop function. The target variable (y) is set as the Activity
column, which represents whether to 'Play' based on weather conditions.
3. Training a Decision Tree Classifier: A Decision Tree classifier is instantiated using
DecisionTreeClassifier from scikit-leam. The classifier is then trained on the encoded features (X) and
target variable (y) using the fit method.
4. User Input and Prediction: The code prompts the user to input values for 'Outlook', 'Temperature',
'Humidity', and 'Windy' using the input function. User input is stored in a DataFrame and one-hot
encoded similar to the training data. The user input DataFrame is reindexed to match the columns of
the encoded training data. Finally, the trained classifier is used to predict the 'Activity' based on the user
input, and the predicted activity is printed to the console. ___________ ___________ ________

A p p lication s o f D e c is io n Based A lgorithm s in M L

Decision-based algorithms are widely used in machine learning due to their interpretability, flexibility,
and effectiveness in various applications. Some common applications of decision-based algorithms in
machine learning are;
1. Credit Scoring:
Decision Trees and ensemble methods like Random Forests are widely used in credit scoring to
assess the creditworthiness of individuals applying for loans or credit cards. These algorithms
analyze various factors such as income, credit history, and debt levels to predict the likelihood
of default.
2. Fraud Detection:
Decision Trees are employed in fraud detection systems to identify fraudulent transactions
or activities. By analyzing patterns in transaction data, decision-based algorithms can flag
suspicious behavior and reduce financial losses for businesses.
 Supervised L e a r n in g ^ 3.73

3. Healthcare Diagnostics:
Decision Trees are utilized in healthcare for diagnostics and disease prediction. These
algorithms can analyze patient data, sjmiptoms, and test results to assist in diagnosing medical
conditions, recommending treatments, and predicting patient outcomes.
4. Predictive Maintenance:
Decision-based algorithms are used in predictive maintenance applications to anticipate
equipment failures and schedule maintenance activities proactively. By analyzing sensor data
and historical maintenance records, these algorithms can predict when machinery is likely to
malfunction.
5. Marketing Campaign Optimization:
Decision Trees are applied in marketing to optimize campaign strategies and target specific
customer segments effectively By analyzing customer demographics, behavior, and responses
to past campaigns, decision-based algorithms can help businesses tailor marketing efforts for
better engagement and conversion rates.
6. E-commerce Product Recommendations:
Decision-based algorithms power recommendation systems in e-commerce platforms to
suggest products to users based on their browsing history, purchase behavior, and preferences.
These algorithms enhance the user experience and increase sales by offering personalized
product recommendations.
7. Chum Prediction:
Decision Trees are used in churn prediction models to forecast customer attrition or churn.
By analyzing customer interactions, usage patterns, and feedback, decision-based algorithms
can identify customers at risk of leaving a service or subscription, allowing businesses to take
proactive retention measures.
8. Energy Consumption Forecasting:
Decision-based algorithms are employed in energy consumption forecasting to predict
electricity demand and optimize energy distribution. These algorithms anal3rze historical
consumption data, weather patterns, and other factors to forecast energy usage and improve
resource planning.

Advantages and Disadvantages of Decision Tree Based Algorithms

I Advantages of Decision Tree Based Algorithms

O Decision Trees are easy to interpret and understand, making them valuable for explaining the reasoning
behind predictions to stakeholders and domain experts.
O Decision-based algorithms can provide insights into feature importance, helping in feature selection
and understanding theimpacL of variables^on the model^predictions.
O Decision Trees can capture non-linear relationships between features and the target variable, making
them suitable for complex datasets with non-linear patterns.______
3.74 V :^;Madiine L earning

0 Decision Trees can handle missing values in the data without the need for imputation techniques,
simplifying the preprocessing steps.
3 Decision-based algorithms are robust to outliers in the data and can handle noisy data witiiout
significantly impacting model performance._______________________________________

Disadvantages of Decision Tree Based^gor|tJhu^.

Decision Trees are prone to overfitting, especially when the tree depth is not controlled or when the
model is too complex. This can lead to poor generalization on unseen data.
Decision Trees have high variance, meaning they can be sensitive to small changes in the training data,
leading to instability in the model’s predictions.
Decision Trees can exhibit bias towards features with more levels or categories, potentially affecting
the model's performance and predictive accuracy.
Decision-based algorithms may struggle with imbalanced datasets, where one class significantly
outnumbers the others, leading to biased predictions towards the majority class.
Decision Trees create piecewise constant predictions, which may not capture subtle variations in the
data and result in less smooth decision boundaries. *

1. What is supervised learning?

2. Mention two main types of supervised learning?
3. What is classification? Give an example.
4. Give two real time examples of Classification tasks.
5. What is Discrete Output Variable? Give an example.
6. What is Confusion Matrix ?
7. What is Regression? Give an example.
8. Give two real time examples of Regression tasks.
9. Mention the Types of Machine Learning Classification Algorithms?
10. Mention the Types of Machine Learning regression Algorithms?
11. What is Bayes'Theorem?
12. What is K-Nearest Neighbors (K-NN) Algorithm?
13. Why do we need K-NN Algorithm?
14. How to select the value of K in the K-NN Algorithm?
15. What are Linear Models?
16. Write the general form of a linear model.
17. What is Logistic Regression?
18. What is Linear Regression?
19. What is Naive Bayes Classifier ?
20. What are Decision Tree Based Algorithms in Classification? What is the use of it?
 Supervised Learning y 3 .7 5

21. What is Attribute Selection Measure (ASM) ?

22. What is an Information Gain?
23. What is an entropy? Give an example.
24. How to calculate an entropy?
25. What is Gain Ratio?
26. What is Gini Index?
27. What do you mean by Missing Data in classification? Give an example.
28. What do you mean by Continuous Data in classification? Give an example.
29. What is Pruning? Why it is required?
30. What is Splitting? Why it is required?
31. What is CART[ Classification and Regression Tree)?
32. What is Decision Tree Algorithm?

Five M arks Questions

1. Explain the Key Characteristics of a Classification in Supervised Learning.

2. Explain the Importance and Benefits of Classification in Machine Learning.
3. Explain the Key Characteristics of a Regression in Supervised Learning.
4. Explain the Importance and Benefits of Regression in Machine Learning.
5. Explain the Performance Evaluation Metrics in Classification.
6. Explain the Differences between Regression and Classification.
7. How K-Nearest Neighbors (K-NN) Works ?
8. Write the Applications of KNN Algorithm.
9. Write the Advantages and Disadvantages of KNN Algorithm.
10. What are Linear Models? Explain the general form of a linear model.
11. Explain the Characteristics of Linear Models.
12. What is Linear Regression? Explain how it works?
13. What is Logistic Regression? Explain how it works?
14. Write the Applications of Linear Models.
15. Mention the Advantages and Disadvantages of Linear Models.
16. How Naive Bayes Classifier works?
17. Explain the types of Naive Bayes Classifiers.
18. Write the Applications of Naive Bayes Classifiers.
19. Write the Advantages and Disadvantages of Naive Bayes Classifiers.
20. Write Decision Tree Algorithm and explain how it works?
21. Explain different Attribute Selection Measures (ASM) used in Classification.
22. What is an Information Gain? Explain how to calculate Information Gain?
23. What is Gain Ratio? How to calculate it?
24. What is Gini Index? How to calculate it?
 r-.

25. Write IDS Algorithm.

26. What are the Advantages and Disadvantages of IDS Algorithm.
27. Explain the Key Improvements of C4.5 over IDS.
28. Write C4.5 Algorithm. How C4.5 Algorithm Works?
29. What are the Advantages and Disadvantages of C4.5 Algorithm.
30. What are the Advantages and Disadvantages of CART Algorithm.
51. Write the Applications of Decision Based Algorithms in ML.
52. Write the Advantages and Disadvantages of Decision Tree Based Algorithms
Eight Marks Questions

1. Explain the tjqjes of Supervised Learning.

2. Explain the types of Classification Algorithms in Machine Learning.
S. Explain the Performance Evaluation Metrics in Classification.
4. Explain the types of Regression Algorithms in Machine Learning.
5. Explain the Performance Evaluation Metrics in Regression.
6. What is K-Nearest Neighbors (K-NN) Algorithm? Mention the Characteristics of K-Nearest
Neighbors (K-NN) Algorithm.
7. How K-Nearest Neighbors (K-NN) Works ? Write its algorithm.
8. How K-Nearest Neighbors (K-NN) Works ? Explain with an example for both classification and
regression tasks.
9. Write a Python Code for Classification Task Using KNN Classifier
10. Write a Python Code for Regression Task Using KNN Regressor.
11. What is Linear Regression? Explain How Linear Regression Works with an example.
12. Explain How to predict a car price Using Linear Regression.
13. Explain How to predict a house price Using Linear Regression.
14. Write a P5nlion Code to demonstrate Linear Regression.
15. What is Logistic Regression? Explain How Binary and Multi-Class Classification is done using
Logistic Regression.
16. Write a Python Code to Demonstrate Binary Classification using Logistic Regression.
17. Write a Python Code to Demonstrate Multi-Class Classification using Logistic Regression.
18. What is Naive Bayes Classifier ? How it works?
19. Write a Python Code for Spam Email Detection using the Naive Bayes classification algorithm.
20. Explain IDS Algorithm with an example.
21. Explain C4.5 Algorithm with an example.
22. What is CART( Classification and Regression Tree)? How it works?
23. Write CART Algorithm with an example.
24. Write a P}:thon Code to Demonstrate Classificatipn Tasks using CART.
 Unit

UNSUPERVISED LEARNING

^ sn sn n D

•■ Introduction
•- Types of Unsupervised Learning
•- Clustering
•- K-Means Clustering Algorithm
•■ Using Clustering for Image Segmentation
•■ Using Clustering for Preprocessing
Using Clustering for Semi-Supervised Learning
- DBSCAN
Other Clustering Algorithms.
Review questions.
 V .

In tr o d u c tio n
In many aspects, unsupervised learning differs greatly from supervised machine learning. This sort of
machine learning does not require supervision. It means that in unsupervised machine learning, we
train the machine with a dataset that hasn’t been labelled or trained, and the machine then predicts
the results without any supervision of the underlying data.
Unsupervised learning is the training of a machine using information that is neither classified nor
labelled and allowing the algorithm to act on that information without guidance. Here the task of the
machine is to group unsorted information according to similarities, patterns, and differences without
any prior training of data
The unsupervised learning algorithm's main goal is to classify or group the unsorted dataset
according to the pattern, similarities, and differences that it can identify in the data. The machines
are given instructions to find hidden pattern in the input dataset, and the findings are then analysed.
Unsupervised learning includes all types of machine learning scenarios where there is no predefined
output or instructor to guide the learning algorithm. In unsupervised learning, the learning algorithm
is just shown the input data and asked to extract knowledge from this data.

Let's use an example to better understand how

<. INPUT RAW
unsupervised learning functions. Imagine that
we have a dataset of photos of dogs and cats (^gorithm)
as our input We don't assign a label to the
corresponding data in our training set
S ’ (^^Processing^
U nlabeled D ata

Example of Unsupervised Learning

In this case, we have used unlabelled input data, which means that neither its category nor any associated
outputs are provided. Now, the machine learning model is being trained using the unlabelled input data. It will
first analyse the raw data to identify any hidden patterns in the data and then will apply suitable algorithms
such as k-means clustering, DBSCAN etc. Once the appropriate algorithm has been applied, the algorithm splits
the data objects into groups according to the similarities and differences between the objects._____________

Why Unsupervised Learning?

f i
The common reasons for using Unsupervised Learning in Machine Learning.
> Unsupervised Machine Learning Methods finds all kind of unknown patterns in data
> Unsupervised Machine Learning Methods find features which can be useful for categorization.
> It is taken place in real time, so all the input data to be analysed and labelled in the presence of
learners.
> It is easier to get unlabelled data from a computer than labelled data, which needs manual
intervention. ______
 U nsupervised U a n » l n B ' V ^ 4 . 3 j

m Types of Unsupervised Learning

Unsupervised learning includes a diverse set of techniques aimed at extracting patterns, structures,
and insights from datasets consisting of input data without labeled data. The most common
unsupervised learning methods are listed below:
1. Clustering: Clustering is the process of grouping a set of objects or data points in such a way
that objects in the same group (called a cluster) are more similar to each other than to those
in other groups. It's often used in exploratory data analysis to find natural groupings, identify
patterns and outliers, and simplify complex data sets.
Common Clustering Algorithms:
• K-Means Clustering: Divides a set of samples into disjoint clusters, each described by
the mean of the samples in the cluster.
• Hierarchical Clustering: Builds a multilevel hierarchy of clusters by creating a cluster
tree.
. DBSCAN (Density-Based Spatial Clustering of Applications with Noise): Defines
clusters as areas of high density, allowing it to find arbitrarily shaped clusters and to be
more robust to outiiers than K-means.
• Gaussian Mixture Models (GMM): Models clusters as a mixture of multiple Gaussian
distributions. Points are probabilistically assigned to clusters, which allows for soft
clustering.
2. Association: Association is a rule-based machine learning method used to discover interesting
relations between variables in large databases. It's often used in market basket analysis, where
it can reveal combinations of products frequently bought together.
Common Association Algorithms:
• Apriori Algorithm: Identifies fi-equent individual items in the database and extends them
to larger and larger item sets as long as those item sets appear sufficiently fi-equentiy in
the database.
. FP-Growth: Used for finding fi-equent item sets in a dataset for association rule learning.
3. Dimensionality Reduction: Dimensionality reduction methods aim to reduce the number of
features in a dataset while preserving as much relevant information as possible.
Common Dimensionality Reduction: Techniques:
• Principal Component Analysis (PCA): A statistical procedure that uses an orthogonal
transformation to convert a set of observations of possibly correlated variables into a set
of values of linearly uncorrelated variables.
• t-Distributed Stochastic Neighbor Embedding (t-SNE): A tool to visualize higji-
dimensional data by reducing it to two or three dimensions while maintaining the relative
distances between points.
• Autoencoders: Neural networks used for learning efficient codings of unlabeled data.
The aim is typically to reduce dimensionality by learning a network that encodes the data
to a lower-dimensional space and then decodes it back to the original space.
4. Anomaly Detection: Anomaly detection is the identification of rare items, events, or
observations which are significantly different and deviate from the majority of the data.
 ^ r p in g ' -■

Common Anomaly Detection Techniques:

• Isolation Forest: Uses tree structures to isolate observations
. One-Class SVM: Specially useful for novelty detection when the dataset has just one class
and we want to detect outliers.
5. Neural Networks : Certain neural network models are used in an unsupervised manner to
learn better representations and features from unlabeled data:
Common Neural Networks Techniques:
. Self-Organizing Maps (SOMs): Neural networks trained using unsupervised learning
to produce a low-dimensional (typically two-dimensional) representation of the input
space of the training samples.
. Generative Adversarial Networks (GANs): An algorithm where two neural networks
contest with each other in a game. Given a training set, this technique learns to generate
new data with the same statistics as the training set

B k I clustering
Clustering is a type of unsupervised learning method. Clustering is also known as grouping is
the process of categorizing a set of objects in a way that objects within the same group or cluster are
more similar to each other than to those in other groups. This is similar to dividing data objects into
subclasses based on their similarities.
Clustering is a unsupervised learning method that involves grouping similar data points together
based on their characteristics. Unlike classification, clustering does not involve predefined groups or
labels, but rather relies on finding similarities between data points to group them into clusters.
There are different definitions for clusters, but they generally involve a set of like elements that are
distinct from elements in other clusters. One common definition is that the distance between points
in a cluster is less than the distance between a point in the cluster and any point outside it
A term closely aligned with clustering is "database segmentation," where similar tuples or records
within a database are grouped togeflier. This segmentation aims to partition or segment the database
into distinct components, providing users with a more generalized perspective of the data. In this
context, there is no explicit differentiation between segmentation and clustering.
Determining how to cluster data is not always straightforward, and there are different methods and
algorithms for clustering.

Definitions of Clustering
L
Clustering is the task of dividing the population or data points into a number of groups such
that data p o i n t s in4he^ame-groups_are more similar to other data points in the same group and
dissimilar to the data points in other groups.
Clustering is basically a collection of objects on the basis of similarity and dissimilarity between
them.
Clustering or Cluster analysis is the method of grouping the entitiesjased^onjimila^
 Unsu

W h a t is C lu s te r in g ?

Clustering is a technique used to group similar data objects together based on their characteristics
or attributes. The goal of clustering is to identify patterns or structures in the data that can help in
understanding the underl)dng relationships and associations between the data objects. Clustering
is an unsupervised learning technique, meaning that it does not require labeled data and can be
used to explore and discover patterns in the data without prior knowledge of the data’s structure.

Example 1 Understanding Clustering

Let's understand the clustering technique with the real-worid example of Mall: When we visit any shopping
mall, we can observe that the things with similar usage are grouped together. Such as the t-shirts are grouped
in one section, and trousers are at other sections, similarly, at vegetable sections, apples, bananas. Mangoes,
etc., are grouped in separate sections, so that we can easily find out the things. The clustering technique also
works in the same way. Other examples of clustering are grouping documents according to the topic________

gSg Example 2 Understanding Clustering

Take an example of fruit datasets. Let’s say we have a

huge collection of image dataset containing three fruits
(i) strawberries, (ii) pears, and (iii) apples.
In the dataset, i.e. rawdata, all the images are mixed up
and use-case is to group similar fruits together, create Algorithm
three groups with each one of them containing one tj^je Raw Data

of fhiit This is exactly what a clustering algorithm will

Output
do.

Important Note
Clustering is somewhere similar to the classification algorithm, but the difference is the type of dataset that
we are using. In classification, we work with the labeled data set, whereas in clustering, we work with the
unlabelled dataset.

Real World Examples of Clustering

Real World Exomples of Clustering

1. Customer Segmentation in Marketing : Companies often use clustering to segment their customer
base into distinct groups based on purchasing behavior, demographics, or other relevant attributes. By
identifying different customer segments, businesses can tailor their marketing strategies and product
offerings to better meet the needs and preferences of each group.
2. Image Segmentation in Medical Imaging : In medical imaging, clustering techniques are used for
image segmentation, which involves partitioning an image into multiple regions or segments based on
similarities in pixel intensity, color, or texture. This is valuable in tasks such as tumor detection, organ
delineation, and tissue classification in medical diagnostics.____________________________________ _

<)r- --
 M achin e Learning ; ' . . ,. *^x 'Ot'^j"''-

3. Anomaly Detection in Network Security : Clustering is employed in network security to detect

anomalies or unusual patterns in network traffic By clustering normal network behavior, any
deviations from the established patterns can be identified as potential security threats, such as
network intrusions or malicious activities.
4. Recommendation Systems in E-commerce: Clustering is used in recommendation systems to group
similar products or customers together based on their attributes or behavior. By identifying patterns
in customer behavior, such as purchase history or product preferences, clustering algorithms can
recommend products to customers that are likely to be of interest to them.

Clustering Use, Case: .Customer Segmentation in Marketing

Background : A retail company wants to improve its marketing strategies by better understanding
its diverse customer base. The company has a large database of customer transaction data including
purchase history, demographic information, and online behavior.
Application of Clustering: The company applies clustering techniques to segment its customer base
into distinct groups based on various attributes such as purchasing behavior, age, location, and product
preferences. By using clustering algorithms, the company can identify natural groupings within the
customer data without predefined categories.
Benefits:
Targeted Marketing : With the identified customer segments, the company can tailor its
marketing campaigns to address the specific needs and preferences of each group. For
example, different segments may respond better to different types of promotions or product
recommendations.
Product Customization : Understanding the distinct preferences of each segment allows the
company to customize its product offerings to better meet the demands of different customer
groups. This can lead to increased customer satisfaction and loyalty,
v' Resource Allocation: By knowing the characteristics of each segment, the company can allocate
resources more effectively by focusing on the segments with the highest potential for sales and
customer engagement.
Customer Retention: The insights gained fi'om customer segmentation can help in developing
targeted retention strategies such as personalized loyalty programs or communication strategies
tailored to the needs of each segment
Conclusion: Through the application of clustering for customer segmentation, the retail company can
gain valuable insights into its customer base, leading to more effective marketing strategies, improved
customer satisfaction, and ultimately, increased sales and profitability.

Importance of Clustering in Unsupervised Learning

Clustering is a fundamental technique in unsupervised learning that serves several important
purposes:
1. Pattern Discovery:
Clustering helps in identifying inherent patterns and structures within data that may not
be apparent initially. By grouping similar data points together, clustering algorithms reveal
underlying relationships and similarities in the dataset.
 ■:.^-i'{'' Unswpervised Learning
m
2. Data Exploration:
Clustering enables exploratory data analysis by organizing data into meaningful clusters,
allowing for a better understanding of the dataset's characteristics and distributions. It helps
in gaining insights into the natural groupings present in the data.
3. Segmentation:
Clustering is widely used for customer segmentation, market segmentation, and image
segmentation, among others. By dividing data into distinct clusters, businesses can tailor
their strategies, products, and services to different customer segments based on common
characteristics.
4. Anomaly Detection:
Clustering can also be used for anomaly detection by identifying data points that do not belong
to any cluster or form a separate cluster. These anomalies may represent outliers, errors, or
unusual patterns in the data that require further investigation.
5. Feature Engineering:
Clustering can help in feature engineering by creating new features based on the cluster
assignments of data points. These engineered features can be valuable inputs for machine
learning models to improve predictive performance.
6. Data Preprocessing:
Clustering can be used as a preprocessing step for supervised learning tasks by grouping similar
data points together and reducing the complexity of the dataset. It can help in improving the
efficiency and effectiveness of subsequent machine learning algorithms.
7. Decision Making:
Clustering results can assist in decision-making processes by providing insights into the
structure of the data and facilitating data-driven decisions. It helps in organizing and
summarizing complex datasets for better decision support.

Applications of Clustering
Clustering, as a key technique in unsupervised learning and it is used in applications across various
domains and industries. Some common applications of clustering include:
1. Customer Segmentation:
Businesses use clustering to segment customers based on their purchasing behavior,
demographics, or preferences. This segmentation helps in targeted marketing, personalized
recommendations, and improving customer satisfaction.
2. Image Segmentation:
In image processing, clustering is used for segmenting images into regions with similar
characteristics such as color, texture, or intensity. This is valuable in medical imaging, object
recognition, and computer vision applications.
3. Anomaly Detection:
Clustering algorithms can identify outliers or anomalies in datasets, which is crucial for fraud
detection, network security, and quality control in manufacturing processes.
 M achim l ^ m i i g .
I
4. Document Clustering:
Text documents can be clustered based on their content to group similar documents together.
This is useful in information retrieval, document organization, and topic modeling.
5. Genomics and Bioinformatics:
Clustering is applied in genomics to group genes with similar expression patterns or in protein
clustering for structural analysis. It helps in understanding genetic relationships and biological
functions.
6. Recommendation Systems:
Clustering techniques are used in recommendation systems to group users or items with
similar preferences. This enables personalized recommendations in e-commerce, streaming
services, and social media platforms.
7. Spatial Data Analysis:
Clustering is utilized in geographical data analysis to identify spatial patterns, cluster locations,
and regional trends. It is valuable in urban planning, resource allocation, and environmental
studies.
8. Market Research:
Clustering assists in market segmentation, where customers are grouped based on their
buying behavior, demographics, or psychographics. This information helps businesses in
product positioning, pricing strategies, and targeted advertising.
9. Healthcare Analytics:
Clustering is used in healthcare for patient segmentation, disease clustering, and medical
image analysis. It aids in personalized medicine, treatment planning, and healthcare resource
optimization.

Clustering Attributes
Clustering attributes refer to the characteristics or features of the data that are used to group similar

Cluistering Attribute Description ^ -x

In geographic-based clustering, In urban planning, clustering based on
Geographic-Based data points are clustered based geographic attributes can be used to identify
Clustering on their geographical location or areas with similar population densities,
spatial features. infrastructure, or land use patterns.
In retail, clustering based on size attributes of
Size-based clustering involves customer purchases can help identify groups of
Size-Based Clustering grouping data points based on customers who prefer large, medium, or small­
their size-related attributes.
sized products.
In weather analysis, clustering based on
Temporal-based clustering temporal attributes can be used to identify
Temporal-Based
involves clustering data based patterns in temperature variations over
Clustering on time-related attributes
different time periods.
 ised Learning

Density-based clustering groups

data points based on their In environmental science, clustering based on
Density-Based Clustering density attributes can be used to identify areas
density or concentration in
given space wath high or low concentrations of pollutants.

Feature-based clustering In marketing, clustering based on customer

involves grouping data points demographic features such as age, income,
Feature-Based Clustering
based on specific features or and buying behavior can help identify distinct
characteristics. customer segments for targeted marketing
campaigns.
Behavior-based clustering In cybersecurity, clustering based on behavior
Behavior-Based involves grouping data points attributes can be used to identify groups of
Clustering based on their behavioral users with similar access patterns or suspicious
patterns or tendencies. behavior.

Attribute-Based Attribute-based clustering In healthcare, clustering based on patient

Clustering involves grouping data points attribute values such as age, gender, and
based on their attribute values. medical histoiy can help identify groups with
similar health conditions or risk factors.
Graph-based clustering involves In social network analysis, clustering based on
grouping data points based graph attributes can be used to identify groups
Graph-Based Clustering
on their relationships or of users with similar social connections or
connections in a graph. interests.
Hybrid clustering involves In customer segmentation for e-commerce,
combining multiple clustering hybrid clustering based on demographic,
Hybrid Clustering attributes to create more behavioral, and purchase history attributes can
complex and accurate clustering help identify distinct customer segments for
models. personalized marketing strategies.

Types of Clustering Methods

The clustering methods are broadly divided into Hard clustering (datapoint belongs to only one
group) and Soft Clustering (data points can belong to another group also). But there are also other
various approaches of Clustering exist Belowr are the main clustering methods used in Machine
learning:
1. Partitioning Clustering
2. Density-Based Clustering
3. Distribution Model-Based Clustering
4. Hierarchical Clustering
5. Fuzzy Clustering

1. Partitioning Clustering

Partitional clustering is a type of clustering algorithm that divides a dataset into non-overiapping
clusters, where each data point belongs to only one cluster. The goal of partitional clustering is to
group similar data points together and separate dissimilar data points into different clusters.
 jMachine Learmng ■ r?“.

In other words, partitional clustering aims to partiUon the data into a s e t of dusters, where each
cluster contains data points that are similar to each other and dissim ilar to data points in other
clusters. The number of clusters is usually determined by the user, and th e algonthm tries to find the
best partition of the data into the specified number of clusters.

It is a type of clustering that divides the data into non-

hierarchical groups. It is also known as the centroid-
based m eth od . The most common example of
partitioning clustering is the K-Means Clustering
algorithm .
In this type, the dataset is divided into a set of k
groups, where K is used to define the number of pre­
defined groups. The cluster center is created in such a
way that the distance between the data points of one
cluster is minimum as compared to another cluster
centroid.

A company may use partitional clustering to group customers into segments based on their purchase history,
such as frequency of purchases, amount spent, or types of products purchased. The algorithm would assign
each customer to the closest cluster based on their purchasing behavior, and each cluster would represent a
segment of customers with similar purchasing behavior.
Once the customers are segmented, the company can tailor their marketing strategies to each segment For
example, they may offer discounts or promotions to customers in a particular segment to encourage them
to make more purchases. They may also use different marketing channels o r messaging for each segment to
better target their marketing efforts.

Types of P artitio n al Clustering Algorithm s:

M inim um Spanning Tree: This algorithm creates a tree-like stru ctu re that connects all the
data points in a dataset while minimizing the total distance b etw een them. The tree is then cut
at a certain level to form clusters.
Squ ared E rro r Clustering: This algorithm aims to minimize th e sum of squared distances
between data points and their assigned cluster centers. It s ta rts by randomly selecting initial
cluster centers and then iteratively updates them until convergence.
K-M eans Clustering: This algorithm is similar to squared e rr o r clustering but uses a different
distance metric. It also starts by randomly selecting initial clu ster centers and iteratively
updates them until convergence. K-means is widely used due to its simplicity and efficiency.
N earest Neighbor Algorithm: This algorithm assigns each d a ta point to the nearest cluster
center based on a distance metric. It starts by randomly selectin g initial cluster centers and
then assigns each data point to the nearest center. The centers a re then updated based on the
assigned data points, and the process is repeated until convergence.
 2. Density-Based Clustering

Density-Based Clustering identify clusters based on the density of d a ta points in a dataset High
density region s contain data points that are densely packed with m an y neighboring points within
a specified radius, indicating cohesive clusters. In contrast, low d ensity regions have fewer nearby
points, potentially representing noise or outliers. Unlike partitioning clustering where the number
of clusters is predefined, density-based methods group dosely packed points and distinguish hlgh-
density clu sters from low-density areas, offering flexibility to capture clusters of varjing shapes and
sizes while effectively handling complex data distributions and outliers. ’ ■

In density-based clustering algorithms like DBSCAN,

clusters are form ed by grouping together data points
that belong to high-density regions while separating
them from low-density regions. The algorithm
identifies c o re points (data points in high-density
areas), b o rd e r points (points on the edge of high-
density reg ion s), and noise points (isolated points in
low-density are a s).

Example
Example: In a retail scenario, a company may use DBSCAN to duster custom ers based on their shopping
behavior. The algorithm can identify clusters of customers who frequently shop together in certain areas of
a store (high-density regions) and separate them from customers *vho shop less frequently or in different
areas (low -density regions). By segmenting customers based on shopping patterns rather tfian predefined
categories, the company can tailor marketing strategies and promotions to each cluster effectively.

Types o f D en sity-B ased Clustering A lgorithm s:

■f DBSCAN : One of the most popular density-based clustering algorithm s is DBSCAN (Density-
Based Spatial Clustering of Applications with Noise). DBSCAN defines clusters as continuous
regions o f high density separated by regions of low density. It w oiics by, identifying core points,
b ord er points, and noise points in the d ataset ■ ^*

In the distribution model-based clustering method, the data is divided based on the probability of
how a d ataset belongs to a particular distribution. - i ^

This ap p roach involves assuming certain

probability distributions, with the Gaussian
Distribution being a common choice.
One p rom inen t example of Distribution
Model-Based Clustering is the Expectation-
Maximization (EM) Clustering algorithm, which
utilizes Gaussian Mixture Models (GMM).
 M ach in e

Example _______________ ________________________________— ' —------ ------ —------

I ^ a y the dataset Includes features such as the amount spent on groceriM. electroiilcs, and cliJthtog by each
customer. By running the EM algorithm with CMM on this data, the algorithm can parobon customers mto
segments based on their spending preferences. The Gaussian components m the model represent different
customer segments with varying purchasingbehaviors, such as frequentbuyersofelectromcs, regular grocsry

shoppers, or customers with diverse spending across categories.

Through this clustering approach, retailers can gain Insights into customer segments witt similar buying
patterns, enabling targeted marketing strategies, personalized recommendations, and tailored promo ons
for different customer groups. ________ ____________________ _______________________ ________ _

Hierarchical clustering can be used as an

alternative for the partitioned clustering as
there is no requirement of pre-specifying the
number of clusters to be created.
In this technique, the dataset is divided into
clusters to create a tree-like structure, which
is also called a dendrogram. The algorithm
starts by considering each data point as a
separate cluster and then iteratively merges
the closest pairs of clusters based on their
similarity. This process continues until all
data points belong to a single cluster at the
top level of the hierarchy

JS g E x a m p le

Consider a library with a diverse collection of books covering various subjects such as science histoiy,
literature, and a rt Initially, each book is treated as a separate entity, representing a distinct cluster. However,
as the clustering process progresses, books with similar subject m atter or content a r e grouped together to
form dusters representing specific topics or genres. This process continues, with clustei^ of books being
further grouped into broader categories, such as "science and technology," "humanities," or fiction.
At each level of the hierarchy, the clustering process identifies similarities betiA^een books and organizes tiiem
into meaningful groups based on their content This hierarchical structure allows library visitors to navigate
t h e c o l l e c t i o n b a s e d o n t i i e i r i n t e r e s t s ,s t a r t i n g f r o m s p e c i f i c b o o k s a n d g r a d u a l l y m o v i n g t o b r o a d e r c a t e g o n e s

and topics. It also enables librarians to manage and organize the collection more effectivej^
“ ----------------------- -------------------
T ypes o f H ierarchical Clustering Algorithm s:
-h Agglomerative: This bottdm-up approach starts with each data point as an individual cluster
and then merges the d o sestp air of clusters until all points are merged into a single cluster.
^ Divisive: This top-down approach begins with all data points in a single cluster, which is then
split recursively into smaller clusters.
 ' - ' Unsoperviiwl learnir

In fuzzy clustering, data objects are allowed to belong to multiple clusters simultaneously, with each
object having membership coefficients indicating the degree to which it belongs to each cluster.
Unlike traditional hard clustering methods where data points are assigned to a single cluster, fuzzy
clustering assigns membership values that represent the likelihood of a data point belonging to
different clusters. The Fuzzy C-means algorithm, also known as the Fuzzy k-means algorithm, is a
popular example of fuzzy clustering.

Similarity and Distance Measures

In the context of data clustering, similarity and distance measures serve as the building blocks for
organizing a dataset into meaningful groups. These measures are like special tools that help us
understand how data points relate to each other.
Similarity and distance measures are used to quantify the similarity or dissimilarity between two
data points.
1. Sim ilarity Measures are used to identify how similar two data points are to each other. It's
like comparing two fhiits to see if they are similar in color, shape, or taste. A good similarity
measure should have the following characteristics:
• If two data points are identical, their similarity should be 1.
• If two data points are completely dissimilar, their similarity should be 0.
• If two data points are somewhat similar, their similarity should be between 0 and 1.
2 . D istan ce Measures are used to quantify the dissimilarity between two data points. It's like
measuring the physical distance between two objects. A lower distance score indicates a
greater closeness or less dissimilarity. A good distance measure should have the following
characterisrtcs:
• If two data points are identical, their distance should be 0.
• If two data points are completely dissimilar, their distance should be high.
• If two data points are somewhat dissimilar, their distance should be between 0 and high.

K K H K-Means Clustering Algorithm

K-means clustering is a widely used partitional clustering algorithm designed to partition a given
d ataset into K clusters, with K being a param eter specified by the user.
To begin the K-means algorithm, K points are randomly selected from the dataset as the initial
centroids o f the clusters. Each data point is then assigned to the nearest centroid, and the centroid
of each cluster is updated as the mean of all the data points assigned to that cluster. This process of
assigning data points to clusters and updating the centroids is repeated until convergence, which is
achieved when the centroids no longer change or the change is below a certain threshold.
It is im portant to note that the K-means algorithm is sensitive to the initial selection of centroids,
and different initializations can lead to different results. To mitigate this issue, it is common to run
the algorithm multiple times with different initializations and choose the best result based on some
criterion, such as the sum of squared errors.
4.14 ^ S ^ ch in e Laming
■;

IZI Definition : K-Means Clustering Algorithm

It is an iterative algorithm that divides the unlabeled dataset into k different dusters in such a way that each
dataset belongs only one group that has similar properties.
The algorithm takes the unlabeled dataset as input, divides the dataset into k-number of clusters, and repeats
the process until it does not find the best clusters. The value of k should be predetermined in this algorithm.

jJ H How K-Means Clustering Worksf^

The K-Means Clustering Algorithm is a popular method for partitioning a d a ta se t into k distinct
clusters based on the similarity of data points. A step-by-step explanation of h o w the algorithm
works is given below:
Step 1: Initialization
• Choose the num ber of clusters, k.
• Initialize k cluster centroids (means) randomly or based on some heuristic.
Step 2: Assign D ata Points to Clusters
• For each data point in the dataset:
• Calculate the distance between each data point and each cluster centroid.
• Assign each data pointtothe cluster with the closest centroid [minimum distance).
Step 3: Update Cluster Centroids
• After assigning all data points to clusters, calculate the new centroid fo r each cluster by
taking the mean of all data points assigned to that cluster.
Step 4 : Convergence Check
• Repeat the assignment and centroid update steps until a stopping criterio n is met, such
as:
• No data point changes its cluster assignment.
• The centroids do not change significantly between iterations.
• Maximum number of iterations is reached.
Step 5: A lgorithm Iteration
• Iterate between assigning data points to clusters and updating clu ster centroids until
convergence.
Step 6: Final Output
• Once the algorithm converges:
• The final output is a set of k clusters, w here each cluster contains d ata points that
are m ore similar to each other than to data points in other clusters.
• The centroids represent the mean of data points within each cluster.
 ^. .^' 5 i*j‘-;.^>,'
-• ’ ’ . ■ Unsupervised Learnili

Algorithm K-Means Clustering Algorithm

Input:
D = (t,>tj, — ,t^) //Set of elements
k //Number of desired clusters
Output:
K //Set of clusters
K-means algorithm:
assign initial values for means ... ,m,^;
repeat
assign each item tj to the cluster which has the closest mean;
calculate new mean for each cluster;
_____ until convergence criteria is met;

The algorithm starts by randomly selecting K points from the dataset as the initial centroids of the
clusters. In each iteration, each data point is assigned to the nearest centroid, and the centroid of each
cluster is updated as the m ean of all the data points assigned to that cluster. This process is repeated
until convergence, which is achieved when the centroids no longer change or the change is below
a certain threshold. The output of the algorithm is K clusters, with each cluster containing the data
points that are closest to its centroid.

The K-means algorithm is widely used for clustering datasets with multiple attributes and can be
applied to a variety of domains, such as image segmentation, customer segmentation, and anomaly
detection.

K-Means Clustering Algorithm

Suppose we have four data points: Data Points: A(2,3], B(3, 3], C(8 , 6 ), and D(9, 5)
And we want to create two clusters (k=2)
Step 1: initialization:
• Randomly choose two initial cluster centroids (starting points):
Cluster 1 Centroid (Cl): (2,3)
Cluster 2 Centroid (C2): ( 8 , 6 )
Step 2: Assignment
• Calculate the distance from each data point to each cluster centroid and assign each point
to the nearest centroid using Euclidean distance:
For D ata Point A:
d([Link]) : Distance from A (2,3) to C l(2,3): V((2-2)^ + (3-3)^) = 0
d(A.C2) : Distance from A(2,3) to C2(8,6) : V((2-8)^ + (3-6)^) = V40
So, Data Point A is assigned to Cluster 1 (Cl).
For D ata Point B:
d(B,Cl) : Distance from B(3,3) to Cl(2,3) : ■/((3-2)^ + (3-3)^) = 1

d(B,C2) : Distance from 0(3,3) to C2(8,6): '/((3-8)=' + (3-6)=^) = VS4

______________________So, Data Point B is assigned to Cluster 1 (Cl).
4.16 L e a rn in g ;

For Data Point C:

d(C,Cl): Distance from C(8 ,6 ) to Cl(2,3) :V((8-2)" + (6-3)==) = VS8
d(C,C2); Distance from C(8 ,6 ) to C2(8,6): VC(8 -8 )" + ( 6 -6 )^) = 0
So, Data Point C is assigned to Cluster 2 (C2).
For Data Point D:
d(D,Cl): Distance from D(9,5) to C l(2 ,3): aA((9-2)^ + (5-3)^) = VS8
d(D,C2): Distance from D(9,5) to C2(8,6) : -/((9-8)^ + (5-6)^) = 1
So, Data Point D is assigned to Cluster 2 (C2).
Based on the distances calculated, we can see that Data Points A and B are closer to centroid Cl,
while Data Points Cand Dare closer to centroid C2. Therefore, Data Points A and B are assigned
to Cluster 1 (Cl), and Data Points C and D are assigned to Cluster 2 (C2).
Step 3: Update
• Recalculate the centroids for each cluster based on the points assigned to them.
For Cluster 1 (€1): New C l = Average of A and B = ((2+3)/2, (3+3)/2)= (2.5,3)
For Cluster 2 (C2): New C2 = Average of C and D: ((8+9)/2, (6+S)/2) = (8.5,5.5)
Step 4: Convergence Check:
• Check if the cluster centroids have stopped moving significantly. This is usually done
by comparing the old centroids with the new centroids. If they are almost the same
(within a predefined tiireshold), the algorithm converges. Otherwise, return to Step 2 for
reassignment
Step 5 : Output Clusters:
• The algorithm will converge after a few iterations, and we will have two clusters:
Cluster 1 contains Data Points A and B.
Cluster 2 contains Data Points C and D.

Example A Python Code to Demonstrate K-Means Clustering

# Import necessary libraries
import numpy as np
from [Link] import KMeans

# Create a dataset with Indian student performance data (student name, study hours,
exam score)
students = [Link]([['Srikanth', 100], ['Snigdha', 75], ['Mary', 35], ['Nirmala',
55], ['Raju', 85], ['Rama',30], ['Sita', 45], ['Lava', 65], ['Kusha', 25], ['Hanuman',
50]])

# Extract only the numeric features for clustering

student_features = students[:, 1: ] .astype(fioat)

# Apply K-Means clustering

kmeans = KMeans (n_clusters=3)
[Link](student_f eatures)
cluster labels = [Link]
 Unsupervised 4.17

# Output the clusters

clusters = {}
for i, label in enumerate(cluster_labels):
if label not in clusters:
clusters[label] = []
clusters[label].append(students[i]) # Append student data

# Display the clusters

for cluster, student_data in [Link]():
print(f"Cluster {cluster + 1}:")
for student in student_data:
print(f"Student Name: {student[0]},Exam Score: {student[l]}")
print 0
Output
Cluster 1:
Student Name: SrikanthjExam Score: 100
Student Name: Snigdha^Exam Score: 75
Student Name: Raju^Exam Score: 85

Cluster 2:
Student Name: Nirmala^Exam Score: 55
Student Name: Sita,Exam Score: 45
Student Name: Lava,Exam Score: 65
Student Name: HanumanjExam Score: 50

Cluster 3:
Student Name: Mary,Exam Score: 35
Student Name: Rama,Exam Score: 30
Student Name: Kusha,Exam Score: 25
Explanation
Data Preparation: The code initializes a dataset with student names and exam scores. Each student's
data is represented as an array with ± e student's name and exam score.
Feature Extraction: It extracts only the numeric feature (exam score) from the dataset for clustering.
By selecting only the exam scores as the feature for clustering, the code prepares the data for the
K-Means clustering algorithm.
K-Means Clustering: The code applies the K-Means clustering algorithm with n_clusters=3 to cluster
the students based on their exam scores into three clusters. K-Means is an unsupervised machine
learning algorithm that partitions the data into K clusters based on similarity.
Cluster Assignment: After clustering, the code assigns each student to a cluster based on the clustering
results. It creates a dictionary clusters where each key represents a cluster label, and the corresponding
value is a list of students belonging to that cluster.
Output Display: Finally, the code displays the clusters along with the student names and exam scores
in each cluster. It iterates through the clusters dictionary and prints the student names and exam scores
for each cluster, providing insights into how the students are grouped based on their exam performance.
 M o d iin e L e arn ii^

A p p lic a tio n s o f K -M e a n s C lu ste rin g

K-Means Clustering is a versatile algorithm with various applications across different dom ains. Some
common applications of K-Means Clustering include:
1. Customer S egm en tation : Businesses use K-Means to segment customers based on purchasing
behavior, demographics, o r other features to tailor marketing strategies and im prove customer
satisfaction.
2. Image Segm entation: In image processing, K-Means is used to segment images in to distinct
regions based on pixel intensity or color similarity, aiding in object recognition and image
analysis.
3. Anomaly Detection: K-Means can identify outliers or anomalies in datasets by clustering data
points and flagging th ose th at do not fit well into any cluster, helping in fraud detection or
error identification.
4 . Document Clustering: T ext documents can be clustered based on their content o r similarity
using K-Means, enabling document organization, topic modeling, and information retrieval.
5. Recom m endation S y stem s: E-commerce platforms and content providers u se K-Means to
group users with similar preferences and behaviors, facilitating personalized recommendations
and content delivery.
6. Genetics and Biology: K-Means clustering is applied in genomics to group genes w ith similar
expression patterns or in bioinformatics for protein sequence analysis and classification.
7. M arket Segm entation: Market researchers use K-Means to segment m arkets based on
consumer behavior, preferences, or geographic location, aiding in targeted marketing
campaigns and product positioning.
8. Network Traffic A n aly sis: K-Means clustering helps in analyzing network traffic patterns,
detecting anomalies or identifying clusters of similar network behavior for n etw o rk security
and optimization.
9. Healthcare: K-Means clustering is used in healthcare for patient segm entation, disease
diagnosis, and medical im age analysis, assisting in personalized treatment plans and healthcare
management
1 0 . Spatial Data Analysis: Geographic data can be clustered using K-Means to identify spatial
patterns, urban planning, resource allocation, and location-based services.

Advantages or B en efits of K-Means Clustering Algorithm

K-Means is computationally efficient and can handle large datasets with low computational co st
The algorithm is easy to implement and interpret, making it accessible to users of different skill levels.
K-Means can utilize various distance metrics to measure data point similarity.
It is well-suited for scenarios where clusters are spherical and exhibit similar variance.
Results from K-Means are easily interpretable, aiding in understanding data grouping.
The algorithm can be robust to outliers, minimizing their impact on clustering results.
K-Means can be parallelized to speed up the clustering process by distributing the computation across
multiple processors or nodes^______________________________________________________________
 Limits of K-M eans Clustering Algorithm

Limits of JSr^Meaiis Clustermg Algorithm

Understanding the limitations is crucial for selecting the appropriate clusteriflg algorithm based on the
characteristics of the data and the desired clustering outcomes.
• Multiple Runs for Optimal Solutions: K-Means may converge to suboptimal solutions due to its
sensitivity to the initial random centroids. To mitigate this, the algorithm needs to be run multiple times
with different initializations to improve the chances of finding the best clustering solution.
• Manual Selection of Number of Clusters: One of the challenges with K-Means is the need to specify
the number of clusters (K) beforehand. Determining the optimal number of clusters can be subjective
and may require domain knowledge or trial-and-error, making it a cumbersome task.
• Limited Cluster Shape Flexibility: K-Means assumes that clusters are spherical and of sim ilar size and
density. When clusters have varying sizes, different densities, or nonspherical shapes, K-Means may
struggle to accurately cluster the data, leading to suboptimal results.
• Inability to Handle Complex Cluster Shapes: In scenarios where the clusters exhibit complex shapes
such as ellipsoids, K-Means may fail to capture the true underlying structure of the data. This limitation
is evident when the clusters have different dimensions, orientations, and densities, as K-Means tends to
produce suboptimal or incorrect cluster assignments.
• Alternative Clustering Algorithms: Depending on the nature of the data and the shapes o f the clusters,
different clustering algorithms like Gaussian Mixture Models (GMM) may outperform K-Means. GMM is
more flexible in capturing clusters with varying shapes and densities, making it a suitable alternative
______ for datasets with non-spherical clusters.

Using Clustering fo r Image Segmentation

What is Image Segm entation ?

Image segmentation is a fundam ental task in image processing that involves dividing an image into
multiple segments or regions to simplify its representation and make it easier to analyze. The goal of
image segmentation is to partition an image into meaningful parts that correspond to ob jects or areas
of interest within the image.
By segmenting an image, we can extract important information, identify objects, boundaries (lines,
curves, etc.), and textures, and enable further analysis and processing tasks. The resu lt of image
segmentation is a set of segm en ts that collectively cover the entire image, or a set of boundaries
extracted from the image. Each o f the pixels in a region is similar v^^ith respect to some characteristic,
such as color, intensity, or textu re.

■ Use Case , Image Segmentation Use Cases

1. Tumor Detection in MR! Images
In medical imaging, MRI scans are used to visualize internal structuresofthe body, including detecting
tumors. Image segmentation plays a vital role in identifying and delineating tumors from surrounding
tissues in MRI images. By segmenting the MRI scan, doctors can accurately locate and analyze the size,
shape, and characteristics o f the tumor for diagnosis and treatment planning.
 2. Autonomous VehicIes:lmagesegmentationisessentiaIinthefieldofautonomousvehidesfo^^^^^^
like object detection, lane detection, and obstacle avoidance. By segmenting different elements th
scene such as vehicles, pedestrians, road markings, and traffic signs, autonomous vehicles can m^e
informed decisions for safe navigation. Semantic segmentation is often used to classify each pixe
an image into predefined categories, enabling the vehicle to understand its surroundings and react
accordingly to ensure safe driving._______________ ________________ ________ ________________

H m H How Image Segmentation Works?

Image segmentation involves dividing an image into distinct regions or segments, each represeMed
by l s l « or labeled Images. This segmentation enables the selective processing o f specific image
segments. It is u seM for targeted analysis rather than processing the enure image.
. One common technique in image segmentation is to detect abrapt changes in pixel values
which often correspond to edges that delineate different regions withm the im age. These edges
serve as boundaries between areas with varying characteristics, aiding m th e segmentaUon
process.
. Another approach involves identifying similarities among regions in an image. Techniques
such as region growing, clustering, and thresholding are utilized to group pixels with similar
attributes together, forming coherent segments based on shared characten stics.
Over time, a variety of domain-specific approaches have been developed to tecW e ^egment^on
challenges in specific application domains effectively. These tailored m ethods levemge domain
knowledge to address the unique requirements and characteristics of different typ es of images and
segmentation tasks. K-Means clustering is a method commonly used (or image segmentation.

K -M e a n s C l u s t e r i n g for Im ag e S e g m e n ta t i o n

K-Means clustering is a popular unsupervised machine learning alp)rithm

points into a specified num ber of clusters, in the context of image segmentation K-Means * s te n n g
can be applied to group pixels in an image into distinct clusters based on tiieir feam re similari^. By
clustering pixels based on features like color intensity, texture, or spaOal p roxim l^ , we « n identify
regions in an image that share common properties. This helps in separating objects or areas of
interest fi-om the background.

The K-means algorithm is a popular clustering technique used in image segm enUUon to partition an
image into K clusters based on p ix e l similarities
K -M e q n «
■VHrevm* Clustering for...
Image
w ^Segmentation
^ -- ---------- ---------------
1. Initialization: The algorithm starts by randomly initializing K duster centers in the feature space,
where K is the predefined number of dusters.
2 . - r - " - " ' step: Each pixel in the image is assigned to the cluster whose centroid is closest to It in
'te rm s of feature similarity. The distance m etric,olten Euclidean distance,is used to measures,milanty.

3. Update Step: After assigning ail pixels to clusters, the centroids of the clusters are recalculated based
on the m ean o f th e p ixel values within each cluster. ----------------------- -------------------------------- ---
 4. Iteration: Steps 2 and 3 are repeated iteratively until convergence criteria are met, such as a maximum
number of iterations or minimal change in cluster assignments.
5. Segmentation: Once the algorithm converges, each pixel in the image is associated with the cluster it
belongs to. It is effectively segmented the image into distinct regions based on pixel similarity.______

E xam ple A P^hon Code to Demonstrate Image Segmentation Using K-Means Clustering Method.
import [Link] as pit
from [Link] import KMeans
from [Link] import imread
from [Link] import resize
import nuoqpy as np

# Load the image

imagejaath = '[Link]'
original_i«age = imread (image_path)

# Reshape the image to be a list of pixels

pixels = original_image.reshape(-l, 3)

# Use KMeans clustering to segment the image

kmeans = KMeans(n_clusters=5) # Using 5 clusters
[Link](pixels)

# Replace each pixel value with its nearest centroid

segmented_img = kmeans. cluster_centers_[ kmeans. labels^]
segmented_img = segmented_img.reshape(original_image.shape)

# Convert data type to 'uintS' which is appropriate for displaying images

segmented_img = [Link](segmented_img, dtype=np.uint8 )

# Display the original and segmented images

fig, axes = [Link](l, 1, figsize=(1 2 , 6 ))
axes[0 ].imshow(original_image)
axes[0 ].set_title('Original Image')
axes[0 ].axis('off')

axes [1 ]. imshow (segmented_img)

axes[l].set_title('Segmented Image when K=5‘)
axes[l].axis('off')

# Create a legend for the segmented image

patches = [[Link]([], []j marker="o", ms=10, ls=''''j niec=Npne, color=(centroid / 255),
label="Cluster {}".format(idx),)[0] for idx, centroid in
enumerate(kmeans.cluster_centers_) ]
[Link](handles=patches, bbox_to_anchor=(1.05, - l)y. •loc=2, borderaxespad=0.,
title= ■Legend')

plt.tight_layout()
[Link] ______ _________ ___________________' .a— -------- — ----------------
4 .2 2 ^
HI
Output

Origrnal Image Segmented Image when K s3

Explanation
1. Image Loading and Reshaping:
• The code starts by loading an image from the given path [Link] using the imread function
from the [Link] module.
• The image is then reshaped into a two-dimensional array where each row represents a pixel and
each column represents the color channels (Red, Green, and Blue).
2. K-Means Clustering:
• KMeans clustering from scikit-leam is applied to the array of pixels with n_clusters=5, meaning
the image will be segmented into 5 different regions based on the color of the pixels.
• The fit method of the KMeans object is used to compute the clusters.
3. Centroid Assignment:
• Each pixel in the image is assigned to the nearest cluster centroid after the K-means algorithm
converges. The RGB values of each pixel are replaced with the RGB values of the centroid of the
cluster it belongs to, resulting in a segmented image where each segment has a uniform color.
4. Image Display:
• The code sets up a figure with two subplots using matplotlib to display both the original and the
segmented images side by side.
• The original image is displayed on the left, and the segmented image is displayed on the right
with the title 'Segmented Image when K=5'.
5. Legend Creation:
• A legend is created to help identify which colors correspond to which clusters.
• For each cluster, a colored dot is plotted with a label "Cluster {idx}", where {idx} is the index of the
cluster. This dot's color represents the color of the cluster centroid in RGB space.
Note: To run this code, the scikit-image library needs to be installed using pip install scildt-image

Using Clustering for Preprocessing

Clustering as a preprocessing step in a machine learning pipeline can enhance the performance of
downstream algorithms. Clustering can be a valuable tool for preprocessing data before applying
machine learning algorithms. Let us understand how clustering can be utilized for preprocessing
tasks
 Unsupervised Learning

1. O utlier Detection and H andling:

• Clustering algorithms can help identify outliers by grouping data points into clusters
based on their similarity. Outliers are data points that do not fit well within any cluster
and are often assigned to their own cluster or considered noise.
• Once outliers are identified, they can be handled by either removing them from the
dataset, assigning them to a specific cluster, or transforming their values to be within a
certain range.
E xam p le: In a dataset of customer transactions, outliers can be detected using KMeans
clustering. Outliers, representing potentially fraudulent transactions, can be assigned to
a separate cluster for further investigation or flagged for review.
2. Handling Missing Values:
• Clustering can be used to handle missing values by grouping data points with similar
characteristics and inferring the missing values based on the values of other data points
in the same cluster.
• By clustering data points with complete information, missing values can be estimated
based on the cluster's characteristics.
E xam p le : In a dataset of customer demographics, missing values for income can be
estimated by clustering customers based on similar age, education level, and occupation,
and inferring the missing income values from the cluster's income distribution.
3. D im ension Reduction:
• Clustering can serve as a preprocessing step for dimensionality reduction by grouping
similar features together and representing them with cluster centroids or representative
points.
• Techniques like KMeans clustering can be used to reduce the dimensionality of the
dataset by clustering similar features and retaining only the cluster centroids as new
features.
E xam p le : In a dataset with multiple correlated features, clustering can group similar
features together and represent them viath cluster centroids by reducing the number of
features while preserving the essential information for modeling.
4. F eatu re Selection:
• Clustering can help in feature selection by identifying clusters of features that are
highly correlated or redundant. Features within the same cluster may provide similar
information, and selecting representative features from each cluster can reduce
redundancy.
• By clustering features based on their importance or relevance to the target variable,
feature selection can be performed to retain only the most informative features for the
model.
Exam ple: In a dataset with numerous customer behavior features, clustering can
group together features that provide similar information (e.g., purchase frequency and
total spend), allowing for the selection of representative features from each cluster for
modeling.
 5. Data Transformation:
. Clustering can transform the data into a more suitable representation for machine learning
algorithms. By grouping similar data points together, the data can be transformed into
clusters or segments that capture underlying patterns and relationships.
• Data transformation through clustering can help in creating new features, encoding
categorical variables, o r normalizing data for b etter model performance.
Example: In a dataset of product reviews, clustering can group similar reviews together
based on sentiment and topics by enabling the creation of new features representing
sentiment clusters for sentiment analysis tasks.
6. Anomaly Detection:
• Clustering can be used for anomaly detection by identifying data points th at do not
conform to the patterns exhibited by the majority of the data.
• Outlier detection techniques within clustering algorithms can help in flagging anomalies
or unusual data points that may require special attention during preprocessing.
Example: In network traffic data, clustering can identify unusual patterns in network
behavior that deviate from normal traffic, helping in detecting potential cyber threats or
anomalies in network activity_________________________________________ __________
Exam ple I A Python Code WHanAe^Outlfers Using Clustering’ A ^ t . :

# Import necessary libraries

import numpy as np
import pandas as pd
from [Link] import KMeans
from [Link] import StandardScaler

# Create a sample student dataset with outliers

data = {
■StudentID': [1, 2, 3, 4, 5, 6, 7],
'ExamScore': [85, 70, 90, 65, 75, 120, 110],
'StudyHours': [5, 3, 6, 2, 4, 15, 12]
}
df = [Link](data)

# Standardize the features

scaler = StandardScaler()
scaled_features = scaler.fit_transform(df[['ExamScore', 'StudyHours ]])

# Apply KMeans clustering to identify outliers

kmeans = KMeans(n_clusters=2, random_state=42)
df['Cluster'] = k m e a n s . fit_predict(scaled_features)

# Remove outliers based on cluster assignment

df_cleaned = df[df['Cluster'] == 0] __________________ ______
# Display the original and cleaned datasets
print("Original Student Dataset:")
print(df)

print("\nCleaned Student Dataset after removing outliers:")

print(df_cleaned)
Output ^
Original Student Dataset:
StudentID ExamScore StudyHours Cluster
0 1 85 5 0
1 2 70 3 0
2 3 90 6 0
3 4 65 2 0
4 5 75 4 0
5 6 120 15 1
6 7 110 12 1

Cleaned Student Dataset after removing outliers:

StudentID ExamScore StudyHours Cluster
0 1 85 5 0
1 2 70 3 0
2 3 90 6 0
3 4 65 2 0
4 5 75 4 0
Explanation
1. Data Preparation:
• The sample student dataset contains columns for 'StudentID', 'ExamScore', and 'StudyHours',
with some entries having outlier values (e.g., ExamScore of 120 and 110).
• The features 'ExamScore' and 'StudyHours' are standardized using StandardScaler to ensure they
are on the same scale for clustering analysis.
2. Outlier Identification with KMeans Clustering:
• KMeans clustering with 2 clusters is applied to the standardized features to identify outliers. The
clustering algorithm groups data points into clusters based on similarity.
• Outiiers are identified as data points that do not fit well within any cluster and are assigned to a
separate cluster [e.g.. Cluster 1) due to their dissimilarity from the majority of the data.
3. Outlier Removal and Cleaned Dataset:
• After clustering, the dataset is cleaned by removing outliers based on their cluster assignment.
In this case, data points belonging to the outiier cluster (e.g.. Cluster 1) are filtered out to create a
cleaned dataset
• The cleaned dataset contains only the data points that are considered non-outliers based on the
_________clustering analysis, providing a more refined dataset for further analysis or modeling.__________
 Uarning . ... Z :r'- :y

Example A Python Code to D e t^ Anomalies Using Clustering

import numpy as np
import pandas as pd
from [Link] import KMeans
from [Link] import make_blobs
import [Link] as pit
from [Link] import cdist

# Generate synthetic data ^ ^

X, _ = make_blobs(n_samples=300, centers=4, cluster_std=0.60, random_state-0)

# Visualize the data

[Link](X[:, 0]j X[;, 1], s=50)
[Link]'Data Distribution")
[Link]'Feature 1")
[Link]'Feature 2")
[Link]

# Apply KMeans clustering

kmeans = KMeans(n_clusters=4)
[Link](X)
cluster_centers = kmeans.cluster_centers_

# Assign clusters and calculate the distance from each point to its assigned cluster

X dist = cdist(X, cluster_centers, 'euclidean') # Distance of X to cluster centers

cIosest_cluster_index = [Link](X_dist, axis=l) # Index of the closest cluster
min_distances = X_dist[[Link](len(X_dist)). closest_cluster_index] # Min distance
to cluster center

# Define a threshold for which data points to consider anomalies

threshold = [Link](min_distances, 95) # Setting threshold at the 95th percentile

# Detect anomalies
outliers = X[min_distances > threshold]

# Visualizing the results

[Link](X[:, 0], X[:, 1], c='blue', label='Normal Data')
[Link](outliers[:, 0], outliers[;, 1], c='red', label='Anomalies ) .
[Link](cluster_centers[:, 0], cluster_centers[:, 1], c='green', marker= x ,
label='Cluster Centers')
[Link]'Anomaly Detection using K-Means Clustering")
[Link]'Feature 1")
[Link]("Feature 2")
[Link]
[Link] _________________ _______ _____________ _____ ________________ _
 Data Distribution Anomaly Detection using K-Means austering
• Noftnal Data
/Xmnalies
Ouster Centers

' ■ m - •• •••

• • •
’ • '-m •;
- 3 - 2 - 1 0 1 2 3
Feature 1
Explanation'
1. Data Generation: The code begins by generating synthetic data using make_blobs from Scikit-learn,
simulating a dataset with 300 samples grouped into 4 clusters.
2. Lustering with K-Means: K-Means clustering is applied to the dataset specifying 4 clusters. Each data
point is then assigned to the nearest cluster.
3. Distance Calculation: The distance of each data point to its nearest cluster center is calculated. This
distance metric helps in identifying how far a point is from the cluster's core.
4. Setting a Threshold: A threshold is set at the 95th percentile of these distances. Data points whose
distance to the nearest duster center exceeds this threshold are considered anomalies.
5. Anomaly Detection: Points are labeled as anomalies if their distance to the nearest cluster center is
greater than the defined threshold. These are visualized in red, whereas normal points are in blue.
6 . Visualization: The data, along with identified anomalies and cluster centers, are visualized using
Matplotlib. This visual representation helps to clearly see the normal data, the anomalies, and the
cluster centers.

m Using Clustering for Sem i-Supervised Learning

Semi-supervised learning is a machine learning paradigm that combines elem ents o f both supervised
and unsupervised learning. In this approach, the m odel is trained on a dataset th at contains a small
amount of labeled data along with a larger am ou nt o f unlabeled data. The goal is to leverage the
information from the labeled data to make p red ictions on the unlabeled data, thereby improving the
model's performance.
In semi-supervised learning:
• Supervised L earn in g: The model learns from a small set of labeled examples where the
input data is paired with the corresponding output labels. This labeled data provides explicit
information on how the model should m ake predictions.
• Unsupervised Learning: The model also le a m s from a larger set of unlabeled examples where
the input data is not paired with output lab els. The model must infer patterns and structures
from the unlabeled data on its own.
4.28 'l^chine'iiearning'^

How C lustering is Used m Sem i-Sup ervised Learning?

Clustering plays a crucial role in enhancing semi-supervised learning by leveraging the inform ation
present in both labeled and unlabeled data. In many practical applications, acquiring a large am ount
of labeled data can be expensive or im practical or time consuming. Clustering techniques offer a way
to extract valuable insights from the unlabeled data to improve the m odel's performance.

1. Leveraging Unlabeled Data:

In semi-supervised learning, the availability of unlabeled data presents an opportunity to
uncover hidden patterns and stru ctu res vvithin the dataset. Clustering algorithms, such as
KMeans or DBSCAN, can group sim ilar data points together based on their features, even in
the absence of explicit labels. By identifying clusters in the unlabeled data, the model can learn
from the inherent structure o f the data, potentially improving its ability to make accurate
predictions.
2. P seudo-Labeling:
One of the key ways clustering enhan ces semi-supervised learning is through pseudo-labeling.
Pseudo-labeling involves assigning labels to the unlabeled data based on the clusters they
belong to. By using clustering to create pseudo-labels, the unlabeled data can be incorporated
into the training process as if it w ere ftilly labeled. This approach effectively expands the
labeled d ataset and allows the m odel to learn from a larger pool o f data.
By assigning pseudo-labels derived from clustering, the model can benefit from the additional
information present in the unlabeled data. This process helps in refining the learning model
and potentially improving the m odel's generalization capabilities.

The process of using clustering in sem i-supervised learning involves several key steps to effectively
leverage the information from both labeled and unlabeled data. A structured approach to incorporating
clustering in semi-supervised learning:
1. D ata P rep aratio n :
• S p lit th e Data: Divide the d ataset into labeled and unlabeled portions. The labeled data
contains input features along with corresponding output labels, while the unlabeled data
lacks explicit labels.
2. C lu stering U nlabeled D ata:
. Apply Clustering A lg o rith m s: Utilize unsupervised clustering algorithms [e.g., KMeans,
DBSCAN) on the unlabeled data to identify clusters based on similarities in feature
space.
3. P seudo-Labeling:
. A ssign Pseudo-Labels: For each cluster generated by the clustering algorithm, assign
pseudo-labels to the data points within the cluster. This can be done by considering the
m ajority label of the labeled data points in the same cluster.
. L abel P ropagation: Propagate th e pseudo-labels to the unlabeled data points w ithin the
clusters for effectively creatin g a partially labeled dataset.
 m
4. Model Training:
• Combine Labeled and Pseudo-Labeled Data: Merge the labeled data with the newly
pseudo-labeled data to create an augmented training set.
• T rain the Model: Use a semi-supervised learning algorithm (e.g., self-training, co ­
training) to train the m odel on the combined dataset by leveraging both labeled and
pseudo-labeled data for learning.
5. Model Evaluation:
• Validate the Model: Evaluate the trained model on a separate validation set to assess its
performance and generalization to unseen data.
• Fine-Tuning: Iterate on the model training process by adjusting h3 q)erparameters and
clustering parameters as needed to improve performance.
6. P red iction and Inference:
• Make Predictions: Use th e trained model to make predictions on new, unseen data,
leveraging the knowledge gained from both labeled and unlabeled data.
• In corp orate Feedback: Continuously update and refine the model based on feedback
and new labeled data to enhance its predictive capabilities.
Example A Python Code to Use Clustering in Semi-Sup'ervised Learning
import numpy as np
import pandas as pd
from [Link] import KMeans
from [Link] import StandardScaler
from [Link] import DecisionTreeClassifier
from [Link] import accuracy_score

# Create a sample DataFrame

data = {
•Student ID': [1, 2, 3, 4, S, 6 ] ,
■Exam Score': [85, 35, 90, 75, 25, 78],
■Attendance': [90, 55, 95, 85, 40, 88],
■Label': [■Pass', 'Fail', 'Pass^, None, None, None]
}
df = [Link](data)

# Display initial data

print("Initial Data:")
print(df)

# Prepare data for clustering

df['Label'] = df['Label'].map({'Pass': 1, 'Fail': 0})
features = ['Exam Score’, 'Attendance']
# Scale features
scaler = StandardScaler()
scaled_features = s c a l e r . fit_transform(df [features] )

# Use KMeans to infer labels for the unlabeled data

kmeans = KMeans(n_clusters=2, random_state=42)
clusters = k m e a n s . fit_predict(scaled_features)

# Assuming the larger cluster is 'Pass' and the smaller is Fail

if [Link](clusters == 0) > [Link](clusters == 1)-
cluster_to_label = {0: Ij 1' 0}
else:
cluster_to_label = {1: 1, 0: 0}

# Corrected Pseudo-Label Assignment using the correct tuple indexing

df['Pseudo Label-] = [[Link][cluster] if [Link](row[l]C'Label 1) else
row[l]['Label']
for cluster, row in zip(clusters, [Link]())j

# Display data after pseudo-labeling

print("\nData after Pseudo-Labeling:")
p r i n t ( d f [ [ 'Student ID', 'Exam Score', 'Attendance', 'Pseudo Label ]])

# Train a decision tree classifier on the pseudo-labeled data

X_train = scaled_features
y_train = df['Pseudo Label'].astype(int)

classifier = DecisionTreeClassifier(random_state=42)
[Link](X_train, y_train)

# Make predictions (in a real scenario, use a separate test set)

predictions = classifier. predict(X_train)__________ _______ — _

Initial Data:
student ID Exam Score Attendance Label
0 1 85 90 Pass
1 2 35 55 Fail
2 3 90 95 Pass
2 4 75 85 None
4 5 25 40 None
5 6 78 ________ 88 None
 -^Unsupervisedj

Data after Pseudo-Labeling:

Student ID Exam Score Attendance Pseudo Label
0 1 85 90 1 .0

1 2 35 55 0 .0 -•fitr/.'
2 3 90 95 1 .0 ' '> . .aSc ::
3 4 75 85 1 .0
4 5 25 40 0 .0

5 6 78 88 1 .0

Explanation
1. Data Preparation and Initial Display;
• The code begins by creating a pandas DataFrame from a dictionary that includes students’ IDs
their exam scores, attendance rates, and some labeled data ('Pass', 'Fail', and None for uhlabeled).
• The initial data is printed to give an overview of what is being processed. This hel^s In
understanding the dataset structure before any operations are applied.
2. Data Preprocessing t
• Label Conversion: The categorical labels 'Pass' and 'Fail' are converted into numerical fonnat
(1 for 'Pass', 0 for 'Fail'). This conversion is necessary for mathematical operations and model
training that will follow.
• Feature Scaling: The 'Exam Score' and 'Attendance' features are scaled using StahdaidScaler
from scikit-learn. Scaling is crucial as K-Means clustering is sensitive to the scale'df data, and
scaling ensures that each feature contributes equally to the distance calculations in the clustering
process. ’ , . . . I ,,,

3. Clustering for Pseudo-Labeling M '^ ! . . ' *

• K-Means Clustering: The scaled features are clustered using K-Means with 2 clustei^lJhe fdea
is to identify two groups within the data which should ideally correspond to-'Pass' and 'Fail*. , .
• Cluster to Label Mapping: A mapping from clusters to labels is created based on the assumption
that the larger cluster corresponds to 'Pass'. This is a heuristic and might need adjustment
depending on the actual distribution of data.
4. Assignment of Pseudo-Labels * ' v? "
• A new column 'Pseudo Label' is added to the DataFrame. For each data point, if the label is already
known (not NaN), the original label is retained. If the label is NaN (missing), the label infeired
from the cluster assignment is used. ‘ » ^'
• This step effectively uses clustering to provide labels for unlabeled data, leveraging the patterns
found in the features to make educated guesses about the appropriate labels. «,
5. Model Training and Evaluation "
• Decision Tree Training: A Decision Tree Classifier is trained on the dataset using both original
and pseudo labels. Decision trees are suitable for this kind of task because they can handle both
numerical and categorical data and are easy to interpret______ ': ' ■ . ■
 DBSCAN
DBSCAN is a popular clustering algorithm which is fundamentally different from k-means and its
variants. DBSCAN is an acron 5on for "Density-Based Spatial Clustering of Applications with Noise".
DBSCAN is particularly useful for identifying clusters o f varying shapes and sizes in a dataset, which
can include noisy and outlier points. The algorithm is widely used due to its simplicity and the
robustness it offers, especially in dealing with outliers.
DBSSCAN is an unsupervised clustering algorithm. DBSCAN clustering can work with clusters of any
size from huge amounts of data and can work with datasets containing a significant amount of noise.
It is basically based on the criteria of a minimum num ber of points within a region.

What is DBSCAN Algorithm?

DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is a clustering algorithm in machine
learning that groups together points that are closely packed based on a density criterion. It is particularly
useful for identifying clusters of varying shapes and sizes in a dataset, while also being robust to noise
and outliers.

m l Importance o f DBSCAN
1. Density-Based: DBSCAN works on the idea of density connectivity and density reachability.
It groups together points that are closely packed together,. It mark the points that lie alone in
low-density regions as outliers.
2. Robust to Noise: DBSCAN is robust to noise and can identify outliers as noise points, making
it suitable for datasets with noise or outliers.
3. Handles Clusters o f Varying Shapes and D en sities: DBSCAN can identify clusters of arbitrary
shapes and sizes, unlike K-means, which assum es spherical clusters.
4. No Need to Specify Number of Clusters: Unlike K-means, DBSCAN does not require specifying
the number of clusters beforehand, making it m ore flexible.
5. Efficient: DBSCAN is computationally efficient and can scale well to large datasets.

H ow D B S C A N W o rk s ?

DBSCAN, or Density-Based Spatial Clustering of Applications with Noise, is a clustering algorithm

that groups data points based on their density. Let us understand how DBSCAN works:
1. Core Points, B o rd e r Points, and Noise P o in ts:
• Core Points: Imagine core points as central hubs in a cluster. A point is considered a core
point if it has a minimum number of neighboring points within a specified distance.
• B order Points: Border points are on th e outskirts of a cluster. They are reachable from
core points but do not have enough neighbors to be core points themselves.
• Noise Points: Noise points are outliers th a t do not belong to any cluster.
ir 2. Param eters:
. ..' r **'N E)
• Epsilon (e p s): Epsilon is defined as the radius of each data point around which the
density is considered. This defines the maximum distance between two points for them
to be considered neighbors.
• Minimum Samples (m in_sam ples): It is the number of points required within the
radius so that the data point becomes a core point
3. Algorithm Steps:
• Initialization: The algorithm begins by randomly selecting a point from the dataset that
has not been visited. This initial point serves as the starting point for forming a cluster.
• Expand:
• For each core point or border point (reachable from a core point), the algorithm
expands the cluster by adding neighboring points recursively.
• It checks the neighboring points of the current point to determine if they should be
included in the cluster.
• If a neighboring point meets the criteria to be a core point or a border point, it is
added to the cluster.
• This process continues iteratively, expanding the cluster by including points that
are within the specified distance (epsilon) and have the minimum number of
neighbors (min_samples).
• Term ination: The algorithm stops when all points have been visited.
4. Output:
• Clusters: Points that belong to the sam e cluster based on density.
• Noise: Outliers or points that do not fit into any cluster.

U nderstanding Core Points, B o rd e r Points and Noise Points

In the DBSCAN algorithm, a circle with a radius epsilon is drawn around each data point and the data point
is classified into Core Point, Border Point, or Noise Point. The data point is classified as a core point if it has
min_samples of data points with epsilon radius. If it has points less than [Link] it is known as Border
Point and if there are no points inside epsilon radius it is considered a Noise Point.
Let us understand working through an example.
In the above figure, we can see that point A has no
points inside epsilon(e) radius. Hence it is a Noise
Point. Point B has min_samples(=4) number of
points with epsilon(e) radius, thus it is a Core Point
While the point C has only 1 ( less than minPoints)
point, hence it is a Border Point..
JS q Example Working of DBSCAN Algorithm
Suppose we have a dataset of points representing customers in a shopping mall based on their spending score
and annual income. We want to group these customers into clusters using DBSCAN.
1. Core Points, Border Points, and Noise Points:
• Core Points: A core point could be a customer who has at least 5 other customers within a
distance of 10 units. These core points act as central hubs in a cluster.
• Border Points: Border points are customers who are reachable from core points but do not have
enough neighbors to be core points themselves.
• Noise Points: Noise points are customers who do not belong to any cluster, perhaps because
they are outliers in terms of spending score and income.
2. Parameters:
• Epsilon (eps): Let's set epsilon to 10 units, meaning points within a distance of 10 units are
considered neighbors.
• Minimum Samples (min_samples): We require at least 5 points within the epsilon radius for a
point to be considered a core point.
3. Algorithm Steps:
• Initialization: Start by randomly selecting a customer who has not been visited as the Initial
point for forming a cluster.
• Expand:
o For each core point or border point, expand the cluster by adding neighboring customers
recursively based on the epsilon and min_samples criteria.
o Check If the neighboring customers meet the criteria to be core points or border points
and add them to the cluster.
• Termination: The algorithm stops when all customers have been visited and clustered.
4. Output:
• Clusters: Customers grouped together based on their spending score and Income density.
• Noise: Outliers or customers who do not fit well into any cluster.
In this example, DBSCAN would help Identify clusters of customers with similar spending behaviors and
income levels, while also highlighting outliers who do not conform to any specific cluster pattern.

Exam ple A Python Code to Demonstrate DBSCAN

import numpy as np
import [Link] as pit
from [Link] import DBSCAN
from [Link] import StandardScaler

# Creating a dummy dataset: Coordinates of locations

# Example points (x, y) and some outliers
 ---- - ~x>£t::- ■1 ...; A ' 'v a

data = np. array ([ '

[1. 2], [2, 2], [2, 3], [8 , 7], [8 , 8 ], [25, 80],
[6 , 5], [5, 4], [5, 5], [5, 6 ], [4, 5], [7, 6 ],
[6 0 , 7 0 ],
[100, 2], [103, 2], [104, 3], [1 0 0 , 4], [102, 3],
])

# Standardize the data

scaler = StandardScaler()
‘^3 ta_scaled = scaler.fit_transform(data)

# DBSCAN algorithm
dbscan = DBSCAN(eps=0,5, inin_samples=2 )
clusters = dbscan.fit_predict(data_scaled)

# Plotting the results

[Link](figsize=(8 , 4))
unique_labels = [Link](clusters)
colors = [[Link](each) for each in [Link](0 , 1, len(unique_labels))]

for k, col in zip(unique_labels, colors):

if k == -1 :
# Black used for noise (outlier),
col = [0 , 0 , 0 , 1 ]

class_member_mask = (clusters == k)

# Plot data points that are clustered

xy = data[class_member_mask]
[Link](xy[:, 0 ], xy[:, 1 ], 'o', markerfacecolor=tuple(col),
markeredgecolor='k ', markersize=14 if k == -1 else 12)

# Plot outliers
xy = data[~class_member_mask]
pit•plot(xy[:, 0 ], xy[:, 1 ], 'o', markerfacecolor=tuple(col),
markeredgecolor='k ', markersize=6 )

[Link]( 'DBSCAN: Clusters and Outliers')

[Link]('Feature 1 ')
[Link] 2 ')
[Link]
 nR<;rAM- riii<;ters and Outliers

Feature 1

Explanation,
1. Data Preparation
. The daaset consists of manually specified points in a 2D space repi^sendng '
The cooKiinates include typical clusters and distinct outliers (e.g„ points at [25, 80] [6 ,
70]).
2. Feature Scaling , .
. Before applying DBSCAN, the data is standardized using [Link] scaling adjusts each
f e a ^ t v T z e r o mean and unit variance, which is crucial because DBSCAN is sensitive to fte
" between points. Standardizing the data helps prevent features with larger scales from
dominating how clusters are formed.
3 . DBSCAN Clustering
. DBSCAN is initiated with an eps value of 0.5 and [Link] of 2. These parameters dictate th
clustering behavior; .
. eps (epsilon) is the maximum distance between two points for one to be considered as m t e
neighborhood of the other.
. min samples is the minimum number of points required to form a dense region (i.e., a duster).
. The'algorithm is expected to identify core points, border points, and ouUiers based on these
parameters.
4. Identification of Clusters and Outliers
• The DBSCAN algorithm categorizes the points into clusters and noise (outliers).
. In this dataset; Two main clusters are identified in denser regions of the dataset, where points are
close together. . ^ .
. Two points located at [25. 80) and [60, 70] are marked as noise because they do not meet th<
__________ renulred density fdefined by eps and min_samples) to be mcluded m a cluster-----------------------
II ‘SUnsu'peryised Learning
E)
5. Visualization
• The results are plotted using matplotlib, where different clusters are marked with different
colors, and outliers are colored in black.
• This visual representation helps illustrate DBSCAN's effectiveness at separating closely-knit
groups from sparse points, enhancing the understanding of cluster formation and outlier
________ detection in spatial data.___________________________________________________________

Applications of DBSCAN

DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is a popular clustering

algorithm that is particularly useful in various real-world applications. Some of the applications of
DBSCAN include:
> Anomaly Detection: DBSCAN can be used for anom aly detection in various domains such as
fraud detection in finance, network intrusion detection in cybersecurity, and detecting outliers
in data.
> Spatial Data Analysis: DBSCAN is commonly used in geographical information systems (CIS]
for spatial data clustering, such as identifying hotspots in crime analysis, clustering GPS data
points, and segmenting satellite images.
> Customer Segm entation: In marketing and custom er relationship management, DBSCAN
can be applied to segment customers based on th e ir behavior, preferences, or geographical
locations.
> Image Segm entation: DBSCAN can be used for im age segmentation tasks, such as grouping
pixels with similar characteristics together in medical imaging, object detection, and computer
vision applications.
> Recommendation System s: DBSCAN can help in building recommendation systems by
clustering users or items based on their sim ilarities, preferences, or interactions.
> Density Estim ation: DBSCAN can be used for density estimation tasks, such as estimating the
density of data points in a high-dimensional space.
> Robotics and A utonom ous Vehicles: DBSCAN can assist in robotics and autonomous vehicle
navigation by clustering sensor data to identify obstacles, map environments, and plan paths.
> Biom edical Data A nalysis: DBSCAN is applied in biomedical research for clustering gene
expression data, identifying disease patterns, and analyzing medical imaging data.
> Social Network A nalysis: DBSCAN can be used to analyse social networks by clustering users
based on their interactions, interests, or connections.
> Tim e Series Analysis: DBSCAN can be adapted for time series clustering tasks, such as
grouping similar temporal patterns in financial data, sensor data, or loT applications.
4.38 l^ r n in g ''

A d v a n ta g e s an d D isad van tag es of DBSCAN

I Advantages of the DBSCAN Algorithm

Robust to Noise: DBSCAN is n>bust to noise and can [Link], handle outliers in the dati without

being influenced by them.

2. Ability to Identiiy Arbit«r,-Shap«i Clusters: DBSCAN can identlly clusters of vanous shapes a .
Sizes making it suitable for datasets with com plex cluster structures.

4 E m r e r 'f o r ta rg e Datasets: DBSCAN is efficient for large d ^ « as it o „^ need= to con,pute

pairwise distances between points within a specified neighborhood radius.

s L m e t e r Robustness: DBSCAN Is less sensitive to its parameten>, such as the neighboi^ood radius
■ Ceps) and minlmun. number of points ([Link]), compared to other clustenng algonthms.
6 Handles Uneven Cluster Densities: DBSCAN can handle clusters with vanring densities, making i
suitable for datasets where clusters have different densities._______________________________________

| O j ^ D i s a d ^ t a ^ s:^ th eD B S C A N ^ gciirith m ^

1 Sensitive to Parameters: While DBSCAN is less sensitive to parameters compared to some ottier
■ clustering algorithms, choosing the right values for epsilon (eps) and minimum points ([Link])
can still be challenging and may impact the clustering results.
2 Difflcuity with Varying Density: DBSCAN may struggle with datasets where clusBrs have varying
" a l l t r e l i e l a s i n g l e e p s i , o n valueto define thenelghborhood radius forall points.

3 Dimcnity witb High-Dimensional Data: In high-dimensional spaces, the conceptofdistancebecomes

lessmeaningfiil,whlchcanaffecttheperformanceofDBSCAN.
i Not suitable for Clusters of Varying Densities: DBSCAN may not perform well on daBseB ™th
' clusters of significantly varying densities, as it uses a single epsilon value for deflmng neig or oo ^
5. Memory intensive: DBSCAN requires storing the entire dataset in memory B. compute the density-
based clusters, which can be memory-intensive for veiy large datasets.
6. Border Point Sensitivity: The assignment of border points to dusters can be sensitive to the order o
data points, leading to potential variations in clustering results.--------------- - ---- ------------------------------

Other Clustering Algorithms7

provided by Scikit-Learn, which cater to a variety of needs and data charact*nsO cs.
1. AggjomerativeClustering: *S
• D escription: Agglomerative clustering starts with individual instances as separate
clu sters and Iteratively merges the closest pair of d usters u n til a ll instances belong to a
single cluster. This process creates a hierarchy of clusters, re p rese n te d as a dendrogram.
• Advantages: It can capture clusters of various shapes and sizes, does not require
sp ecifyin g the number of clusters beforehand, and is su itab le for datasets with a large
n u m ber of instances.
• Scalability: Agglomerative clustering can scale well to large d atasets if a connectivity
m atrix is provided, indicating which instances are neighbors. W ithout a connectivity
m atrix, th e algorithm may not scale efficiently for large d atasets.
2. BIRCH (Balanced Iterative Reducing and Clustering using Hierarchies):
• D escription: BIRCH is designed for handling large datasets efficien tly by building a tree
stru ctu re during training. It uses a compact representation to assign new instances to
clu sters without storing all instances in memory.
• Advantages: BIRCH can handle large datasets with lim ited m em ory, making it suitable
for sce n a rio s where memory usage is a concern. It provides re s u lts comparable to batch
K-M eans clustering.
• Lim itation: BIRCH works best for datasets with a moderate n u m b e r of features, typically
less th a n 2 0 , due to the tree structure's complexity.
3. Mean-Shift:
• D escription: Mean-Shift clustering is a non-parametric clu sterin g algorithm that
iterativ ely shifts data points towards the mode of the d ata distribution. It identifies
clu sters b y finding density peaks in the data.
• Advantages: Mean-Shift can discover clusters of arbitrary sh a p e s and sizes without
req u irin g the number of clusters as an input parameter. It is effectiv e in handling datasets
with irregu lar cluster shapes.
• Lim itation: The computational complexity of Mean-Shift is q u ad ratic (0(m ''2)], making
it less suitable for large datasets due to its high com putational cost.
4. Affinity Propagation:
• D escription: Affinity Propagation identifies exemplars in the data that represent clusters
and assig n s data points to these exemplars based on sim ilarity m easures. It uses message
passing to determine the exemplars and cluster assignments.
• Advantages: Affinity Propagation can automatically d eterm in e th e number of clusters
and is effective in identifying clusters of varying sizes and shapes.
• Lim itation: The algorithm's computational complexity is q u ad ratic (0(m ''2)), making it
less e fficien t for large datasets due to its high computational dem ands.
 .............. . .........................

5. S p ectral Clustenng: ^ low er-dim ensional space

• r „ r r e ;~ “ L ;m ,^

.
in idendiyiDg clusters In graph data, such as sodal networks.

Each clustering algorithm offers unique a lg ^ t ^ is crucial for

the desired clustering outcomes,

l i f l l R e v ie w Q u estion s

1. M ention the two kinds of unsupervlsed learning?

2. W hat is clustering?
3. Give tw o examples of using clustering to solve real life problem s.

4. What is Partitioning Clustering?

5. Mention any two Partitioning Clustering Algorithms.

6. What is Density-Based Clustering?

7. What is Hierarchical Clustering?
8. Mention any two Hierarchical Clustering Algorithms.
9. What is Distribution Model-Based Clustering?

10. What is Fuzzy Clustering?

11. What is K-Means Clustering?
12. What is the objective of K-Means Clustering?
13. Write any 2 applications of k means clustering?
14. What is Image Segmentation ?
15. W hat is DBSCAN?
16 M ention the twoparameters used in DBSCAN.
17. W hat are Core Points, Border Points, and Noise Points in DBSCAN.

18. Can we find outliers using k-raeans? lustily.

1. Explain the Importance of Clustering in Unsupervised Learning.

2. W rite any 5 applications of Clustering.

3. W rite a note on Clustering Attributes.
4. W h at is Partitioning Clustering? Mention any two Partitioning Clustering Algorithms.

5. W h at is Density-Based Clustering? Explain with an example.

6. W h at is Hierarchical Clustering? Explain with an example.
7. Explain Similarity and Distance Measures in Clustering.
8. W rite th e applications of K-Means Clustering.
9. W rite th e Advantages and Limits of K-Means Clustering Algorithm.

10. Explain the limitations of k-means clustering.

11. W h at is Image Segmentation? How Image Segmentation W orks?
12. How Clustering is used in Preprocessing?
13. How Clustering is Used in Semi-Supervised Learning?
14. W h at is DBSCAN Algorithm? W rite the Importance of DBSCAN.

15. W rite th e Applications of DBSCAN.

16. W rite th e Advantages and Disadvantages of DBSCAN.
17. Explain how a cluster formed in DBSCAN clustering algorithm ?
18. W rite a note on a) Agglomerative Clustering b) BIRCH (Balanced Iterative Reducing and
Clustering using Hierarchies)
19. W rite a note on a) Mean-Shift b) Affinity Propagation

20. W rite a note on Spectral Clustering.

1. Explain the different types of Unsupervised Learning.

2. W h at is Clustering? Explain the Importance of Clustering in Unsupervised Learning.

3. Explain the types of Clustering Methods or Techniques.

4. How K-Means Clustering Works? Write an algorithm.
5. W rite a Python Code to Demonstrate K-Means Clustering.
6. Explain K-Means Clustering for Image Segmentation. W rite an algorithm.
7. W rite a Python Code to Demonstrate Image Segmentation Using K-Means Clustering Method.
8. Explain How Clustering is used in Preprocessing.

9. W rite a Python Code to Handle Outliers Using Clustering.

 jLerqrning

10. Write a Python Code to Detect Anomalies Using Clustering.

11. Explain the Process of Using Clustering in Semi-Supervised Learning.
12. Write a P3^ o n Code to Use Clustering in Semi-Supervised Learning.
13. Explain How DBSCAlSh Works?
14. Write a Python Code to Demonstrate DBSCAN.
15. Analyze the given dataset of student exam scores and manually cluster them into three
performance categories (high, medium, low) using the K-Means clustering algorithm.

Student ID Exam Score'

1 85
2 70
3 95
4 78
5 88
6 65
7 92
8 75
9 82
10 60
 LAB PROGRAMS

vS ni.

B a :

- Install and set up Python and essential libraries like NumPy and pandas.
•■ Introduce scikit-learn as a machine learning library.
-■ Install and set up scikit-learn and other necessary tools.
^ Write a program to Load and explore the dataset of .CVS and excel files using pandas.
^ Wnte a program to Visualize the dataset to gain insights using Matplotlib o r Seaborn by
plotting scatter plots, b a r charts. ,

^ ^Iling^ to Handle missing data, encode categorical variables, and perform feature

- Write a program to im plem ent a k-Nearest Neighbours (k-NN) classifier using scikitlearn and
Tram the classifier on th e dataset and evaluate its performance.
- Write a program to im plem ent a linear regression model for regression tasks and
Train the model on a d ataset with continuous target variables.

- Write a program to implement a decision tree classifier using scikit-learn and visualize the
decision tree and understand its splits.

- Write a program to Im plem ent K-Means clustering and Visualize clusters.

A .2 ^ : Machine Learning
r r # A '' . ' -, ’

Program 1 InstaU and set up Python and essential libraries like NumPy and pandas.
Setting up Python and essential libraries on a Windows system for machine learning involves a series of
straightforward steps thatpreparetheenvironmentfordataanalysisandalgorithmdevelopmentBymstalhng
Python along with NumPy and Pandas, users can handle a wide array of data manipulation tasks efficien y
Follow the below steps to set up Python and essential libraries such as NumPy and Pandas for machme
learning on Windows.
Step 1: Install Python
Download Python: Go to the official Python website at [Link], navigate to the "Downloads"
section, and download the latest version for Windows. Choose the executable installer.
install Python: Execute the downloaded file. It is crucial to check the box labeled "Add Python to
PATH" at the start of the installation wizard. Select "Customize installation" and ensure all options,
including "pip", are selected. In the "Advanced Options," choose "Install for all users" and set the
installation path to C:\Python. Proceed by clicking "Install".

Step 2: Install PIP

PIP generally comes installed with Python 3.4 and later. To confirm its installation, open Command
Prompt and execute:
pip -version
If pip is not installed or if we need to update it, we can use the following command to install or upgrade
pip:
python -m ensurepip --upgrade
After installation, we can verify that pip is installed correctly by running:

python -m pip -v ersio n

Step 3: Workspace Creation:
A dedicated directory for machine learning projects should be created for organizational clarity. This
can be set up using Command Prompt _____________________ _

C:\> mkdir C:\ML_Projects

C:\> cd C:\ML_Projects

Step 4: Creating a Virtual Environment

It is recommended to work in an isolated environment to manage dependencies more effectively
and avoid conflicts between projects. The virtual environment tool should be installed and a new
environment created:
C:\> pip install virtualenv # Install^irtualenv
C:\> virtualenv ml_env # Create a new virtual environment named ml_env
C:\> m l_ en v \ S crip ts\ a ctiv a te # Activate the v ir tu a l environment_________
While activated, any packages installed using pip will only affect this environment.
To exit the virtual environment, simply run: deactivate ^ __________ —
 ^.Appendix A : Lab Programs A.3

Step 5: installing Necessaiy Tools:

Essential Libraries: Libraries such as Iup3 ^er, NumPy, pandas, Matplotlib, and Scikit-
Leam should be installed if they are not already present These can be installed using
pip, which is Python's package manager. Open Command Prompt and enter the following
commands:
(ml_env) C:\> python -tn pip install --upgrade pip
(inl_env) C:\> pip install matplotlib numpy pandas scikit-learn

Program 2 introduce scikit-learn as a machine learning library.,

Scikit-learn is a populcir open-source machine learning library in P)rthon that offers a comprehensive set
of tools and algorithms for data analysis, modeling, and machine learning tasks. It is built on foundational
libraries like NumPy, SciPy, and Matplotlib. Scikit-learn provides a user-friendly and efficient framework for
both beginners and experts in the field of data science.
Some key points to introduce scikit-learn as a machine learning library;
1. Comprehensive Machine Learning Library: Scikit-learn offers a wide range of machine learning
algorithms and tools for various tasks such as classification, regression, clustering, dimensionality
reduction, and more.
2. User-Friendly and Easy to Use: It is designed with a user-friendly interface and simple syntax, making
it accessible for both beginners and experienced machine learning practitioners.
3. Integration witfi Scientific Computing Libraries: Scikit-learn integrates well with other scientific
computing libraries in Python such as NumPy, SciPy, and Matplotlib, providing a powerful environment
for machine learning tasks.
4. Extensive Documentation and Community Support The library comes with comprehensive
documentation, tutorials, and examples to help users understand and implement machine learning
algorithms effectively. Additionally, there is a vibrant community around scikit-learn that provides
support and contributions.
5. Efficient Implementation of Algorithms: Scikit-learn is built on top of NumPy, SciPy, and Cython,
which allows for efficient implementation of machine learning algorithms and scalability to large
datasets.
6. Support for Model Evaluation and Validation: The library provides tools for model evaluation,
h)rperparameter tuning, cross-validation, and performance metrics, enabling users to assess and
improve the quality of their machine learning models.
7. Flexibility and Customization: Scikit-learn offers flexibility for customization and parameter tuning,
allowing users to adapt algorithms to their specific requirements and datasets.
8. Wide Adoption and Industry Usage: Due to its ease of use, performance, and versatility, scikit-learn
is widely adopted in academia, research, and industry for various machine learning applications.
OveraU, scikit-learn is a powerful and versatile machine learning library in Python that empowers users to
1__-------------------- --------------------------------- —
Tncfaill and set up scikiUearn and o tter necessary tools/
Same as Program 1.
 Machine Learning

Program 4 Wnte a program t o Load [Link]^,tiie_dataset o f CSV and excel files using pandas.
Step 1: Creating CSV and Excel Files with Dummy Data
• Create CSV File: Open a text editor like Notepad or any other code editor. Enter the following data
Name, Age,Score
Srikanth,28,85
Snigdha,22,78
Mary,31,92
Save this file as sample_data.csv in the C:\ML_Projects directory.
• Create Excel File: We can use Microsoft Excel or Google Sheets to create this file. Enter the below data:

’'"Course. Sem
Rajesh BCA 1

Ramesh BCA 2

Swati BCOM 1

Fiorina BCOM 3
Pooja BBA 2

Raghu BBA 4
Save this file as sample_data.xlsx in the C:\ML_Projects directory.
Step 2: Python Code to Load and Explore the Data
import pandas as pd

# Define the file paths

csv_file_path = 'C:\\ML_Projects\\sample_data.csv'
excel_file_path = 'C:\\ML_Projects\\sample_data.xlsx'

# Load the CSV file

data_csv = pd.read_csv(csv_file_path)
printC'CSV File Data:")
print(data_csv)

# Load the Excel file

data_excel = pd.read_excel(excel_file_path)
print("\nExcel File Data:")
print(data_excel)

# Basic Data Exploration

print("\nData Descriptions:")
print("CSV Data Description:")
print(data_csv.describe())

print("\nExcel Data Description:")

print(data_excel.describe()) _______________
# Displaying data types
print("\nData Types in CSV File:")
p rint (dat a_c sV .dtypes)

print("\nData Types in Excel File:")

print (data_excel. dtypes)
Output
CSV File Data: Excel Data Description:
Name Age Score Sem
0 Srikanth 28 85 count 6.000000
1 Snigdha 22 78 mean 2.166667
2 Mary 31 92 std 1.169045
min 1.000000
Excel File Data: 25% 1.250000
Name Course Sem 50% 2.000000
0 Rajesh BCA 1 75% 2.750000
1 Ramesh BCA 2 max 4.000000
2 Swati BCOM 1
3 Fiorina BCOM 3 Data Types in CSV File:
4 Pooja BBA 2 Name object
5 Raghu BBA 4 Age int64
Score int64
Data Descriptions: dtype: object
CSV Data Description:
Age Score Data Types in Excel File
count 3.000000 3.0 Name object
mean 27.000000 85.0 Course object
std 4.582576 7.0 Sem int64
min 22.000000 78.0 dtype: object
25% 25.000000 81.5
50% 28.000000 85.0
75% 29.500000 88.5
max 31.000000 92.0
Explanation
Importing pandas: The script begins by importing the pandas library, which is essential for data
manipulation and analysis.
Loading Data: The pd.read_csvQ function is used to load data from the CSV file, and [Link] is
for loading data from the Excel file.
Printing Data: The script prints the data loaded from both files to ensure they are read correctly.
Data Exploration: The .describeQ method provides a statistical summary of the numerical columns,
which helps quickly assess data distribution, count, mean, std, min, max, and percentiles.
Data Types: The .dtypes attribute of the DataFrame is used to print the data types of each column,
helping to confirm data formats and identify any potential issues with type mismatches.
Program 5 1scatter plots, bar charts. _______ ________________ ------------------------------------------

^“ ' • ^ ' r “ a * w ir e ™ * b e ,o w d a « o f s .d e „ t s .d y h o u .a n d e x a .s c „ ..S a v e « s

CSV.
student I D ,S tu dy H o u r s , E x a m Score
1 ,5 ,8 2
2 ,2 ,4 8
3 ,8 ,9 0
4 ,1 ,3 5
5 ,3 ,5 0
6 ,4 ,6 6
7 ,9 ,9 5
8 ,6 ,7 5
9 ,7 ,8 8
1 0 .0 .5 .3 0
1 1 ,1 0 ,9 6
.
12 0 .2 0
1 3 ,1 2 ,9 8
Step 2: Python Code:
import pandas as pd
import [Link] as pit

It Load the data

data = pd.read_csvCC;\\ML_Projects\\study_data.csv )

# Scatter plot of Study Hours vs. Exam Scores

[Link](figsize=(14, 7)) ^ i 4-
1 +- c iih n ln -h M 9 1^ # 1 TOW, 2 COlumHS, Ist S U b p lO t
p it s « t t r 4 ; ; t a i . s U H o u r s ’ L d a ta ['E .a .S c ^ ^ ^ co lo ^ .d o d g .rb lu a ', ed g eco lo r- . .

alpha=0.7)
[Link]('Study Hours vs. Exam Scores')
[Link]('Study Hours')
[Link] Scores')
[Link](True)

# Bar chart of Average Exam Score by Study Hour Range

# Creating bins for study hour ranges
bins = [0 , 2 , 4 , 6 , 8 , 1 0 , 12] ^ ^
la b .ls .la .e ls .

e r o u p e S a ta = [Link] Hour Ranee')['Exam .Score'[Link] -------- ---- ---

[Link](l, 2j 2) # 1 rowj 2 columns^ 2nd subplot
groupecl_data.plot(kind='bar', color='salmon‘)
[Link] Exam Score by Study Hour Range')
[Link] Hour Range')
[Link]('Average Exam Score')
[Link](rotation=0) # Keep the category labels horizontal

plt.tight_layout() # Adjust subplots to fit into figure area.

[Link] ___________
Output
[Link]& Aw
tr»9«eiamSca»ctySfci<rHM
rKat*9«

I.

Explanation
1. Data Loading: The script uses pandas to load a CSV file containing students' study hours and exam
scores from a specified path.
2. Scatter Plot: Matplotlib is employed to create a scatter plot, plotting 'Study Hours' against 'Exam
Scores' to visually explore the relationship between these variables.
3. Bar Chart Setup: The data is categorized into bins based on study hours using [Link]. It then
calculates the average exam score for each category and uses this data to generate a bar chart showing
average scores by study hour range.
4. Visualization Configuration: Both plots are configured in a single figure, with clear titles and labels
for axes, using pltsubpIotQ to arrange them side by side for easy comparison.
5. Display: The script concludes with pltshowQ to display the configured plots, providing insights into
how stu(fy time correlates with academic performance through both detailed and aggregated views.

Write a program to Handle missing'data, encode categorical variables, and perform

feature scaling. , ^ ■ - — 3-
import pandas as pd
from [Link] import Simplelmputer
from [Link] import OneHotEncoder, StandardScaler

# Create du m y data
data = { ^___________________________
 ■Age': [25, 30, None, 28, 35],
■Gender': ['Female', 'Male', 'Male', 'Female', 'Male'],
'Income': [50000, 60000, 45000, None, 70000]
}
df = [Link](data)

# Handling missing data

imputer = Simplelmputer(strategy='mean')
df[['Age', 'Income']] = imputer.fit_transform(df[['Age', 'Income']])

# Print data after handling missing values

printC'Data after handling missing values:")
print(df)

# Encoding categorical variables

encoder = OneHotEncoderO
encoded_data = encoder.fit_transform(df [['Gender']]).toarray()

# Print data after categorical encoding

encoded_df = [Link](encoded_data, columns=encoder.get_f eature_names_
out(['Gender']))
print("\nData after categorical encoding:")
print(encoded_df)

# Feature scaling
scaler = StandardScaler()
scaled_data = scaler.fit_transform(df [['Age', 'Income']])

# Print data after feature scaling

scaled_df = [Link](scaled_data, columns=['Scaled Age', 'Scaled Income'])
print("\nData after feature scaling:")

Data after handling missing values:

Age Gender Income
0 25.0 Female 50000.0
1 30.0 Male 60000.0
2 29.5 Male 45000.0
3 28.0 Female 56250.0
4 35.0 Male 70000.0
 ndix A : Lab Programs
m

Data after categorical encoding:

Gender_Fetnale Gender_Male
® 1.0 0.0
1 0.0 1.0
2 0.0 1.0
3 1.0 0.0
4 0.0 1.0

Data after feature scaling:

Scaled Age Scaled Income
-1.382164 -0.727778
0.153574 0.436667
0.000000 -1.310001
-0.460721 0.000000
1.689312 1.601112
iExplanation
1. Data Preparation;
The code creates a dummy dataset with columns 'Age', 'Gender', and 'Income' containing numerical
and categorical data.
It uses the pandas library to create a DataFrame from the dummy data, which will be used for further
processing.
2. Handling Missing Data:
The code uses Simplelmputer from [Link] to fill missing values in the 'Age' and 'Income'
columns with the mean of each respective column.
This step ensures that the dataset is ready for further processing without missing values affecting the
analysis.
3. Categorical Encoding:
The code utilizes OneHotEncoder from [Link] to encode the categorical variable
'Gender' into a one-hot encoded format.
The [Link] method is applied to convert the categorical data into a numerical representation
suitable for machine learning algorithms.
4. Printing Data after Categorical Encoding:
After encoding the categorical variable 'Gender', the code creates a DataFrame encoded_df to display
the data in its one-hot encoded form.
The get_feature_names_out method is used to retrieve the feature names for the encoded columns
based on the original categorical variable.
5. Feature Scaling:
The code uses StandardScaler from [Link] to standardize the numerical columns 'Age'
and 'Income'.
Standardization ensures that all numerical attributes have a mean of 0 and a standard deviation of 1 ,
which can improve the performance of certain machine learning algorithms.
A.10 ^ Machine

Write a program to implement a k-Nearest Neighbours (k-N^ classifier using

scikillearn and Train the classifier on the dktaset and evaluate its performance.
import numpy as np
from sklearn.model_selection import train_test_split
from [Link] import KNeighborsClassifier
from [Link] import accuracy_score

# Dummy student data: exam score 1, exam score 2, pass/fail (features)

X = [Link]([[80, 75], [95, 90], [60, 50], [45, 38], [30, 40], [85, 95], [70, 60], [50, 55],
[40, 45], [60, 70]])
y = [Link]([1, 1, 0, 0, 0, 1, 1, 0, 1]) # Binary classes fordemonstration

# Split the data into training and testing sets

X_train, X_test, y_train, y_test = train_test_split(X, y,test_size=0.2, random_state=42)

# Initialize the k-NN classifier with k=3

knn = KNeighborsClassifier(n_neighbors=3)

# Train the classifier on the training data

[Link](X_train, y_train)

# Evaluate the classifier's performance

y_pred = [Link](X_test)
accuracy = accuracy_score(y_test, y_pred)
print("Accuracy on the test set: {: .2f}".format(accuracy))

# Take user input for exam scores

exam_scorel = float(input("Enter Exam Score 1: "))
exam_score2 = float(input("Enter Exam Score 2: "))

# Prepare the user input for prediction

user_input = [Link]([[exam_scorel, exam_score2]])

# Use the trained k-NN classifier to predict the outcome

predicted_outcome = knn. predict (user_input)

if predicted_outcome[0] = = 1 :
printC'Based on the exam scores provided, the student is predicted to pass. )

else: .
printC'Based on the exam scores provided, the student is predicted to fail. )______^_______
 A ; Lab Programs ^ a .II

O utput
Output 1:
Accuracy on the test set: 1.00
Enter Exam Score 1: 45
Enter Exam Score 2: 50
Based on the exam scores provided, the student is predicted to fail.
Output 2:
Accuracy on the test set: 1.00
Enter Exam Score 1: 75
Enter Exam Score 2: 89
Based on the exam scores provided, the student is predicted to pass.
i'Ezplanation
1. Data Preparation:
The code initializes a numpy array X with exam scores as features and y with binary pass/fail labels.
It represents a simple dataset where each row corresponds to a student's exam scores and pass/fail
outcome.
2. Model Training and Evaluation:
It splits the data into training and testing sets using train_test_split with a test size of 20 % and a
random seed for reproducibility.
The code initializes a K-Nearest Neighbors (KNN) classifier with n_neighbors=3 and trains it on the
training data (X_train, y_train).
The model's performance is evaluated by predicting on the test set [X_test) and calculating the accuracy
using accuracy_score.
3. User Interaction:
The code prompts the user to input exam scores for a new student using inputQ function.
It prepares the user input as a numpy array userjnput to make a prediction using the trained KNN
classifier.
4. Prediction and Output:
The code predicts the outcome (pass/fail) for the new student based on the input exam scores using
the trained KNN classifier.
It then prints a message indicating whether the student is predicted to pass or fail based on the model's
prediction.

Write a\pro^am; to implement a linear riegression model Yor^ej^ession tasks and

Train the model on a dataset with continuous target variable^!'-' ■‘i-'' ' ' ’
import numpy as np
from sklearn.linear_model import LinearRegression

# Dummy house price prediction data: features (house size, number of bedrooms) and
target variable (house price)
X = [Link]([[1000, 2], [1500, 3], [1200, 2], [1800, 4], [900, 2], [2000, 3]])
y = [Link]([300000, 400000, 350000, 500000, 280000, 450000])
A .1 2 k ;

# Initialize the Linear Regression model

model = LinearRegressionO

# Train the model on the dataset

[Link](X, y)

# Take input from the user for new house data

size = float(input ("Enter the size of the house in sqft: "))
bedrooms = int(input("Enter the number of bedrooms: "))
new_data = [Link]([[size, bedrooms]])

# Predict the price for the new house data

predicted_price = model. predict(new_data)

# Print the predicted price for the new house data

print("Predicted price for a house with size {} sqft and {} bedrooms: Rs.{:.2f} .
format (size, bedrooms, predicted_price[0]))______ __________________________ ___
Output
Enter the size of the house in sqft: 1600
Enter the number of bedrooms: 3
Predicted price for a house with size 1600.0 sqft and 3 bedrooms: Rs.418163.93
Explanation
1. Model Training and Prediction:
The code initializes a Linear Regression model and trains it on the dummy house price prediction data
provided in the arrays X (features - house size and number of bedrooms) and y (target variable - house
price).
After training the model, it takes input from the user for new house data (size and number of bedrooms)
to predict the price for a new house.
The model then predicts the price for the new house data using the predictQ method and stores the
result in predicted_price.
2. User Input and Output;
The code prompts the user to enter the size of the house in square feet and the number of bedrooms
using the inputQ function.
It converts the user input into a NumPy array new_data to match the format expected by the model for
prediction.
Finally, it prints the predicted price for the new house data input by the user in the format "Predicted
price for a house with size [size] sqft and [bedrooms] bedrooms: Rs.[predicted_price]"._____________
 •Appendix A : Lab Programs^^ a .131

Write a program to implement a decision, tree classifier using scikit-learn and visualize
the decision tree and understand its splits. ’ . . -. -
import numpy as np
from [Link] import DecisionTreeClassifier, plot_tree
from [Link] import export_text
import [Link] as pit

# Custom dummy data for fruit classification

# Features: [Weight, Texture] -> Target: [Fruit Type]
X = [Link]([[150, 0], [170, 1], [120, 0], [140, 1], [200, 1], [130, 0]])
y = [Link](['Apple', 'Orange', 'Apple', 'Orange', 'Melon', 'Apple'])

# Initialize the Decision Tree Classifier

clf = DecisionTreeClassifier(random_state=42)
[Link](X, y)

# Visualize the Decision Tree splits

tree_rules = export_text(clf, feature_names=['Weight', 'Texture'])
print("Decision Tree Classifier Rules:\n", tree_rules)

# Plot the Decision Tree

[Link](figsize=(10, 6))
plot_tree(clf, fiIled=True, feature_names=['Weight', 'Texture'], class_names=[Link](y))
[Link]

Decision Tree Classifier Rules:

I— Texture <= 0.50
|--- class: Apple
--- Texture > 0.50
I--- Weight <= 185.00
I 1— class: Orange
I--- Weight > 185.00
I I— class: Melon
 l U i rnliigl"?:;;

1. Data Preparation:
The code defines a custom dummy dataset for fruit classification with features representing Weight
and 'Texture' of fruits and the target variable 'Fruit Type' (e.g., Apple, Orange, Melon).
The features and target labels are stored in NumPy arrays X and y, respectively.
2. Decision Tree Classifier Initialization:
A Decision Tree Classifier is initialized with random_state=42 to ensure reproducibility of results.
The classifier is then trained on the custom dummy dataset using the fitQ method.
3. Visualization of Decision Tre« Splits:
The export_text function is used to generate text-based rules of the Decision Tree Classifier based on
the features provided.
These rules provide insights into how the Decision Tree makes splits based on the Weight and
'Texture' features to classify different types of fruits.
4. Plotting the Decision Tree;
The [Link] function is utilized to visualize the Decision Tree structure graphically.
The Decision Tree is displayed with filled nodes, and the feature names ('Weight', 'Texture') and class
names (unique fruit types) are specified for better interpretation.
5. Displaying the Decision Tree Visualization:
A Matplotlib figure is created witfi a specific size to accommodate the Decision Tree plot
The Decision Tree visualization is shown using plt-showQ, allowing us to observe the tree structure
and decision-making process visually._____________ _______________________________________

Program 10 Write a program to Implement K-Means clustering and Visualize clusters.

import numpy as np
import [Link] as pit
from [Link] import KMeans

# Generate dummy customer data (Age, Income)

X = n p .a rra y ([[3 0 , 5 0 0 0 0 ], [35, 6 0 0 0 0 ], [4 0 , 80000], [25, 30000], [4 5 , 100000],
[2 0 , 2 0 0 0 0 ], [50, 12 0 0 0 0 ], [ 5 5 , 150000], [60, 14 0 0 0 0 ], [28, 40 0 0 0 ]])

# I n it i a l iz e K-Means with 2 c lu ste rs

kmeans = KMeans(n_clusters=3, random_state=0)
[Link](X)

# Get cluster labels and cluster centers

labels = kmeans.labels_
centers = kmeans. cluster_centers_ ______________________________ _______
 ''Appendix A : Lab'Pn^his'

# Visualize the clusters

[Link](figsize=(8, 6))
[Link](X[:, 0], X[:, 1], c=labels, cmap='viridis', s=50, alpha=0.8)
[Link](centers[:, 0], centers[:, 1], c=’red', s=200, marker='X', label='Centroids')
[Link]('Age')
[Link]('Income’)
[Link]('K-Means Clustering of Customers')
[Link]()
[Link]

K-Means Clustering of Custnmefs

Explanation
1. Dummy Data Generation:
The program generates dummy customer data with features representing 'Age' and 'Income' of
customers.
K-Means Clustering:
K-Means clustering is applied to the customer data with n_clusters=3 to create 3 clusters.
The algorithm assigns each data point to one of the clusters based on the similarity of features.
2. Visualization;
The clusters are visualized using a scatter plot where each point represents a customer.
Different clusters are distinguished by colors, and cluster centers (centroids) are marked in red.
3. Plot Interpretation:
The plot helps visualize how customers are grouped into clusters based on their 'Age' and 'Income'.
Centroids represent the center of each cluster, showing the average 'Age' and 'Income' values for
customers in that cluster.
 .a : - ; * , ; , ; . , . ■;

l^ O ^ E M V E S T X ^ p A P E R S

Model Question Paper - 1

T im e : 2 % Hours
Max. M a rk s : 60
I n s tr u c tio n s : Answer All Sections

S ection -*/!
I. Answer any Four questions. Each question carries Two morks ( 4 X 2 = 8)
1. W h at is Machine Learning? Give an example.
2. W h at is Scikit-Ieam ?
3. W h at is Labeled Data and Unlabeled Data? Give an exam ple.
4. W h at is Classification? Give an example.
5. W h at is Clustering?
6. WhatisDBSCAN?

s e c tio n s

II. Answer any Four question. Each question carries Five marks 4 X 5 = 20)
7. W hy Use Machine Learning ?
8. W rite the Applications of Machine Learning.
9. W h at is Feature Engineering? Explain the Key Components of Feature Engineering.
10. H ow Naive Bayes Classifier works?
11. How K-Means Clustering Works? Write an algorithm.
12. W rite a Python Code to Demonstrate K-Means Clustering.

I Section~C
III. Answer any Four questions. Each question carries Eight marks 4 X 8 = 32)
13. Explain the Types of Machine Learning.
14. Explain the Essential Libraries and Tools required for M achine Learning Projects.
15. a) Discuss the Sources of Real-World Data.
b) Explain the Process of Selecting and Training a M achine Learning Model
16. Explain How to Discover and Visulaize the Data to Gain Insights in Data Preparation.
17. a) W rite a Python Code to Demonstrate Classification Tasks using CART,
b) W rite the applications of K-Means Clustering.
18. a ) How Clustering is Used In Semi-Supervised Learning?
b) W rite a note on a] Mean-Shift b) Affinity Propagation
 . -

Time: 2 % Hours Max. M a r k s : 6 0

Instructions: Answer All Sections

Section-;^
I. Answer any Four questions. Each question carries Two marks (4X2 = 8 )

1. W hat is Supervised Machine Learning? Give an example.

2. Why P)rthon is used for Machine Learning?
3. What is Data Preparation ?
4. W hat is Regression? Give an exam ple
5. W hat is Discrete Output Variable? Give an example.
6. Mention the two kinds of Unsupervised Learning

II. Answer any Four question. Each question carries Five marks 4 X 5 = 20)

7. W hat is Unsupervised Machine Learning? Explain the Key Components of Unsupervised

Machine Learning.
8. What is SciPy? Why it is needed for ML? Explain its features.
9. How to Handle Missing Values and Ouliers? Explain with an example.
10. Explain the Process of Getting th e Data.
11. Explain the Differences betw een Regression and Classification.
12. Explain the Limitations of K-Means Clustering.

SecUon-C
III. Answer any Four questions. Each question carries Eight marks { 4 X 8 = 32)

13. Explain the Main Challenges o f Machine Learning

14. How Semi-Supervised Machine Learning Works? Explain with an example.
15. a) How to Create a Test Set?
b) Why Data Reduction is Im portant in ML?
16. a) W hat is Logistic Regression? Explain how it works?
b) Write a Python Code for Spam Email Detection using the Naive Bayes classification algorithm.
17. a) Write Decision Tree Algorithm and explain how it works?
b) Explain how a cluster form ed in DBSCAN clustering algorithm?
18. a) Explain the types of Clustering Methods or Techniques.
b) Write a Python Code to Use Clustering in Semi-Supervised Learning.

I
 Q u estio n Pap ers b.3

lodel Qufesfion Paper - 3

Time: 2 % Hours M ax. M ark s: 60
Instructions: Answer All S ections

Section-1^

I. Answer any Four questions. Each question carries Two marks (4X2 = 8)

1. What is Reinforcement Learning? Give an example.

2. Write any two applications of Supervised Machine Learning.
3. What is Data Transformation?
4. What is Unear Regression?
5. What is Bayes'Theorem ?
6. What are Core Points, B ord er Points, and Noise Points in DBSCAN

Sectiofi-'B
II. Answer any Four question. Each question carries Five marks { 4 X 5 = 20)

7. Explain the Differences between Supervised and Unsupervised Learning.

8. Why Python is Preferred Choice for Machine Learning Applications ?
9. What is Data Spitting? Explain Common Types and Methods of Data Splits.
10. Write Decision Tree Algorithm and explain how it works?
11. Explain different A ttribute Selection Measures (ASM) used in Classification.
12. Explain the Features o f Machine Learning.

E m i
III. Answer any Four questions. Each question carries Eight marks ( 4 X 8 = 32)

13. Explain tiie Machine learning Life Cycle.

14. How Unsupervised M achine Learning Works? Explain with an example.
15. Explain the Steps in D ata Preparation Process.
16. a) Write a Python Code for Classification Task Using KNN Classifier,
b) Write the Applications of Naive Bayes Classifiers
17. a) Mention the Advantages and Disadvantages of Linear Models.
18. a] Write the Applications of DBSCAN.
b) Write a note on a ) Mean-Shift b) Affinity Propagation
 -Machine Learning'
1
Model Question Paper - 4
Max. M a rk s : 6 0
T im e : 2 % H ours
In stru ction s : Answer All Sections

(4X2 = 8 )
I. Answer any Four questions. Each question carries Two marks

1. Mention the Real Life Examples of Machine Learning.

2. Mention the types of Supervised Machine Learning
3. W hat is Dimensionality Reduction?
4. W hat is Decision Tree Algorithm?
5. W hat is Logistic Regression?
6. W hat is Image Segmentation ?

4 X 5 = 20)
II. Answer any Four question. Each question carries Five marks

7. How Supervised Machine Learning Works? Explain with an example.

8. W hat is Scikit-learn ? Explain its features.
9. Why Data Transformation is Important in ML? Explain th e Common Methods of Data
Transformation.
10. How Naive Bayes Classifier works? Explain with an Example.
11. W hat is CART( Classification and Regression Tree)? How it w orks?
12. W rite a Python Code to Detect Anomalies Using Clustering.

SecUofi-C
( 4 X 8 = 32)
III. Answer any Four questions. Each question carries Eight marks

13. Explain the types of Supervised and Unsupervised Machine Learning Algorithms.
14. W hat are NumPy and Pandas ? Why it is needed for ML? Explain its features.
15. a) Why Visualizing the Data is Needed During Data Preparation?
b) How to Load the Data and Explore the Data in ML?
16. How K-Nearest Neighbors (K-NN) Works ? Explain with an example for both classification and
regression tasks
17. a] W rite the Advantages and Disadvantages of Decision Tree Based Algorithms
b] How Clustering is used in Preprocessing?
18. Explain K-Means Clustering for Image Segmentation. W rite an algorithm.
1