S.V.

GROUP OF INSTITUTIONS
North Rajupalem, Kodavaluru(V&M) , S.P.S.R Nellore (Dt)-524316

For UG ([Link])

Long Answer (10Marks) Question Bank

Name of the College: SVCN College Code: KN&JN
Subject: Machine Learning Subj. Code: 23A31401T
Branch: CSE Year& Sem: III-II
Faculty Name: [Link] & [Link] Department: CSE
***************************************************************************************
Unit No: I(Unit Name: Introduction to Machine Learning) M BL COs
1. Explain the evolution of Machine Learning. 10 2 CO1

Answer:

Machine Learning (ML) is a rapidly evolving branch of Artificial Intelligence (AI) that
focuses on developing systems capable of learning from data and improving their performance
without being explicitly programmed. The evolution of Machine Learning can be understood through
different historical phases, driven by advances in theory, data availability, and computational power.

1. Early Foundations (1940s–1950s)

The roots of Machine Learning lie in early research on artificial intelligence and statistics.

 In 1943, McCulloch and Pitts proposed the first mathematical model of a neuron, forming the
basis of neural networks.
 In 1950, Alan Turing proposed the Turing Test, raising the idea of machine intelligence.
 Early learning systems were simple and relied on logical reasoning and handcrafted rules.

This period established the theoretical foundations for learning machines.

2. Rule-Based and Symbolic Learning (1960s–1970s)

During this phase, learning was primarily symbolic.

 Knowledge was stored as rules such as IF–THEN statements.

 Expert systems like DENDRAL and MYCIN were developed.
 Learning occurred by manually encoding expert knowledge rather than automatic learning
from data.

Although effective in limited domains, these systems lacked flexibility and were difficult to scale.

3. Emergence of Statistical and Inductive Learning (1980s–1990s)

This era marked a shift from symbolic AI to data-driven approaches.

 Decision trees, nearest neighbor methods, and Bayesian learning became popular.
 Learning by induction allowed systems to generalize rules from examples.
 Artificial Neural Networks (ANNs) re-emerged with the development of backpropagation.
Statistical learning helped ML models handle uncertainty and noise in real-world data.

4. Machine Learning with Improved Algorithms (2000s)

With growing datasets and stronger computers:

 Algorithms such as Support Vector Machines (SVMs), Random Forests, and Boosting
techniques were developed.
 Model selection and evaluation approaches became systematic.
 ML started being used in practical applications like spam detection and recommendation
systems.

This phase emphasized performance optimization and generalization.

5. Big Data and Scalable Machine Learning (Late 2000s)

The rise of the internet led to massive data generation.

 ML systems began handling large-scale, high-dimensional datasets.

 Distributed computing frameworks such as Hadoop and MapReduce supported large ML
workloads.
 Feature engineering gained importance.

ML evolved into an essential tool for data analytics.

6. Deep Learning Era (2010–Present)

This marks the most significant milestone in ML evolution.

 Deep Neural Networks with multiple layers achieved breakthroughs in image recognition,
speech processing, and natural language processing.
 Success was driven by:
o Availability of big data
o GPUs and TPUs for computation
o Advanced architectures like CNNs and RNNs
 Applications include self-driving cars, voice assistants, and medical diagnosis.

Deep learning minimized manual feature engineering by learning representations automatically.

7. Modern Trends and Future Directions

 Reinforcement learning has achieved human-level performance in games like Go.

 AutoML aims to automate model selection and tuning.
 Ethical AI and explainable ML are becoming necessary.
2. Describe the different paradigms of Machine Learning. 10 2
CO1

Answer:

Machine Learning (ML) is a subfield of Artificial Intelligence (AI) that focuses on building
algorithms and models that enable computers to learn from data and improve with experience
without explicit programming for every task. In simple words, Machine Learning teaches systems
to learn patterns and make decisions like humans by analyzing and learning from data.
There are several types of machine learning, each with special characteristics and applications.
Some of the main types of machine learning algorithms are as follows:
1. Supervised Machine Learning
2. Unsupervised Machine Learning
3. Reinforcement Learning
Additionally, there is a more specific category called Semi-Supervised Learning and Self-
Supervised Learning, which combines elements of both supervised and unsupervised learning

1. Supervised Machine Learning

Supervised learning is defined as when a model gets trained on a "Labeled Dataset". Labelled
datasets have both input and output parameters. In Supervised Learning algorithms learn to map
points between inputs and correct outputs. It has both training and validation datasets labelled.
Example: If you train a model using labeled images of cats and dogs, it learns the features of each.
When shown a new image, it predicts whether it’s a cat or a dog.
There are two main categories of supervised learning that are mentioned below:
1. Classification
Classification predicts categorical outputs, meaning it assigns data into predefined classes like
spam/non-spam emails or disease risk categories. These algorithms learn to map input features to
discrete labels.
2. Regression
Regression, predicts continuous values, such as house prices or product sales. It learns the
relationship between input features and a numerical target variable. Here are some regression
algorithms:
Where to Use Supervised Learning
 When you have labeled data and want to predict outcomes.
 Ideal for classification (like spam detection) or regression tasks (like price forecasting).
 Best used in domains where historical data with outcomes is already available.
Applications
Supervised learning is used in a wide variety of applications, including:
 Image, speech and text processing: For tasks like image classification, speech recognition and
sentiment analysis.
 Predictive analytics: To forecast sales, customer churn, stock prices and weather conditions.
 Recommendation and personalization: Powering systems that suggest products, movies or
content.
 Healthcare and finance: Used for medical diagnosis, fraud detection and credit scoring.
 Automation and control: In autonomous vehicles, manufacturing quality checks and gaming
AI.
2. Unsupervised Machine Learning
Unsupervised Learning works with unlabeled data, meaning there are no predefined outputs. The
algorithm finds hidden patterns, groups or relationships within the data on its own. It’s mainly used
for clustering, dimensionality reduction and data visualization.
Example: If you have customer data without labels, the algorithm can group similar customers
based on purchase behavior useful for segmentation and marketing.
There are two main categories of unsupervised learning that are mentioned below:
1. Clustering
Clustering is the process of grouping data points into clusters based on their similarity. This
technique is useful for identifying patterns and relationships in data without the need for labeled
examples
2. Dimensionality Reduction Techniques
Dimensionality reduction helps reduce the number of features while preserving important
information. Common techniques include:
3. Association Rule Learning
Association rule learn ing is a technique for discovering relationships between items in a dataset. It
identifies rules that indicate the presence of one item implies the presence of another item with a
specific probability.
Where to Use Unsupervised Learning
 When data is unlabeled or unstructured.
 Useful for exploratory analysis, clustering or feature extraction.
 Common in marketing, recommendation systems and fraud detection where patterns matter
more than labels.
Applications of Unsupervised Learning
Here are some common applications of unsupervised learning:
 Clustering and segmentation: Group similar data points, customers or images.
 Anomaly detection: Spot unusual patterns or outliers in data.
 Dimensionality reduction: Simplify large datasets while retaining key information.
 Recommendation and marketing: Identify user preferences and improve product suggestions.
 Data preprocessing and analysis: Clean data, detect patterns and support exploratory data
analysis (EDA).
3. Reinforcement Learning
Reinforcement learning trains an agent to make a sequence of decisions through trial and error. The
agent interacts with the environment, receives feedback in the form of rewards or penalties and
learns optimal actions over time.
Example: An AI agent learning to play chess gets positive feedback for good moves and
negative for poor ones. Over time, it learns strategies to win more often.

Types of Reinforcement Learning

 Positive Reinforcement: Rewards desired behavior (e.g., giving points for correct answers).
 Negative Reinforcement: Removes negative outcomes to encourage good actions (e.g., turning
off a buzzer after the right move).
Where to Use Reinforcement Learning
 When you need an agent to learn by interacting with an environment.
 Best for decision-making or optimization tasks involving trial and feedback loops.
 Used when long-term performance or adaptive behavior is more important than immediate
accuracy.
Applications of Reinforcement Learning
Here are some applications of reinforcement learning:
 Gaming and simulation: Teaching agents or NPCs to play and adapt intelligently.
 Robotics and automation: Enabling robots to perform tasks autonomously.
 Autonomous vehicles: Helping self-driving cars make real-time decisions.
 Healthcare and finance: Optimizing treatment plans, trading and resource allocation.
 Recommendation and personalization: Improving user experience through adaptive
suggestions.
 Industrial and energy management: Optimizing control systems and energy use.
Semi-Supervised Learning: Supervised + Unsupervised Learning
Semi-Supervised learning Semi-Supervised Learning combines both Supervised and Unsupervised
approaches. It uses a small set of labeled data and a large set of unlabeled data for training useful
when labeling is costly or time-consuming.
Example: Consider that we are building a language translation model, having labeled translations
for every sentence pair can be resources intensive. It allows the models to learn from labeled and
unlabeled sentence pairs, making them more accurate. This technique has led to significant
improvements in the quality of machine translation services.
Self-Supervised Learning
Self-Supervised Learning (SSL) is a modern approach where models generate their own labels
from raw data. It doesn’t rely on manual annotation instead, the model learns by predicting parts of
data from other parts.
Example: In NLP, models like BERT or GPT learn by predicting masked words in sentences,
using surrounding context as supervision. This helps them learn language understanding without
human labeling.

3. Explain Learning by Rote and Learning by Induction with examples. 10 2

CO1

Answer:
 Learning mechanisms determine how a machine acquires, stores, and uses knowledge.
Learning by Rote and Learning by Induction are two fundamental approaches in Machine
Learning and Artificial Intelligence. They differ in their ability to generalize, adapt, and reason from
data.

Learning by Rote

Learning by Rote is the simplest form of learning in which a system memorizes information exactly
as it is presented. The system does not analyze, generalize, or infer new knowledge from the stored
data.

 Input–output pairs are stored directly in memory.

 When a new input is given, the system searches for an exact match.
 If found, it returns the stored output.
 If not found, the system fails to respond.

Characteristics

 No reasoning or understanding
 No generalization
 High memory usage
 Very fast retrieval

Example (Machine Learning Context)

Consider a system that memorizes multiplication facts:

Input Output

3 × 4 12

5 × 6 30

If asked 3 × 4, the system answers correctly.

If asked 3 × 5, it fails because the pair is not stored.

Real-Life Analogy

A student memorizing answers without understanding concepts. The student can recall exact answers
but cannot solve new problems.

Advantages

 Simple to implement
 Accurate for known cases
 No training process required

Limitations

 Cannot handle unseen data

 Not flexible or scalable
 Inefficient memory usage
2. Learning by Induction

Learning by Induction is the process of learning general rules or patterns from specific examples.
The system can apply learned rules to new, unseen data.

1. Collect training examples

2. Analyze patterns and relationships
3. Generate a general hypothesis or rule
4. Use that rule to make predictions

Characteristics

 Learns from examples

 Supports generalization
 Handles unseen cases
 Requires training process

Example (Machine Learning Context)

Consider training data for weather prediction:

Outlook Temperature Wind Play?

Sunny Hot Weak No

Rainy Mild Strong No

Sunny Mild Weak Yes

Inductive rule learned:

If outlook is Sunny and wind is Weak, then Play = Yes

This rule can be applied to new days not present in training data.

Real-Life Analogy

A child learning grammar rules from examples instead of memorizing sentences.

Advantages

 Capable of generalization
 Efficient for large data
 Adaptable to new situations

Limitations

 May generate incorrect rules

 Requires good quality data
 Computationally intensive

4. Explain Reinforcement Learning in detail. 10 2 CO1

Answer:

Reinforcement learning is a machine learning approach where an agent (software entity) is

trained to interpret the environment by performing actions and monitoring the results. For every good
action, the agent gets positive feedback and for every bad action the agent gets negative feedback. It's
inspired by how animals learn from their experiences, making decisions based on the consequences
of their actions.

The following diagram shows a typical reinforcement learning model −

Reinforcement Machine Learning

In the above diagram, the agent is represented in a particular state. The agent takes action in an
environment to achieve a particular task. As a result of the performed task, the agent receives
feedback as a reward or punishment.

How Does Reinforcement Learning Work?

In reinforcement learning, there would be an agent that we want to train over a period of time so that
it can interact with a specific environment. The agent will follow a set of strategies for interacting
with the environment and then after observing the environment it will take actions regarding the
current state of the environment. The agent learns how to make decisions by receiving rewards or
penalties based on its actions.

The working of reinforcement learning can be understood by the approach of a master chess player.

Exploration − Just like how a chess play considers various possible move and their outcome, the
agent also explores different actions to understand their effects and learns which action would lead to
better result.

Exploitation − The chess player also uses intuition, based on past experiences to make decisions that
seem right. Similarly, the agent uses knowledge gained from previous experiences to make best
choices.

Key Elements Reinforcement Learning

Beyond the agent and the environment, one can identify four main sub elements of reinforcement
learning system −

Policy − It defines the learning agent's way of behaving at a given time. A policy is a mapping from
perceived states of the environment to actions to be taken when in those states.

Reward Signal − It defines the goal of a reinforcement learning problem. It is a numerical score
received to the agent by the environment. This reward signal defines what are the good and bad
events for the agent.
Value function − It specifies what is good in the long run. The value is the total amount of reward an
agent can expect to accumulate over the future, starting from that state.

Model − Models are used for planning, which means deciding on a course of action by considering
possible future situations before they are actually experienced.

Markov Decision Processes(MDP) provide a mathematical framework for modeling decision-

making in an environment with states, actions, rewards, probability. Reinforcement learning uses
MDP to understand how an agent should act to maximize rewards and to find the best strategies for
decision making.

Markov Decision Processes (MDP)

Reinforcement learning uses the mathematical framework of Markov decision processes(MDP) to

define the interaction between learning agent and environment. Some important concepts and
components of MDP are −

States(S) − Represents all the situations in which an agent can find itself.

Action(A) − The choices available for the agent from the gives states.

Transition Probabilities(P) − The likelihood of moving from one state to another as a result of a
specific action.

Rewards(R) − Feedback received after transitioning to a new state due to an action, indication the
outcome's desirability.

Policy( ) − A strategy that defines the action to take in each state for achieving a reward.

Steps in Reinforcement Learning Process

Here are the major steps involved in reinforcement learning methods −

Step 1 − First, we need to prepare an agent with some initial set of strategies.

Step 2 − Then observe the environment and its current state.

Step 3 − Next, select the optimal policy regards the current state of the environment and perform
important action.

Step 4 − Now, the agent can get corresponding reward or penalty as per accordance with the action
taken by it in previous step.

Step 5 − Now, we can update the strategies if it is required so.

Step 6 − At last, repeat steps 2-5 until the agent got to learn & adopt the optimal policies.

Types of Reinforcement Learning

There are two types of Reinforcement learning:

Positive Reinforcement − When an agent performs an action that is desirable or leads to a good out,
it receives a rewards which increase the livelihood of that action being repeated.
Negative Reinforcement − When an agent performs an action to avoid a negative outcome, the
negative stimulus is removed. For example, if a robot is programmed to avoid an obstacle and
successfully navigates away from it, the threat associated with action is removed. And the robot more
likely avoids that action in the future.

Types of Reinforcement Learning Algorithms

There are various algorithms used in reinforcement learning such as Q-learning, policy gradient
methods, Monte Carlo method and many more. All these algorithms can be classified into two broad
categories −

Model-free Reinforcement Learning − It is a category of reinforcement learning algorithms that

learns to make decisions by interacting with the environment directly, without creating a model of the
environment's dynamics. The agent performs different actions multiple times to learn the outcomes
and creates a strategy (policy) that optimizes its reward points. This is ideal for changing, large or
complex environments.

Model-based Reinforcement Learning − This category of reinforcement learning algorithms

involves creating a model of the environment's dynamics to make decisions and improve
performance. This model is ideal when the environment is static, and well-defined, where real-world
environment testing is difficult.

Advantages of Reinforcement Learning

Some of the advantages of reinforcement learning are −

Reinforcement learning doesn't require pre-defined instructions and human intervention.

Reinforcement learning model can adapt to wide range of environments including static and
dynamic.

Reinforcement learning can be used to solve wide range of problems, including decision making,
prediction and optimization.

Reinforcement learning model gets better as it gains experience and fine-tunes.

Disadvantages of Reinforcement Learning

Some of the disadvantages of reinforcement learning are −

Reinforcement learning depends on the quality of the reward function, if it is poorly designed, the
model can never get better with its performance.

The designing and tuning of reinforcement learning can be complex and requires expertise.

Applications of Reinforcement Learning

Reinforcement learning has a wide range of applications across various fields. Some major
applications are −

1. Robotics

Reinforcement learning is generally concerned with decision-making in unpredictable environments.

This is the most used approach especially for complicated tasks, such as replicating human behavior,
manipulation, navigation and locomotion. This approach also allows robots to adapt to new
environments through trial and error.

2. Natural Language Processing (NLP)

In Natural Language Processing (NLP), Reinforcement learning is used to enhance the performance
of chatbots by managing complex dialogues and improving user interactions. Additionally, this
learning approach is also used to train models for tasks like summarizations.

5. What are the types of data used in Machine Learning? Explain. 10 2 CO1

Answer:

Data in machine learning are broadly categorized into two types − numerical (quantitative) and
categorical (qualitative) data. The numerical data can be measured, counted or given a numerical
value, for example, age, height, income, etc. The categorical data is non-numeric data that can be
arranged in categories with or without meaningful order, for example, gender, blood group, etc.

Further, the numerical data can be categorized into discrete and continuous data. The categorical data
can also be categorized into two types − nominal and ordinal. Let's understand these types of data in
machine learning in detail.

Types of Data in Machine Learning

Data in machine learning is a set of observations or measurement that are used to train, validate and
test a machine learning model. Data is very crucial in machine learning because it is the foundation
of creating accurate machine learning model.

What are Types of Data?

The data used in machine learning can be broadly categorized into two types −

Numerical (quantitative) data

Categorical (qualitative) data

Numerical (Quantitative) Data

The numerical (quantitative) data is data that can be measured, counted or given a numerical value.
The examples of numerical data are age, height, income, number of students in class, number of
books in a shelf, shoe size, etc.

The numerical data can be categorized into the folloiwng two types −

Discrete Data
Continuous Data

1. Discrete Data

The discrete data is numerical data that is countable, finite, and can only take certain values, usually
whole numbers. Examples of discrete data are number of students in class, number of books in a
shelf, shoe size, number of ducks in a pond, etc.

2. Continuous Data

The continuous data is numerical data that can take any value within a specified range including
fractions and decimals. Examples of continuous data are age, height, weight, income, time,
temperature, etc.

What is true zero?

True zero represents the absence of the quantity being measured. For example, height, weight, age,
temperature in Kelvin are examples of data with true zero. As the height with 0 CM represents the
absolute absence of height, 0K temperature represents no heat. But temperature in Celsius (or
Fahrenheit) is an example of data with false zero.

We can categorize the numerical data into the following two types on basis of true zero −

interval data − quantitative data with equal intervals between data points. Examples are temperature
(Fahrenheit), temperature (Celsius), pH, SAT score (200-800), credit score (300-850), etc.

ratio data − same as interval data but with true zero. Examples are weight in KG, number of students,
income, speed, etc.

Categorical (Qualitative) Data

The categorical (qualitative) data can be categorized with or without a meaningful order. For
example, gender, blood group, hair color, nationality, the school grades, level of education, range of
income, ratings, etc.

The categorical data can be divided into the folloiwng two types −

Nominal Data

Ordinal Data

1. Nominal Data

The nominal data is categorical data that can not be arranged in an order or rank. The examples of
nominal data are gender, blood group, hair color, nationality, etc.

2. Ordinal Data

The ordinal data is categorical data can be ordered or ranked with a specific attribute. The examples
of ordinal data are the school grades, level of education, range of income, ratings, etc.

The Four Levels of Data Measurement

 We can categorized data into four level − nominal, ordinal, interval, and ratio. These levels of
measurement are divided on basis of the following four features −

Categories − data can be categorized but not in an order.

Rank Order − data can be categorized with some meaningful order.

Equal Difference − The difference between subsequent data remains same.

True Zero − it represents the absence of quantity being measured.

6. Explain Pattern Matching and its role in Machine Learning. 10 2

CO1

Answer:

Pattern matching identifies specific patterns, sequences, or structures within larger datasets,
often using algorithms to check for exact or approximate matches. In computer science, it scans
sequences like text or tokens for predefined constituents, enabling tasks such as search, validation,
and substitution. This process differs from broader pattern recognition by requiring precise
alignment rather than fuzzy inference

How Pattern Matching Works in ML

The process involves several key stages:

1. Data Collection and Preprocessing: Gathering raw data (images, text, etc.) and cleaning it to
remove noise, errors, and inconsistencies.

2. Feature Extraction: Identifying and isolating the most relevant attributes or "features" from the
preprocessed data that are essential for pattern recognition.

3. Model Selection and Training: Choosing an appropriate ML algorithm (e.g., neural networks,
decision trees, clustering) and training it on the prepared data to learn the specific patterns.

4. Testing and Fine-Tuning: Evaluating the trained model's performance on unseen data and making
adjustments to improve its accuracy and generalization capabilities.

5. Deployment and Prediction: Integrating the finalized model into a system to make real-time
predictions or decisions on new data inputs.

Key Techniques

ML uses various approaches for pattern matching:

 Supervised Learning: The algorithm learns from a labeled dataset to classify new data into
predefined categories (e.g., classifying emails as "spam" or "not spam").

 Unsupervised Learning: The algorithm identifies hidden structures or natural groupings in

unlabeled data (e.g., clustering customers based on purchasing behavior).
 Neural Networks (Deep Learning): These systems, modeled after the human brain, use multiple
layers of processing to spot complex, abstract patterns, particularly in unstructured data like images
and audio.

 Statistical Pattern Recognition: Uses a probabilistic framework to identify patterns based on

statistically similar characteristics in data represented as feature vectors.

 Template Matching: Compares input data to a predefined reference pattern or "template" to find
identical or very similar matches.

Real-World Applications

Pattern matching in machine learning powers numerous AI applications across various industries:

 Computer Vision: Facial recognition, medical image analysis (tumor identification in X-rays), and
object detection for autonomous driving.

 Natural Language Processing (NLP): Grammar and spell checking, sentiment analysis of customer
feedback, and powering chatbots and voice assistants.

 Fraud Detection: Identifying anomalous patterns in financial transactions or network traffic that
indicate suspicious activity.

 Bioinformatics: Analyzing DNA sequences and protein structures to predict gene functions and aid
in drug discovery.

 Recommender Systems: Suggesting products, movies, or music to users based on their past behavior
and preferences.

 Predictive Maintenance: Monitoring sensor data from machinery to predict potential failures before
they occur.

Role in Machine Learning

In machine learning, pattern matching powers core functions like classification, prediction, and
anomaly detection by training models to recognize recurring data structures. Supervised models learn
from labeled examples to match new inputs against known patterns, such as identifying spam emails
via keyword sequences or image classifiers spotting object edges. Unsupervised approaches cluster
similar patterns without labels, while reinforcement learning refines matches through trial-and-error
feedback.

It enables generalization: models trained on historical sales data match seasonal trends to forecast
demand, adapting to variations via techniques like hashing or wildcard substitutions. Error correction
also relies on it, as systems detect deviations from normal patterns in noisy data and auto-correct,
improving reliability in real-world applications.

7. Explain the stages in the Machine Learning process. 10 2 CO1

Answer:
 Machine Learning Lifecycle is a structured process that defines how machine learning (ML)
models are developed, deployed and maintained. It consists of a series of steps that ensure the
model is accurate, reliable and scalable.

Machine Learning Lifecycle

It includes defining the problem, collecting and preparing data, exploring patterns, engineering
features, training and evaluating models, deploying them into production and continuously
monitoring performance to handle issues like data drift and retraining needs. Below are the key
steps of the ML lifecycle:
Step 1: Problem Definition
The first step is identifying and clearly defining the business problem. A well-framed problem
provides the foundation for the entire lifecycle. Important things like project objectives, desired
outcomes and the scope of the task are carefully designed during this stage.
 Collaborate with stakeholders to understand business goals
 Define project objectives, scope and success criteria
 Ensure clarity in desired outcomes
Step 2: Data Collection
Data Collection phase involves systematic collection of datasets that can be used as raw data to
train model. The quality and variety of data directly affect the model’s performance.
Here are some basic features of Data Collection:
 Relevance: Collect data should be relevant to the defined problem and include necessary
features.
 Quality: Ensure data quality by considering factors like accuracy and ethical use.
 Quantity: Gather sufficient data volume to train a robust model.
 Diversity: Include diverse datasets to capture a broad range of scenarios and patterns.
Step 3: Data Cleaning and Preprocessing
Raw data is often messy and unstructured and if we use this data directly to train then it can lead to
poor accuracy. We need to do data cleaning and preprocessing which often involves:
 Data Cleaning: Address issues such as missing values, outliers and inconsistencies in the data.
 Data Preprocessing: Standardize formats, scale values and encode categorical variables for
consistency.
 Data Quality: Ensure that the data is well-organized and prepared for meaningful analysis.
Step 4: Exploratory Data Analysis (EDA)
To find patterns and characteristics hidden in the data Exploratory Data Analysis (EDA) is used to
uncover insights and understand the dataset's structure. During EDA patterns, trends and insights
are provided which may not be visible by naked eyes. This valuable insight can be used to make
informed decision.
Here are the basic features of Exploratory Data Analysis:
 Exploration: Use statistical and visual tools to explore patterns in data.
 Patterns and Trends: Identify underlying patterns, trends and potential challenges within the
dataset.
 Insights: Gain valuable insights for informed decisions making in later stages.
 Decision Making: Use EDA for feature engineering and model selection.
Step 5: Feature Engineering and Selection
Feature engineering and selection is a transformative process that involve selecting only relevant
features to enhance model efficiency and prediction while reducing complexity.
Here are the basic features of Feature Engineering and Selection:
 Feature Engineering: Create new features or transform existing ones to capture better patterns
and relationships.
 Feature Selection: Identify subset of features that most significantly impact the model's
performance.
 Domain Expertise: Use domain knowledge to engineer features that contribute meaningfully
for prediction.
 Optimization: Balance set of features for accuracy while minimizing computational
complexity.
Step 6: Model Selection
For a good machine learning model, model selection is a very important part as we need to find
model that aligns with our defined problem, nature of the data, complexity of problem and the
desired outcomes.
Here are the basic features of Model Selection:
 Complexity: Consider the complexity of the problem and the nature of the data when choosing
a model.
 Decision Factors: Evaluate factors like performance, interpretability and scalability when
selecting a model.
 Experimentation: Experiment with different models to find the best fit for the problem.
Step 7: Model Training
With the selected model the machine learning lifecycle moves to model training process. This
process involves exposing model to historical data allowing it to learn patterns, relationships and
dependencies within the dataset.
Here are the basic features of Model Training:
 Iterative Process: Train the model iteratively, adjusting parameters to minimize errors and
enhance accuracy.
 Optimization: Fine-tune model to optimize its predictive capabilities.
 Validation: Rigorously train model to ensure accuracy to new unseen data.
Step 8: Model Evaluation and Tuning
Model evaluation involves rigorous testing against validation or test datasets to test accuracy of
model on new unseen data. It provides insights into model's strengths and weaknesses. If the model
fails to acheive desired performance levels we may need to tune model again and adjust its
hyperparameters to enhance predictive accuracy.
Here are the basic features of Model Evaluation and Tuning:
 Evaluation Metrics: Use metrics like accuracy, precision, recall and F1 score to evaluate
model performance.
 Strengths and Weaknesses: Identify the strengths and weaknesses of the model through
rigorous testing.
 Iterative Improvement: Initiate model tuning to adjust hyperparameters and enhance
predictive accuracy.
 Model Robustness: Iterative tuning to achieve desired levels of model robustness and
reliability.
Step 9: Model Deployment
Now model is ready for deployment for real-world application. It involves integrating the
predictive model with existing systems allowing business to use this for informed decision-making.
Here are the basic features of Model Deployment:
 Integrate with existing systems
 Enable decision-making using predictions
  Ensure deployment scalability and security
 Provide APIs or pipelines for production use
Step 10: Model Monitoring and Maintenance
After Deployment models must be monitored to ensure they perform well over time. Regular
tracking helps detect data drift, accuracy drops or changing patterns and retraining may be needed
to keep the model reliable in real-world use.
Here are the basic features of Model Monitoring and Maintenance:
 Track model performance over time
 Detect data drift or concept drift
 Update and retrain the model when accuracy drops
 Maintain logs and alerts for real-time issues
Each step is essential for building a successful machine learning model that can provide valuable
insights and predictions. By following the Machine learning lifecycle organizations we can solve
complex problems.

8. Explain Data Acquisition and Feature Engineering in Machine Learning. 10 2

CO1

Answer:

Data acquisition and feature engineering are crucial initial steps in the machine learning
pipeline, preparing raw data for model training.

Data Acquisition

Data acquisition in machine learning refers to the process of gathering and collecting relevant data
from various sources. This raw data forms the foundation for training and evaluating machine
learning models. Sources can be diverse, including:

 Databases: Structured data from relational or NoSQL databases.

 APIs: Data fetched from web services or external platforms.

 Sensors and IoT Devices: Real-time data streams from physical sensors.

 Web Scraping: Extracting data from websites.

 Log Files: Event data from applications or systems.

The goal of data acquisition is to obtain a dataset that is comprehensive, representative, and of
sufficient quality for the specific machine learning task at hand. This step often involves handling
data formats, ensuring data integrity, and establishing efficient data pipelines for continuous data
flow.

Feature Engineering

Feature engineering is the process of transforming raw data into features that are more suitable for
machine learning algorithms. It involves creating new features or modifying existing ones to improve
model performance and interpretability. This process often leverages domain expertise and
creativity. Key aspects include:

 Feature Creation:

Generating new features from existing ones (e.g., calculating age from date of birth, combining
multiple columns into a single meaningful feature).
 Feature Transformation:

Applying mathematical functions or statistical methods to features (e.g., log transformation for
skewed data, polynomial features to capture non-linear relationships).

 Handling Categorical Features:

Encoding categorical variables into numerical representations (e.g., one-hot encoding, ordinal
encoding).

 Feature Scaling:

Normalizing or standardizing numerical features to a common range or scale (e.g., Min-Max scaling,
Z-score standardization).

 Handling Missing Values:

Imputing or removing missing data points using various strategies (e.g., mean imputation, median
imputation, dropping rows).

 Feature Selection:
Identifying and selecting the most relevant features to reduce dimensionality and improve model
efficiency and performance.
Effective feature engineering can significantly enhance model accuracy, reduce overfitting, and
provide better insights into the underlying data patterns.

9. Describe Model Selection, Model Learning, and Model Evaluation. 10 2

CO1

Answer:

Machine learning (ML) is a field that enables computers to learn patterns from data and
make predictions without being explicitly programmed. However, one of the most crucial aspects
of machine learning is selecting the right model for a given problem. This process is called model
selection. The choice of model significantly affects the accuracy, efficiency and reliability of
predictions. A bad model can cause overfitting or underfitting and sometimes even lead to
increased computational costs.
In this article, we are going to deeply explore into the process of model selection, its importance
and techniques used to determine the best-performing machine learning model for different
problems.
Importance of Model Selection
Model selection is a key step in machine learning because it affects how well a system can learn
from data and make accurate predictions. Different models have different ways of processing data
and choosing the right one ensures that the system works efficiently. A simple model cannot
capture details and has poor accuracy, while a model too complex might overfit that is doing very
well on training data but fails on new data. The goal is to find a model that learns patterns
effectively without being too simple or too complex.
 Proper model selection involves experimenting with different models and comparing their
performance using evaluation metrics such as accuracy, precision, recall or mean squared error .
These metrics help in determining which model is best suited for a given task.
 Apart from performance metrics, other factors such as training time, dataset size and available
computing power also play a crucial role in choosing the right model.
  Selecting an appropriate model not only improves prediction accuracy but also enhances
efficiency, making the system faster and more reliable. This ensures that AI-driven
applications perform well in real-world scenarios.
Steps in Model Selection
Understanding the Problem and Data
Before selecting a model, it is important to first analyze the problem we are trying to solve. The
initial step is to determine whether it is a regression problem, where the goal is to predict
continuous values like house prices. If the task involves predicting categorical labels, such as
distinguishing between spam and non-spam emails, it falls under classification problem. On the
other hand, if the objective is to group similar data points, like segmenting customers based on
behavior, then it is a clustering problem. Understanding the type of problem helps in choosing the
most suitable machine learning model.
Another important point is a bit about the nature of the dataset itself. One has to check for missing
values, the number of numerical and categorical variables and the distribution of data .
Understanding the type of problem and the dataset helps in choosing the most suitable machine
learning model.
Selecting Suitable Models
After understanding the problem, we then choose a best model that should solve the problem.
Different types of models work better for different kinds of problems:
 For Regression: Linear Regression, Decision Trees, Random Forest, Neural Networks.
 For Classification: Logistic Regression , Support Vector Machines (SVM), k-Nearest
Neighbors (k-NN), Neural Networks.
 For Clustering: k-Means, Hierarchical Clustering, DBSCAN.

Model Learning (Training)

 What it is: The process where a machine learning algorithm analyzes data (training set) to find
patterns, relationships, and structures, adjusting its internal parameters to build a predictive model.

 Goal: To learn a function that maps input features to output labels or values accurately.

 Example: A linear regression model learns the best slope and intercept for a line that fits data points;
a neural network adjusts its weights and biases.
Model Evaluation
Once we have identified the right models, we must rank each one according to how well it does the
job. The most common method is to split the dataset into two parts.
 Training Set: The data used to train a machine learning model by learning patterns and
relationships.
 Testing Set: This checks how well a model performs over new, unseen data.
We use k-fold cross-validation to further improve the evaluation. In k-fold cross-validation, the
data is split into k subsets. The model is trained on k-1 subsets and tested on the remaining one,
repeating the process k times. This way, our evaluation is not biased by a particular train-test split.
Different machine learning problems require different evaluation metrics.
 For Regression Problems: We make use of Mean Squared Error (MSE), Mean Absolute Error
(MAE) and R-squared.
 For Classification Problems: We make use of Accuracy, Precision, Recall and F1-score.
After evaluating the models, we compare them to identify the one that satisfies performance and
computational efficiency.
Model Selection Techniques in Machine Learning
Grid Search
One of the simplest and most commonly used model selection techniques is grid search. In this
approach, systematically different combinations of hyperparameters are tried and that gives the best
performance chosen. It can be effective, but the main drawback will be computationally intensive,
especially for complex models and many parameters.
 Random Search
Similar to grid search, random search doesn't check all possible combinations. Instead, it randomly
chooses a subset of the hyperparameter combinations. The random search method often runs much
faster than the grid search method and yet achieves equally good results.
Bayesian Optimization
Bayesian optimization is a smarter approach to model selection. Instead of just randomly searching
for the best hyperparameters, it uses probability models to predict which parameters are likely to
perform best and focuses on evaluating those. This method is efficient and often finds better results
than grid or random search.
Cross-Validation Based Selection
This method involves using cross-validation to evaluate multiple models and selecting the one with
the best average performance. Instead of relying on a single train-test split, cross-validation divides
the dataset into multiple parts and trains the model on different subsets. This helps to ensure that
the model’s performance is not just due to a specific split of data. By averaging the results from
different splits, we get how well the model will perform on new, unseen data. This approach
reduces the risk of overfitting and helps in selecting a good model.

10. Explain Search and Learning, and the importance of Data Sets. 10 2 CO1

Answer:

Search and Learning, powered by large, high-quality Data Sets, use patterns in information
(data) to train AI models to find answers (search) and make predictions or decisions (learning), with
datasets acting as the essential fuel, determining accuracy and reliability for tasks from finding
information to recognizing images. Datasets are vital because they provide the raw material for
training, evaluating, and benchmarking AI, enabling systems to generalize and perform well on new
data, making quality crucial for useful outcomes.

Search and Learning: In the context of AI, search involves algorithms that explore vast information
spaces to find relevant results, like Google finding web pages for your query, often using learned
models to rank relevance.

 Learning: This is the process where AI models, especially in Machine Learning (ML) and Deep
Learning, automatically improve from data without explicit programming.

o Training: Models analyze datasets to find patterns and relationships.

o Inference: The trained model uses these learned patterns to make predictions or classifications on
new data.
Dataset is a collection of various types of data stored in a digital format. Data is the key
component of any Machine Learning project. Datasets primarily consist of images, texts, audio,
videos, numerical data points, etc., for solving various Artificial Intelligence challenges such as

 Image or video classification

 Object detection
 Face recognition
 Emotion classification
 Speech analytics
 Sentiment analysis
 Stock market prediction, etc.
 Importance of Data Sets

 Fuel for AI: Data is the fundamental resource for AI; deep learning models are "data-hungry" and
require massive amounts of text, images, audio, etc., to function.

 Accuracy & Reliability: High-quality, reliable datasets lead to accurate models, while poor data
results in flawed performance, regardless of algorithm.

 Model Training & Evaluation: Datasets are used to train models, test their performance on unseen
data (validation), and compare different algorithms fairly (benchmarking).

 Insight Discovery: Analyzing large datasets helps uncover hidden trends, correlations, and patterns,
informing better decisions.

 Real-World Application: Datasets, especially open ones from governments or institutions, enable
innovation, research, and practical applications in various fields.

Key Characteristics of Good Datasets

 Large Quantity: Enough data for models to learn complex patterns.

 High Quality: Accurate, consistent, and relevant.

 Diverse: Represents various scenarios the model might encounter.

Unit No: II (Unit Name: Nearest Neighbor-Based Models) M BL

COs

1. Explain Proximity Measures and their role in Nearest Neighbor–based models. 10 2

CO2

Answer:

Proximity measures (also called similarity or dissimilarity measures) quantify how “close” or
“similar” two objects are in the feature space. They are the foundation of nearest-neighbor (NN)
methods: NN algorithms locate training examples nearest to a query point according to some
proximity measure, then use those neighbors to classify the query, predict a value, detect outliers, etc.
Choosing and using an appropriate proximity measure is therefore critical — it directly affects which
points are considered neighbors and thus the model’s output.

Below is a structured, exam-style treatment covering definitions, common measures (with formulas),
how they are used in NN models, practical issues, and recommendations.

Definitions and terminology

 Distance / dissimilarity: numerical value d(x,y)d(x,y)d(x,y) where smaller means more

similar (e.g., Euclidean distance).
 Similarity: numerical value s(x,y)s(x,y)s(x,y) where larger means more similar (e.g., cosine
similarity).
  Metric: a distance function satisfying non-negativity, identity of indiscernibles, symmetry,
and triangle inequality. Some useful functions (e.g., Mahalanobis, Euclidean) are metrics;
others (e.g., cosine when viewed as 1 − cosine) may not satisfy all metric properties.

What Proximity Measures Are

 Definition: Functions that calculate the degree of likeness or difference between two data objects
(vectors).

 Types:

o Distance Metrics (Dissimilarity): Lower values mean more similar (e.g., Euclidean, Manhattan,
Minkowski).

o Similarity Metrics: Higher values mean more similar (e.g., Cosine Similarity).

 Key Examples:

o Euclidean Distance: The straight-line distance between points (most common).

o Manhattan Distance: Sum of absolute differences across dimensions (city-block distance).

o Minkowski Distance: A generalized metric including Euclidean and Manhattan.

Role in Nearest-Neighbor Models (k-NN)

 Core Principle: Assumes similar items are near each other (e.g., "birds of a feather flock together").

 Function:

1. Calculate Distances: For a new data point (query), calculate its distance to all training data points
using a chosen proximity measure.

2. Identify Neighbors: Find the 'k' closest points (nearest neighbors) with the smallest distances.

3. Predict/Classify:

o Classification: Assign the new point the most common class label among its 'k' neighbors (majority
vote).

o Regression: Predict the value by averaging the values of its 'k' neighbors.

 Impact: The choice of proximity measure heavily influences model performance; a poor metric can
misidentify neighbors and lead to incorrect predictions.

Importance in Data Mining

 Foundation: Essential for any algorithm relying on data point relationships.

 Applications: Drives k-NN, clustering (grouping), anomaly detection (identifying outliers far from
neighbors), and recommendation systems.

2. Explain different Distance Measures used in Machine Learning. 10 2 CO2

Answer:

Measures of distance are mathematical functions used to quantify how similar or dissimilar two
objects are based on their features. These measures are critical for clustering, classification and
information retrieval because they help determine relationships among data points. The choice of
distance depends on the nature of the data and the application domain.
 Used to quantify similarity/dissimilarity between objects.
 Smaller distance = higher similarity, larger distance = higher dissimilarity.
 Important for clustering algorithms (e.g., K-means, hierarchical clustering).
 Choice of distance is context-dependent (numerical, categorical or text data).
Let's see few types of distances.
1. Euclidean Distance

Euclidean Distance

Euclidean distance is considered the traditional metric for problems with geometry. It can be
simply explained as the ordinary distance between two points. It is one of the most used algorithms
in the cluster analysis.
Formula:
d(x,y)=∑i=1n(xi−yi)2
 Best for: Continuous numerical data (when features are normalized).
 Example: Distance between two cities on a 2D map.
2. Manhattan Distance
Manhattan Distance determines the absolute difference among the pair of the coordinates. Suppose
we have two points P and Q to determine the distance between these points we simply have to
calculate the perpendicular distance of the points from X-Axis and Y-Axis. In a plane with P at
coordinate (x1, y1) and Q at (x2, y2). Manhattan distance between P and Q = |x1 – x2| + |y1 – y2|

d(x,y)=∑i=1n∣xi−yi∣d(x,y)=∑i=1n∣xi−yi∣
Formula:

 Best for: High-dimensional data or when diagonal movement has no meaning.

 Example: Distance between blocks in a city (taxicab geometry).
3. Jaccard Index

The Jaccard distance is set-based distance that compares dissimilarity by looking at the ratio of
unique to common elements.

d(A,B)=1−∣A∩B∣∣A∪B∣
Formula:

Best for: Binary or categorical data, especially sets.

 Example: Comparing similarity of shopping carts or tag sets.
4. Minkowski distance
Minkowski distance is a generalized distance measure that includes both Euclidean and Manhattan
distances as special cases, controlled by a parameter p.

d(x,y)=(∑i=1n∣xi−yi∣p)1p
Formula:

 Best for: Flexible distance calculations where p is tuned.

 Example: For p=1, it becomes Manhattan; for p=2, it becomes Euclidean.
5. Cosine Similarity / Cosine Distance
Measures the cosine distance of the angle between two vectors, focusing on orientation rather than
magnitude. Commonly converted to distance as 1−similarity.

Cosine(x,y)=x⋅y∣∣x∣∣∣∣y∣∣
Formula(Similarity):

d(x,y)=1−x⋅y∣∣x∣∣∣∣y∣∣
Formula(Distance):

 Best for: Text mining, NLP, recommendation systems.

 Example: Measuring similarity between two documents regardless of their length.
6. Hamming Distance
The number of positions where two strings (of equal length) differ. Commonly used for error
detection and sequence comparison.
Formula:
d(x,y)=∑i=1n[xi≠yi]
where [ [xi≠yi]=1if symbols differ, else 0.
 Best for: Binary strings, DNA sequences, error correction.
 Example: Hamming distance between “karolin” and “kathrin” = 3.

3. What are Non-Metric Similarity Functions? Explain with examples. 10 2 CO2

Answer:

Introduction

In Machine Learning and pattern recognition, similarity functions are used to measure how alike
two data objects are. While many similarity/distance measures satisfy the properties of a metric,
some useful similarity functions do not satisfy all metric properties. These are known as Non-
Metric Similarity Functions.

Non-metric similarity functions are especially useful in domains such as text mining, information
retrieval, bioinformatics, and recommendation systems, where strict geometric distance
assumptions are not appropriate.

Metric vs Non-Metric (Brief)

A metric distance must satisfy:

1. Non-negativity
2. Identity of indiscernibles
3. Symmetry
4. Triangle inequality

If a similarity or distance function violates any of these properties (especially triangle inequality), it
is considered non-metric.

Definition of Non-Metric Similarity Functions

Non-Metric Similarity Functions are measures that quantify similarity between objects but do
not satisfy one or more properties of metric distances (particularly the triangle inequality). Instead
of “distance,” they often emphasize orientation, overlap, or relevance between objects.

Characteristics

 Do not strictly follow geometric distance rules

 Often used when relative similarity is more important than absolute distance
 Common in high-dimensional and sparse data
 Values usually lie in a bounded range (e.g., 0 to 1)

Common Non-Metric Similarity Functions with Examples

1. Cosine Similarity

Definition

Measures the cosine of the angle between two vectors.

Cosine Similarity(x,y)=x⋅y∥x∥∥y

Why Non-Metric

 Does not satisfy triangle inequality

 Focuses on direction, not magnitude

Example

Consider two documents represented as word vectors:

x=(1,2,0),y=(2,4,0)

Even though magnitudes differ, cosine similarity = 1 (same direction).

Application

 Text classification
 Document similarity
 Search engines

2. Jaccard Similarity

Definition

Measures similarity between two sets.

J(A,B)=∣A∩B∣∣A∪B∣

Why Non-Metric

 The derived distance 1−J(A,B)1-J(A,B)1−J(A,B) does not satisfy triangle inequality

Example

 Document A: {AI, ML, Data}

 Document B: {ML, Data, Mining}

J=24=0.5

Application

 Market basket analysis

 Recommendation systems
 Binary pattern matching

3. Dice (Sørensen–Dice) Coefficient

Definition
Dice=2∣A∩B∣∣A∣+∣B∣

Why Non-Metric

 Emphasizes common elements

 Does not satisfy triangle inequality

Example

Two DNA sequences with overlapping genes — higher weight to shared patterns.

Application

 Bioinformatics
 Information retrieval

4. Overlap Coefficient

Overlap(A,B)=∣A∩B∣min⁡(∣A∣,∣B∣)
Definition

Why Non-Metric

 One set being subset of another gives similarity = 1

 No metric structure

Application

 Plagiarism detection
 Keyword matching

5. Edit Similarity (Inverse of Edit Distance)

Definition

Measures similarity based on minimum operations to transform one string into another.
Why Non-Metric

 Similarity variant does not obey triangle inequality

Example

 “machine” → “marine” (2 edits)

Application

 Spell checking
 DNA sequence matching

Advantages

 Better suited for real-world, unstructured data

 Capture semantic similarity
 Robust to scale differences

Limitations

 Cannot use standard metric indexing structures (KD-trees)

 Increased computational complexity
 Harder theoretical guarantees

Non-Metric Similarity Functions in Nearest Neighbor-Based Models

▪ Non-metric similarity functions are measures used to evaluate the similarity between data
points without strictly adhering to the properties of a metric space (e.g., triangle inequality).
▪ These functions are often used in scenarios where relationships between data points cannot
be easily captured by traditional metric distance measures, such as Euclidean or Manhattan
distance.
Characteristics of Non-Metric Similarity Functions
▪ They focus on measuring similarity rather than distance.
▪ They may not satisfy properties like symmetry, non-negativity, or the triangle inequality.
▪ They are particularly useful in domains where data relationships are non-linear or symbolic
Non-Metric Similarity Functions in Nearest Neighbor-Based Models
▪ Non-metric similarity functions are measures used to evaluate the similarity between data
points without strictly adhering to the properties of a metric space (e.g., triangle inequality).
▪ These functions are often used in scenarios where relationships between data points cannot
be easily captured by traditional metric distance measures, such as Euclidean or Manhattan
distance.
Characteristics of Non-Metric Similarity Functions
▪ They focus on measuring similarity rather than distance.
▪ They may not satisfy properties like symmetry, non-negativity, or the triangle inequality.
▪ They are particularly useful in domains where data relationships are non-linear or symbolic
Non-Metric Similarity Functions in Nearest Neighbor-Based Models
▪ Non-metric similarity functions are measures used to evaluate the similarity between data
points without strictly adhering to the properties of a metric space (e.g., triangle inequality).
▪ These functions are often used in scenarios where relationships between data points cannot
be easily captured by traditional metric distance measures, such as Euclidean or Manhattan
distance.
Characteristics of Non-Metric Similarity Functions
▪ They focus on measuring similarity rather than distance.
▪ They may not satisfy properties like symmetry, non-negativity, or the triangle inequality.
▪ They are particularly useful in domains where data relationships are non-linear or symbolic
Non-Metric Similarity Functions in Nearest Neighbor-Based Models
▪ Non-metric similarity functions are measures used to evaluate the similarity between data
points without strictly adhering to the properties of a metric space (e.g., triangle inequality).
▪ These functions are often used in scenarios where relationships between data points cannot
be easily captured by traditional metric distance measures, such as Euclidean or Manhattan
distance.
Characteristics of Non-Metric Similarity Functions
▪ They focus on measuring similarity rather than distance.
▪ They may not satisfy properties like symmetry, non-negativity, or the triangle inequality.
▪ They are particularly useful in domains where data relationships are non-linear or symbolic
Non-Metric Similarity Functions in Nearest Neighbor-Based Models
▪ Non-metric similarity functions are measures used to evaluate the similarity between data
points without strictly adhering to the properties of a metric space (e.g., triangle inequality).
▪ These functions are often used in scenarios where relationships between data points cannot
be easily captured by traditional metric distance measures, such as Euclidean or Manhattan
distance.
Characteristics of Non-Metric Similarity Functions
▪ They focus on measuring similarity rather than distance.
▪ They may not satisfy properties like symmetry, non-negativity, or the triangle inequality.
▪ They are particularly useful in domains where data relationships are non-linear or symbolic

4. Explain Proximity Measures for Binary Patterns. 10 2 CO2

Answer:

Proximity measures for binary patterns quantify the degree of similarity or dissimilarity
between two data objects where attributes are binary, meaning they can only take on two values
(typically 0 or 1, or true/false). These measures are crucial in various data mining and machine
learning tasks, such as clustering, classification, and anomaly detection.

Understanding the Components:

When comparing two binary patterns (objects) with multiple binary attributes, four types of attribute
matches can occur:

 q: The number of attributes where both objects have a value of 1 (e.g., both are "true").
 r: The number of attributes where the first object has a value of 1 and the second has a value
of 0.
 s: The number of attributes where the first object has a value of 0 and the second has a value
of 1.
 t: The number of attributes where both objects have a value of 0 (e.g., both are "false").

The total number of attributes is N=q+r+s+t

.
Types of Proximity Measures:
Proximity measures can be categorized as either similarity measures (higher values indicate greater
similarity) or dissimilarity measures (higher values indicate greater dissimilarity).
1. Similarity Measures:

 Simple Matching Coefficient (SMC): This measure considers both mutual 1s and mutual 0s
as indicators of similarity.

SMC = (q + t) / N
  Jaccard Coefficient: This measure focuses only on mutual 1s, ignoring mutual 0s, making it
suitable for asymmetric binary attributes where the absence of an attribute (0) is less
significant than its presence (1).

Jaccard = q / (q + r + s)
2. Dissimilarity Measures:

 Hamming Distance: This measures the number of positions at which the corresponding
attributes are different. It is the inverse of the SMC.

Hamming Distance = r + s

 Euclidean Distance (for binary data): While typically used for continuous data, it can be
applied to binary data.

Euclidean Distance = sqrt(r + s)

Choosing the Right Measure:
The choice of proximity measure depends on the nature of the binary attributes and the specific
application. For symmetric binary attributes (where 0 and 1 are equally important), SMC and
Hamming distance are appropriate. For asymmetric binary attributes (where 1 is more significant
than 0), Jaccard is often preferred.

5. Explain classification algorithms based on distance measures. 10 2 CO2

Answer:

Classification algorithms based on distance measures determine the class of a new data point by
evaluating its similarity to existing, labeled data points. This similarity is quantified using various
distance metrics. The fundamental idea is that data points belonging to the same class are typically
"closer" to each other in the feature space than to data points of other classes.

Key Concepts:

 Distance Measures: These are mathematical functions that quantify the dissimilarity between two
data points. Common examples include:
o Euclidean Distance: The straight-line distance between two points in Euclidean space. It is widely
used for continuous numerical data.
d(p,q) = sqrt(sum((pi - qi)^2))

 Manhattan Distance (City Block Distance): The sum of the absolute differences of their
coordinates. Useful for high-dimensional data or when movement is restricted to axes.
d(p,q) = sum(|pi - qi|)

 Minkowski Distance: A generalization of Euclidean and Manhattan distances, incorporating a

parameter 'p'.
d(p,q) = (sum(|pi - qi|^p))^(1/p)

 Hamming Distance: Measures the number of positions at which two strings of equal length
differ. Primarily used for categorical or binary data.
 Exemplars/Neighbors: These are the labeled data points in the training set that are used as
references for classifying new, unseen data.
 Prominent Algorithm: K-Nearest Neighbors (KNN)

The K-Nearest Neighbors (KNN) algorithm is a classic example of a distance-based classification

algorithm. It operates as follows:

 Store Training Data:

The algorithm stores all available labeled training data points.

 Determine K:

A parameter 'K' is chosen, representing the number of nearest neighbors to consider for classification.

 Calculate Distances:

For a new, unlabeled data point, the algorithm calculates its distance to all training data points using a
chosen distance metric (e.g., Euclidean distance).

 Identify K Nearest Neighbors:

The 'K' training data points with the smallest distances to the new point are identified.

 Majority Vote:
The class label of the new data point is assigned based on a majority vote among its 'K' nearest
neighbors. For instance, if K=5 and three of the nearest neighbors belong to Class A and two to Class
B, the new data point is classified as Class A.
Advantages of Distance-Based Classification:

 Simplicity: Algorithms like KNN are relatively easy to understand and implement.

 Non-parametric: They make no assumptions about the underlying data distribution.

 Flexibility: Can be used with various distance metrics and adapted to different data types.
Limitations:

 Computational Cost:

Calculating distances to all training points can be computationally expensive, especially with large
datasets.

 Sensitivity to K:

The choice of 'K' significantly impacts performance and requires careful tuning.

 Curse of Dimensionality:
Performance can degrade in high-dimensional spaces as distances become less meaningful.

6. Explain the K-Nearest Neighbor (KNN) classifier in detail. 10 2 CO2

Answer:

K-Nearest Neighbors (KNN) is a supervised machine learning algorithm generally used for
classification but can also be used for regression tasks. It works by finding the "k" closest data
points (neighbors) to a given input and makes a predictions based on the majority class (for
classification) or the average value (for regression). Since KNN makes no assumptions about the
underlying data distribution it makes it a non-parametric and instance-based learning method.

K-Nearest Neighbors is also called as a lazy learner algorithm because it does not learn from the
training set immediately instead it stores the entire dataset and performs computations only at the
time of classification.
For example, consider the following table of data points containing two features:

The new point is classified as Category 2 because most of its closest neighbors are blue squares.
KNN assigns the category based on the majority of nearby points. The image shows how KNN
predicts the category of a new data point based on its closest neighbours.
 The red diamonds represent Category 1 and the blue squares represent Category 2.
 The new data point checks its closest neighbors (circled points).
 Since the majority of its closest neighbors are blue squares (Category 2) KNN predicts the new
data point belongs to Category 2.
KNN works by using proximity and majority voting to make predictions.
What is 'K' in K Nearest Neighbour?
In the k-Nearest Neighbours algorithm k is just a number that tells the algorithm how many nearby
points or neighbors to look at when it makes a decision.
Example: Imagine you're deciding which fruit it is based on its shape and size. You compare it to
fruits you already know.
 If k = 3, the algorithm looks at the 3 closest fruits to the new one.
 If 2 of those 3 fruits are apples and 1 is a banana, the algorithm says the new fruit is an apple
because most of its neighbors are apples.
How to choose the value of k for KNN Algorithm?
 The value of k in KNN decides how many neighbors the algorithm looks at when making a
prediction.
 Choosing the right k is important for good results.
 If the data has lots of noise or outliers, using a larger k can make the predictions more stable.
 But if k is too large the model may become too simple and miss important patterns and this is
called underfitting.
 So k should be picked carefully based on the data.
Statistical Methods for Selecting k
 Cross-Validation: Cross-Validation is a good way to find the best value of k is by using k-fold
cross-validation. This means dividing the dataset into k parts. The model is trained on some of
these parts and tested on the remaining ones. This process is repeated for each part. The k value
that gives the highest average accuracy during these tests is usually the best one to use.
 Elbow Method: In Elbow Method we draw a graph showing the error rate or accuracy for
different k values. As k increases the error usually drops at first. But after a certain point error
stops decreasing quickly. The point where the curve changes direction and looks like an
"elbow" is usually the best choice for k.
 Odd Values for k: It’s a good idea to use an odd number for k especially in classification
problems. This helps avoid ties when deciding which class is the most common among the
neighbors.
Distance Metrics Used in KNN Algorithm
KNN uses distance metrics to identify nearest neighbor, these neighbors are used for classification
and regression task. To identify nearest neighbor we use below distance metrics:
1. Euclidean Distance
Euclidean distance is defined as the straight-line distance between two points in a plane or space.
You can think of it like the shortest path you would walk if you were to go directly from one point
to another.
distance(x,Xi)=∑j=1d(xj−Xij)2]distance(x,Xi)=∑j=1d(xj−Xij)2]
2. Manhattan Distance
This is the total distance you would travel if you could only move along horizontal and vertical
lines like a grid or city streets. It’s also called "taxicab distance" because a taxi can only drive

d(x,y)=∑i=1n∣xi−yi∣d(x,y)=∑i=1n∣xi−yi∣
along the grid-like streets of a city.

3. Minkowski Distance
Minkowski distance is like a family of distances, which includes both Euclidean and Manhattan
distances as special cases.
d(x,y)=(∑i=1n(xi−yi)p)1pd(x,y)=(∑i=1n(xi−yi)p)p1
From the formula above, when p=2, it becomes the same as the Euclidean distance formula and
when p=1, it turns into the Manhattan distance formula. Minkowski distance is essentially a
flexible formula that can represent either Euclidean or Manhattan distance depending on the value
of p.
Working of KNN algorithm
Thе K-Nearest Neighbors (KNN) algorithm operates on the principle of similarity where it predicts
the label or value of a new data point by considering the labels or values of its K nearest neighbors
in the training dataset.
Step 1: Selecting the optimal value of K
 K represents the number of nearest neighbors that needs to be considered while making
prediction.
Step 2: Calculating distance
 To measure the similarity between target and training data points Euclidean distance is widely
used. Distance is calculated between data points in the dataset and target point.
Step 3: Finding Nearest Neighbors
 The k data points with the smallest distances to the target point are nearest neighbors.
Step 4: Voting for Classification or Taking Average for Regression
 When you want to classify a data point into a category like spam or not spam, the KNN
algorithm looks at the K closest points in the dataset. These closest points are called neighbors.
The algorithm then looks at which category the neighbors belong to and picks the one that
appears the most. This is called majority voting.
 In regression, the algorithm still looks for the K closest points. But instead of voting for a class
in classification, it takes the average of the values of those K neighbors. This average is the
predicted value for the new point for the algorithm.
It shows how a test point is classified based on its nearest neighbors. As the test point moves the
algorithm identifies the closest 'k' data points i.e. 5 in this case and assigns test point the majority
class label that is grey label class here.

7. Explain the Radius Distance Nearest Neighbor algorithm. 10 2 CO2

Answer:

r-Nearest neighbors are a modified version of the k-nearest neighbors. The issue with k-nearest
neighbors is the choice of k. With a smaller k, the classifier would be more sensitive to outliers. If
the value of k is large, then the classifier would be including many points from other classes. It is
from this logic that we get the r near neighbors algorithm.
Intuition:
Consider the following data, as the training set.

The green color points belong to class 0 and the red color points belong to class 1. Consider the
white point P as the query point whose
 If we take the radius of the circle as 2.2 units and if a circle is drawn using the point P as the center
of the circle, the plot would be as follows

As the number of points in the circle belonging to class 1 (5 points) is greater than the number of
points belonging to class 0 (2 points)
Algorithm:
Step 1: Given the point P, determine the sub-set of data that lies in the ball of radius r centered at

Br (P) = { Xi ∊ X | dist( P, Xi ) ≤ r }
P,

Step 2: If Br (P) is empty, then output the majority class of the entire data set.
Step 3: If Br (P) is not empty, output the majority class of the data points in it.
The Radius Distance Nearest Neighbor algorithm, often referred to as Radius Neighbors or r-Nearest
Neighbors, is a variant of the k-Nearest Neighbor (KNN) algorithm used for classification and
regression tasks. Unlike KNN, which considers a fixed number of nearest neighbors (k), the Radius
Neighbors algorithm considers all training data points that fall within a specified fixed radius (r)
around a new, unclassified data point.

Here's how it works:

 Training Phase:

The algorithm simply stores the entire training dataset, including features and corresponding labels
(for classification) or target values (for regression). There's no explicit model building during this
phase.

 Prediction Phase:
When a new data point (query point) needs to be classified or its value predicted:
 Define Radius: A fixed radius r (a floating-point value) is specified by the user. This radius defines
the neighborhood around the query point.

 Identify Neighbors: The algorithm calculates the distance between the query point and all training
data points. Any training point whose distance from the query point is less than or equal to r is
considered a neighbor.

 Prediction:
 For Classification: The class label of the query point is determined by the majority class among its
identified neighbors within the radius r.

 For Regression: The value of the query point is predicted by averaging the target values of its
identified neighbors within the radius r.
Key Characteristics and Differences from KNN:

 Fixed Radius, Variable Neighbors:

Instead of a fixed number of neighbors (k), Radius Neighbors uses a fixed radius (r), meaning the
number of neighbors can vary depending on the density of data points within that radius.

 Handling Sparse Data:

This approach is particularly useful in datasets where data density varies significantly. In dense
regions, more neighbors will be considered, while in sparse regions, fewer neighbors will contribute
to the prediction, preventing distant, potentially irrelevant points from influencing the outcome.

 Sensitivity to Radius Choice:

The choice of r is crucial. A small r might lead to sparse neighborhoods and potentially miss
important information, while a large r could include too many irrelevant points.

 Computational Cost:
Finding all points within a radius can be computationally intensive, especially in high-dimensional
spaces, though optimized data structures like Ball Trees or KD Trees can mitigate this.

8. Explain KNN Regression with an example. 10 2 CO2

Answer:

K-Nearest Neighbors (KNN) is one of the simplest and most intuitive machine learning algorithms.
While it is commonly associated with classification tasks, KNN can also be used for regression.
This article will delve into the fundamentals of KNN regression, how it works, and how to
implement it using Scikit-Learn, a popular machine learning library in Python.
What is KNN Regression?
KNN regression is a non-parametric method used for predicting continuous values. The core idea is
to predict the target value for a new data point by averaging the target values of the K nearest
neighbors in the feature space. The distance between data points is typically measured using
Euclidean distance, although other distance metrics can be used.
How KNN Regression Works
1. Choosing the number of neighbors (K): The initial step involves selecting the number of
neighbors, K. This choice greatly affects the model's performance. A smaller value of K makes
the model more prone to noise, whereas a larger value of K results in smoother predictions.
2. Calculating distances: For a new data point, calculate the distance between this point and all
points in the training set.
3. Finding K nearest neighbors: Identify the K points in the training set that are closest to the
new data point.
4. Predicting the target value: Compute the average of the target values of the K nearest
neighbors and use this as the predicted value for the new data point.
Implementing KNN Regression with Scikit-Learn using Synthetic Dataset
Let's go through a practical example of implementing KNN regression using Scikit-Learn. We will
use a synthetic dataset for demonstration purposes.
Step 1: Import Libraries
In this step, we import the necessary libraries for generating the dataset, splitting the data, training
the KNN model, evaluating the model, and visualizing the results.
import numpy as np
import [Link] as plt
from [Link] import make_regression
from sklearn.model_selection import train_test_split
from [Link] import KNeighborsRegressor
from [Link] import mean_squared_error, r2_score
Step 2: Generate Synthetic Dataset
Here, we generate a synthetic dataset using Scikit-Learn's make_regression function. This function
creates a regression problem with a specified number of samples, features, and noise level.
# Generate synthetic dataset
X, y = make_regression(n_samples=200, n_features=1, noise=0.1, random_state=42)
Step 3: Split the Dataset
We split the dataset into training and testing sets using the train_test_split function. This step
ensures that we have separate data for training the model and evaluating its performance.
# Split the dataset into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
Step 4: Create and Train the KNN Regressor
In this step, we create an instance of the KNeighborsRegressor with a specified number of
neighbors (K=5). We then train the model using the training data.
# Create and train the KNN regressor
knn_regressor = KNeighborsRegressor(n_neighbors=5)
knn_regressor.fit(X_train, y_train)
Step 5: Make Predictions
We use the trained KNN regressor to make predictions on the test data.
# Make predictions on the test data
y_pred = knn_regressor.predict(X_test)
Step 6: Evaluate the Model
Here, we evaluate the model's performance using the Mean Squared Error (MSE) and R-squared
(R²) metrics. These metrics help us understand how well the model is performing.
# Evaluate the model
mse = mean_squared_error(y_test, y_pred)
r2 = r2_score(y_test, y_pred)

print(f'Mean Squared Error: {mse}')

print(f'R-squared: {r2}')
Step 7: Visualize the Results
Finally, we visualize the actual and predicted values using a scatter plot. This step helps us visually
assess the model's performance.
# Visualize the results
[Link](X_test, y_test, color='blue', label='Actual')
[Link](X_test, y_pred, color='red', label='Predicted')
[Link]('KNN Regression')
[Link]('Feature')
[Link]('Target')
[Link]()
[Link]()

9. Explain Performance measures of Classification algorithms. 10 2 CO2

Answer:
 Performance measures for classification algorithms are crucial for evaluating how well a model
categorizes data into predefined classes. These metrics provide insights into the model's accuracy,
reliability, and its ability to handle different types of classification errors.

1. Confusion Matrix:

A fundamental tool that summarizes the performance of a classification algorithm. It's a table
showing the number of correct and incorrect predictions made by the classifier compared to the
actual outcomes (ground truth). It consists of:

 True Positives (TP): Correctly predicted positive instances.

 True Negatives (TN): Correctly predicted negative instances.

 False Positives (FP): Incorrectly predicted positive instances (Type I error).

 False Negatives (FN): Incorrectly predicted negative instances (Type II error).

2. Accuracy:
The most straightforward metric, representing the proportion of correctly classified instances out of
the total number of instances.
Accuracy = (TP + TN) / (TP + TN + FP + FN)
While intuitive, accuracy can be misleading in imbalanced datasets where one class significantly
outnumbers others.

3. Precision:
Measures the proportion of correctly predicted positive instances out of all instances predicted as
positive. It indicates the model's ability to avoid false positives.
Precision = TP / (TP + FP)

4. Recall (Sensitivity):
Measures the proportion of correctly predicted positive instances out of all actual positive
instances. It indicates the model's ability to find all positive instances and avoid false negatives.
Recall = TP / (TP + FN)

5. F1-Score:
The harmonic mean of Precision and Recall. It provides a balanced measure when both false
positives and false negatives are important.
F1-Score = 2 * (Precision * Recall) / (Precision + Recall)

6. Specificity:
Measures the proportion of correctly predicted negative instances out of all actual negative instances.
Specificity = TN / (TN + FP)

7. AUC-ROC Curve (Area Under the Receiver Operating Characteristic Curve):

The ROC curve plots the True Positive Rate (Recall) against the False Positive Rate (1 - Specificity)
at various threshold settings. The AUC represents the area under this curve, providing an aggregate
measure of the model's ability to distinguish between classes across all possible classification
thresholds. A higher AUC indicates better performance.

8. Log Loss (Logarithmic Loss):

A measure of prediction uncertainty. It penalizes incorrect classifications more heavily when the
predicted probability is confident but wrong. Lower log loss indicates better performance.

10. Explain Performance evaluation of Regression algorithms. 10 2 CO2

Answer:

Performance evaluation of regression algorithms involves quantifying how accurately a model

predicts continuous target values. Unlike classification, where metrics focus on correct class
assignments, regression evaluation metrics measure the difference between predicted and actual
values. Commonly used metrics include:

 Mean Absolute Error (MAE):

o Calculates the average of the absolute differences between predicted and actual
values.
o It is robust to outliers as it does not square the errors.
o Formula:

MAE=1n∑i=1n|yi−ŷi|cap M cap A cap E equals 1 over n end-fraction sum from i

equals 1 to n of the absolute value of y sub i minus y hat sub i end-absolute-value

𝑀𝐴𝐸=1𝑛𝑛𝑖=1|𝑦𝑖−𝑦̂𝑖|

 Mean Squared Error (MSE):

o Calculates the average of the squared differences between predicted and actual values.
o Penalizes larger errors more significantly due to squaring.
o Formula:

MSE=1n∑i=1n(yi−ŷi)2cap M cap S cap E equals 1 over n end-fraction sum from i

equals 1 to n of open paren y sub i minus y hat sub i close paren squared

𝑀𝑆𝐸=1𝑛𝑛𝑖=1(𝑦𝑖−𝑦̂𝑖)2

 Root Mean Squared Error (RMSE):

o The square root of MSE.
o Provides an error measure in the same units as the target variable, making it more
interpretable than MSE.
o Formula:

RMSE=1n∑i=1n(yi−ŷi)2cap R cap M cap S cap E equals the square root of 1 over n end-fraction
sum from i equals 1 to n of open paren y sub i minus y hat sub i close paren squared end-root

𝑅𝑀𝑆𝐸=1𝑛𝑛𝑖=1(𝑦𝑖−𝑦̂𝑖)2

 R-squared (𝑅2) Score:

o Represents the proportion of the variance in the dependent variable that is predictable
from the independent variables.
o Ranges from 0 to 1, with higher values indicating a better fit. A value of 1 implies a
perfect fit, while 0 suggests the model explains no variance. It can be negative if the model
 performs worse than a simple mean prediction.
o Formula:

R2=1−SSESSTcap R squared equals 1 minus the fraction with numerator cap S cap S cap E and
denominator cap S cap S cap T end-fraction

𝑅2=1−𝑆𝑆𝐸𝑆𝑆𝑇

where SSE is the sum of squared errors of the regression line and SST is the total sum of squares.

 Huber Loss:
o A hybrid loss function that combines the advantages of MAE and MSE.
o It behaves like MSE for small errors and like MAE for large errors, providing a
balance between outlier sensitivity and robustness.

Considerations for Evaluation:

 Outliers:

MAE and Huber Loss are generally more robust to outliers than MSE and RMSE.

 Interpretability:

RMSE is often preferred over MSE due to its units matching the target variable. R-squared provides
a clear indication of explained variance.

 Context:

The choice of metric often depends on the specific problem and the desired trade-offs between
different error types.

Unit No: III(Unit Name: Models Based on Decision Trees)

[Link] is the basic idea behind constructing a Decision Tree for classification? 10 2
CO3

Answer:

Decision Trees for Classification:

o Decision Tree is a Supervised learning technique that can be used for both
classification and Regression problems, but mostly it is preferred for solving
Classification problems. It is a tree-structured classifier, where internal nodes represent
the features of a dataset, branches represent the decision rules and each leaf node
represents the outcome.
o In a Decision tree, there are two nodes, which are the Decision Node and Leaf
Node. Decision nodes are used to make any decision and have multiple branches,
whereas Leaf nodes are the output of those decisions and do not contain any further
branches.
 o The decisions or the test are performed on the basis of features of the given dataset.
o It is a graphical representation for geFng all the possible solutions to a
problem/decision based on given conditions.
o It is called a decision tree because, similar to a tree, it starts with the root node, which
expands on further branches and constructs a tree-like structure.
o In order to build a tree, we use the CART algorithm, which stands for Classification
and Regression Tree algorithm.
o A decision tree simply asks a question, and based on the answer (Yes/No), it further
split the tree into subtrees.
o Below diagram explains the general structure of a decision tree

Decision Tree Terminologies:

-> Root Node: Root node is from where the decision tree starts. It represents the
entire dataset, which further gets divided into two or more homogeneous sets.

-> Leaf Node: Leaf nodes are the final output node, and the tree cannot be segregated
further after getting a leaf node.

-> Splitting: Splitting is the process of dividing the decision node/root node into sub-
nodes according to the given conditions.

-> Branch/Sub Tree: A tree formed by splitting the tree.

-> Pruning: Pruning is the process of removing the unwanted branches from the tree.

-> Parent/Child node: The root node of the tree is called the parent node, and other
nodes are called the child nodes.

Advantages of the Decision Tree:

-> It is simple to understand as it follows the same process which a human follow while
making any decision in real-life.
 -> It can be very useful for solving decision-related problems.

-> It helps to think about all the possible outcomes for a problem.

-> There is less requirement of data cleaning compared to other algorithms.

Disadvantages of the Decision Tree:

-> The decision tree contains lots of layers, which makes it complex.

-> It may have an overfitting issue, which can be resolved using the Random Forest
algorithm.
-> For more class labels, the computational complexity of the decision tree may
increase.
Applications of Decision Tree in Machine Learning:

1. Select a flight to travel: Decision trees are very good at classification and hence
can be used to select which flight would yield the best “bang-for-the-buck”.Reliable is the
service record of the given airliner, etc.

2. Selecting alternative products: Often in companies, it is important to

determine which product will be more profitable at launch.

3. Sentiment Analysis: Sentiment Analysis is the determination of the overall

opinion of a given piece of text and is especially used to determine if the writer’s comment
towards a given product/service is positive, neutral or negative.

4. Energy Consumption: It is very important for electricity supply boards to

correctly predict the amount of energy consumption in the near future for a particular region.
Fault Diagnosis: In the Engineering domain, one of the widely used applications of decision
trees is the determination of faults

2. What are impurity measures, and why are they used in decision tree splitting? 10 2
CO3

Answer:

While implementing a Decision tree, the main issue arises that how to select the best attribute
for the root node and for sub-nodes. So, to solve such problems there is a technique which is
called as Attribute selection measure or ASM. By this
measurement, we can easily select the best attribute for the nodes of the tree. There
are two popular techniques for ASM, which are:

 Information Gain
 Gini Index
 1. Information Gain:

o Information gain is the measurement of changes in entropy after the

segmentation of a dataset based on an attribute.

o It calculates how much information a feature provides us about a class.

o According to the value of information gain, we split the node and build the
decision tree.
o A decision tree algorithm always tries to maximize the value of information
gain, and a node/attribute having the highest information gain is split first. It can be
calculated using the below formula:

Information Gain= Entropy(S)- [(Weighted Avg) *Entropy(each feature)

Entropy: Entropy is a metric to measure the impurity in a given attribute. It specifies

randomness in data. Entropy can be calculated as:

Entropy(s)= -P(yes)log2 P(yes)- P(no) log2 P(no)

Where,

o S= Total number of samples

o P(yes)= probability of yes

o P(no)= probability of no

2. Gini Index:

-> Gini index is a measure of impurity or purity used while creating a decision tree in
the CART(Classification and Regression Tree) algorithm.

-> An attribute with the low Gini index should be preferred as compared to the high
Gini index.

-> It only creates binary splits, and the CART algorithm uses the Gini index to create
binary splits.

-> Gini index can be calculated using the below formula:

Pruning: Getting an Optimal Decision tree

Pruning is a process of deleting the unnecessary nodes from a tree in order to get the
optimal decision tree.
A too-large tree increases the risk of overfitting, and a small tree may not capture all
the important features of the dataset. Therefore, a technique that decreases the size of the
learning tree without reducing accuracy is known as Pruning. There are mainly two types of
trees pruning technology used:

-> Cost Complexity Pruning-

->Reduced Error Pruning.

3. Explain The properties of decision trees. 10 2 CO3

Answer:

Decision trees are a popular and versatile model for both classification and regression tasks.

Understanding the **properties of decision trees** helps in understanding how they

behave, their advantages, and potential limitations.

Below is a detailed breakdown of the key properties of decision trees.

1. Non-parametric Model

A decision tree does not assume any underlying data distribution (such as Gaussian
or linear). It builds the model by partitioning the data into smaller subsets based on feature
values, without making assumptions about how the data should be distributed.

2. Tree Structure

Node: A decision tree consists of:

Root node: The starting point of the tree where the first split occurs.

Internal nodes: These represent decision points where the dataset is further split based
on certain feature values.

Leaf nodes: These are terminal nodes that provide the final prediction (class label in
classification or continuous value in regression).

-Edges: The edges between the nodes represent the conditions or tests applied to
features that guide the splitting process.
3. Greedy Learning Algorithm

Decision trees use a greedy algorithm for building the tree. At each internal node, the
algorithm selects the best feature and the corresponding threshold that splits the data into two
subsets based on some impurity measure (e.g., Gini, entropy).

4. Overfitting
Decision trees are highly prone to overfitting especially when the tree is allowed to
grow deep with many branches.
 5. Interpretability and Transparency
Decision trees are easy to understand and interpret, making them one of the most transparent
machine learning models.

6. Handling Both Numerical and Categorical Data

Decision trees can handle both numerical and categorical variables. For numerical
variables, splits are based on inequality conditions

8. **non-linearity**
Decision trees do not rely on linearity in the data. Each decision (split) in the tree
represents a non-linear decision boundary, meaning decision trees can model complex, non-
linear relationships.

9. Splitting Criteria (Impurity Measures) - Decision trees use impurity measures to

determine the quality of splits. The most common criteria are:

Gini Impurity (used in the CART algorithm).

Entropy (used in algorithms like ID3, C4.5).
Variance Reduction (used for regression tasks).

10. Handling Missing Data

Decision trees can handle missing values in the dataset by using surrogate splits
(alternative splits) or by assigning the missing data to a specific branch.
11. Bias-Variance Tradeoff
A shallow decision tree may have high bias because it doesn't capture enough
complexity in the data (underfitting). It may make overly simplistic assumptions.

A deep decision tree tends to have high variance meaning it can model the noise in
the data, leading to overfitting and poor generalization.

12. Pruning
Pruning refers to the process of removing parts of the tree that are not contributing
significantly to its predictive power (e.g., branches that lead to overfitting).
Pre-pruning (stopping the tree before it becomes too deep).
Post-pruning (allowing the tree to grow fully, then trimming it back).

13. Sensitivity to Data Changes

Decision trees are relatively sensitive to small changes in the data. A small change in
the dataset can sometimes result in a drastically different tree.

14. Model Complexity

The complexity of a decision tree is determined by the depth of the tree and the
number of nodes. The depth is the longest path from the root to a leaf, and the number of
nodes is the total number of splits.

15. Scalability
Decision trees can handle a large number of features and data points, but the
algorithm's performance can degrade as the dataset grows very large.

[Link] does a decision tree perform regression? 10 2 CO3

Answer:
 Decision tree regression is a machine learning technique that uses a tree-like structure
to predict continuous numerical values:
Decision tree regression is a non-parametric approach for modeling the
relationship between input variables and continuous output variables.
Decision tree regression is a machine learning technique that can be used to
predict continuous outcomes from data. Here are some applications of decision tree
regression
It works by partitioning the feature space into regions and making predictions
based on the mean (or median) of the output variable within each [Link] decision
tree splits the data into smaller subsets, recursively, based on the feature values, until
a stopping condition is met (such as a maximum tree depth or a minimum number of
samples in a node).
A decision tree for regression is a machine learning algorithm that uses a tree-
like structure to predict numerical [Link]'s a popular tool for data mining and
machine learning algorithms.
Regression Trees
Regression trees are similar to decision trees but have leaf nodes which represent
real [Link] regression trees we will start with a simple example. Don’t worry, we’ll
get into the details shortly. For the root node of our tree, we ask: “is dosage less than 14
mg?”.Then using the data points seen in Plot B, we should get a tree that looks similar to the
one below.

How Decision Tree Regression Works

 The process of decision tree regression involves the following steps:

1. Select the Best Split:

At each internal node, the algorithm selects a feature and a threshold that best splits
the data into two groups based on some criterion,typically, variance reduction. The goal is to
minimize the variance within the resulting subsets, so that the samples in each subset are
assimilar as possible in terms of the target variable.
2. Recursive Splitting:
After the first split, the process is recursively repeated for each new subset (child
node) [Link] child node is treated as a new decision node, and the process of choosing
the best split continues until one of the stopping criteria is met.
3. Stopping Criteria:
The tree building stops when:
- A node contains fewer than a predefined number of data points.
- The maximum tree depth is reached.
- Further splits do not reduce the variance within the nodes significantly.

4. The impurity (variance) in the node is sufficiently low (i.e., the data within the
node is homogeneous).
5. Prediction:
Once the tree is built, predictions are made for new data points by traversing the tree
from the root to a [Link] leaf node holds a predicted value, which is typically the
**mean** (or median) of the target variable for the data points that fall into that leaf during
training.
Splitting Criteria for Decision Tree Regression
Naive Bayes classifiers are a collection of classification algorithms based on Bayes' Theorem.

It is not a single algorithm but a family of algorithms where all of them share a common
principle, i.e. every pair of features being classified is independent of each other.
To start with, let us consider a dataset.
A naive Bayes classifier (NBC) is a machine learning algorithm that uses probability to classify
data into categories.
It's a supervised learning algorithm that's often used for text classification, spam filtering,
and medical diagnosis.

Working of Naïve Bayes' Classifier:

Working of Naïve Bayes' Classifier can be understood with the help of the below example:

Suppose we have a dataset of weather conditions and corresponding target variable "Play".
So using this dataset we need to decide that whether we should play or not on a particular
day according to the weather conditions. So to solve this problem, we need to follow the
below steps:

1. Convert the given dataset into frequency tables.

2. Generate Likelihood table by finding the probabilities of given features.
3. Now, use Bayes theorem to calculate the posterior probability.
 In decision tree regression, the goal is to split the data in a way that
minimizes the variance (or another measure of spread) of the target variable within

each subset.
Advantages of decision trees for regression:
 Human-readable: Decision trees can explain which attributes are used and
how they are used to reach predictions.
 Efficient: Decision trees are efficient and simple when dealing with large
numbers of variables and cases.
 Quick processing: Decision trees can quickly process large amounts of data.

Disadvantages of decision trees for regression:
Overfitting
Decision trees can become too complex and deep, capturing noise in the data
instead of underlying patterns.
Instability
Small changes in the data can cause significant changes in the tree's structure.
Bias
Decision trees can become biased towards features with more categories,
which can affect their accuracy.
Complexity
Decision trees can be computationally expensive and difficult to interpret for large
datasets.
Subjectivity
Decision trees are best used when the decision criteria are fairly constant and
the thought process is generally applicable.
Application of decision tree regression:

Predicting the grade of a student on an exam

The number of spam emails per day

The amount of fraudulent transactions on a platform

[Link] is the bias–variance trade-off in machine learning models? 10 2 CO3

Answer:

It is important to understand prediction errors (bias and variance) when it comes to accuracy
in any machine learning algorithm.A trade-off between a model’s ability to minimize bias and
variance which is referred to as the best solution for selecting a value of Regularization
[Link] is a Hyperparameter Known as regularization constant and it is greater than
[Link] understanding of these errors would help to avoid the overfitting and underfitting
of a data set while training the algorithm

Bias:

The bias is known as the difference between the prediction of the values by the ML model and
the correct value. High in biasing gives a large error in training as well as testing data.

It’s recommended that an algorithm should always be low biased to avoid the
problem of [Link] high bias, the data predicted is in a straight-line format, it's not
fitting accurately in the data in the data set. Such fitting is known as Underfitting of [Link]
will happen when the hypothesis is too simple or linear in nature. (find the function that best
maps input to output)
The graph given below for an example of such a situation.

In such a problem, a hypothesis looks like follows.

Variance:
The variability of model prediction for a given data point which tells us spread of our
data is called the variance of the [Link] model with high variance has a very complex fit
to the training data and thus is not able to fit accurately on the data which it hasn’t seen
[Link] a result, such models perform very well on training data but has high error rates on
test [Link] a model is high on variance, it is then said to as Overfitting of Data.
Overfitting is fitting the training set accurately via complex curve and high order
hypothesis but is not the solution as the error with unseen data is [Link] training a data
model variance should be kept low. The high variance data looks like follows

In such a problem, a hypothesis looks like follows.

 Bias Variance Trade-off:
If the algorithm is too simple (hypothesis with linear eq.) then it may be on high bias
and low variance condition and thus is [Link] algorithms fit too complex (hypothesis
with high degree eq.) then it may be on high variance and low [Link] the latter condition, the
new entries will not perform well. Well, there issomething between both of these conditions,
known as Trade-off or Bias Variance [Link] algorithm can’t be more complex and less
complex at the same time. For the graph, the perfect tradeoff will be [Link] best fit will be
given by hypothesis on the tradeoff [Link]–variance tradeoff describes the relationship
between a model's complexity, the accuracy of its predictions

This is referred be the best point chosen for the training of the algorithm which gives
low error in training as well as testing data. The error to complexity graph to show trade-off
is given as –
 [Link]- Variance Trade off or Trade-Off

[Link] is a Random Forest, and how does it improve over a single decision tree?

Answer:

Random Forest is a popular machine learning algorithm that belongs to the supervised learning
[Link] can be used for both Classification and Regression problems in ML. It is based on the
concept of ensemble learning, which is a process of combiningmultiple classifiers to solve a complex
problem and to improve the performance of the model. Ensemble methods is a machine learning
technique that combines several base models in order to produce one optimal predictive model.
 There are two main reasons to use an ensemble over a single model, and they are
related; they are: Performance: An ensemble can make better predictions and achieve better
performance than any single contributing model.  Robustness: An ensemble reduces the
spread or dispersion of the predictions and model performance As the name suggests,
"Random Forest is a classifier that contains a number of decision trees on various subsets of
the given dataset and takes the average to improve the predictive accuracy of that dataset."
The greater number of trees in the forest leads to higher accuracy and prevents the problem of
[Link] below diagram explains the working of the Random Forest algorithm:
Note: To better understand the Random Forest Algorithm, you should have knowledge of the
Decision Tree [Link] for Random Forest Since the random forest combines
multiple trees to predict the class of the dataset, it is possible that some decision trees may predict the
correct output, while others may not. But together, all the trees predict the correct output. Therefore,
below are two assumptions for a better Random Forest classifier:

o There should be some actual values in the feature variable of the dataset so that the classifier
can predict accurate results rather than a guessed result.

o The predictions from each tree must have very low correlations.

Why use Random Forest:

Below are some points that explain why we should use the Random Forest algorithm:

o It takes less training time as compared to other algorithms.

o It predicts output with high accuracy, even for the large dataset it runs efficiently.

o It can also maintain accuracy when a large proportion of data is missing.

Steps for Random Forest algorithm work:

Random Forest works in two-phase first is to create the random forest by combining N
decision tree, and second is to make predictions for each tree created in the first phase.
The Working process can be explained in the below steps and diagram:
Step-1: Select random K data points from the training set.
Step-2: Build the decision trees associated with the selected data points (Subsets).
Step-3: Choose the number N for decision trees that you want to build.
Step-4: Repeat Step 1 & 2.
Step-5: For new data points, find the predictions of each decision tree, and assign the new data points
to the category that wins the majority [Link] working of the algorithm can be better understood by
the below example: Example: Suppose there is a dataset that contains multiple fruit images. So, this
dataset is given to the Random Forest classifier. The dataset is divided into subsets and given to each
decision tree. During the training phase, each decision tree produces a prediction result, and when a
new data point occurs, then based on the majority of results, the Random Forest classifier predicts the
final decision. Consider the below image

7. How does a Random Forest perform regression? 10 2 CO3

Answer:

Random Forest for Regression How It Works:

The procedure for regression is very similar to that for classification, but the final prediction
is calculated as the average of the predictions from all the individual trees, rather than the majority
vote.
Steps Involved:
Bootstrapping: Same as in classification, a random sample of the training data is chosen to
train each [Link] Feature Selection: For each split, a random subset of features is considered.
Tree Construction: Each decision tree is built to predict a continuous value, rather than a class
label.
Prediction: When predicting for new data, each tree produces a continuous value, and the
final prediction is the average of all those values.
Advantages:
Reduces Overfitting: Like in classification, the ensemble nature of random forests helps to
prevent overfitting by averaging out predictions.
Handles Non-Linearity: Random forests can model complex relationships that are difficult for
linear regression models to capture.
 Works Well with Large Datasets: Random forests can be effective even on large and high-
dimensional datasets.
Uses:Predicting house prices based on various features like location, square footage, etc., or
predicting a person's weight based on height and other features.

[Link] is the Bayes Classifier in probabilistic classification? 10 2 CO3

Naïve Bayes algorithm is a supervised learning algorithm, which is based on Bayes theorem and used
for solving classification [Link] is mainly used in text classification that includes a high-
dimensional training [Link]ïve Bayes Classifier is one of the simple and most effective
Classification algorithms which helps in building the fast machine learning models that can make
quick [Link] is a probabilistic classifier, which means it predicts on the basis of the probability
of an [Link] popular examples of Naïve Bayes Algorithm are spam filtration, Sentimental
analysis, and classifying [Link] Bayes Classifier is a probabilistic machine learning model that is
based on Bayes'Theorem. It is used for classification tasks, where the goal is to predict the class (label)
of a given data point based on its [Link] Bayes Classifier is particularly known for its simplicity
and effectiveness,especially when the assumption of feature independence (called Naive Bayes) holds.
Why we called Naïve Bayes:The Naïve Bayes algorithm is comprised of two words Naïve and Bayes,
which can be described as:
Naïve: It is called Naïve because it assumes that the occurrence of a certain feature is
independent of the occurrence of other features. Such as if the fruit is identified on the bases
of color, shape, and taste, then red, spherical, and sweet fruit is recognized as an apple. Hence
each feature individually contributes to identify that it is an apple without depending on each
other.
Bayes: It is called Bayes because it depends on the principle of Bayes' Theorem.

Bayes' Theorem:

Bayes' theorem is also known as Bayes' Rule or Bayes' law, which is used to
determine the probability of a hypothesis with prior knowledge. It depends on the conditional
probability.

o The formula for Bayes' theorem is given as

Where,

P(A|B) is Posterior probability: Probability of hypothesis A on the observed event B.

P(B|A) is Likelihood probability: Probability of the evidence given that the

probability of a

hypothesis is true.

P(A) is Prior Probability: Probability of hypothesis before observing the evidence.

P(B) is Marginal Probability: Probability of Evidence

For Example, let us Consider below one
 So, as we can see from the above calculation that P(Yes|Sunny)>P(No|Sunny) Hence
on a Sunny day, Player can play the game.
10. Explain about the Advantages,Dis-Advantages and Applications of Naïve Bayes Classifier 10 2
CO3

Answer:

Advantages of Naïve Bayes Classifier:

Simple and Fast: Naive Bayes is computationally efficient and easy to implement, especially for
large datasets.
Works Well with High-Dimensional Data: Due to its simplicity and independence assumption,
Naive Bayes often performs well even in high-dimensional spaces, where other classifiers may
struggle.
Handles Missing Data: If a feature is missing for a given instance, Naive Bayes can still make a
prediction by ignoring that feature during the likelihood computation.
Effective for Text Classification: Naive Bayes is commonly used in text classification tasks like
spam filtering and sentiment analysis, where features (words) are assumed to be conditionally
independent.

o Naïve Bayes is one of the fast and easy ML algorithms to predict a class of datasets.

o It can be used for Binary as well as Multi-class Classifications.

o It performs well in multi-class predictions as compared to the other Algorithms.

o It is the most popular choice for text classification problems.

Disadvantages of Naïve Bayes Classifier:

Strong Independence Assumption: The conditional independence assumption can be overly
simplistic, especially in cases where features are highly correlated. This assumption might lead to
suboptimal performance in such cases.
Poor Performance with Small Datasets: Naive Bayes can struggle with small datasets where the
estimated probabilities may be unreliable.
Sensitive to Irrelevant Features: Since Naive Bayes treats all features independently, irrelevant or
highly noisy features can negatively impact the model’s performance.
Naive Bayes assumes that all features are independent or unrelated, so it cannot learn the relationship between
features.
Applications of Naïve Bayes Classifier:
Spam Detection: Classifying emails as spam or non-spam based on the occurrence of certain keywords.
Sentiment Analysis: Determining the sentiment of text (positive, negative, neutral) by analysing word
frequencies.
Medical Diagnosis: Predicting the likelihood of a disease based on various symptoms.
Recommendation Systems: Classifying products or movies as likely or unlikely to be of interest to a user.
It is used for Credit Scoring.
It is used in medical data classification.
It can be used in real-time predictions because Naïve Bayes Classifier is an eager learner.
It is used in Text classification such as Spam filtering and Sentiment analysis
10. Discuss about Naïve Bayes Classifier[NBC]
Naive Bayes classifiers are a collection of classification algorithms based on Bayes' Theorem.
It is not a single algorithm but a family of algorithms where all of them share a common principle, i.e. every
pair of features being classified is independent of each other.
To start with, let us consider a dataset.

A naive Bayes classifier (NBC) is a machine learning algorithm that uses probability to classify data into
categories.
It's a supervised learning algorithm that's often used for text classification, spam filtering, and medical
diagnosis.

Working of Naïve Bayes' Classifier:

Working of Naïve Bayes' Classifier can be understood with the help of the below example:
Suppose we have a dataset of weather conditions and corresponding target variable "Play". So using this
dataset we need to decide that whether we should play or not on a particular day according to the weather
conditions. So to solve this problem, we need to follow the below steps:

4. Convert the given dataset into frequency tables.

5. Generate Likelihood table by finding the probabilities of given features.

6. Now, use Bayes theorem to calculate the posterior probability.

Unit No: IV(Unit Name: Linear Discriminants for Machine Learning)

1. What is a linear discriminant in the context of machine learning? 10 2 CO4

Answer:

Linear discriminant analysis (LDA) is a machine learning technique that uses

dimensionality reduction to separate multiple classes in a [Link] discriminant
analysis (LDA) is an approach used in supervised machine learning to solve multi-class
classification [Link] separates multiple classes with multiple features through data
dimensionality reduction.
Linear Discriminant Analysis (LDA), also known as Normal Discriminant Analysis or
Discriminant Function Analysis, is a dimensionality reduction technique primarily utilized
in supervised classification [Link] facilitates the modelling of distinctions between
groups, effectively separating two or more classes.
LDA operates by projecting features from a higher-dimensional space into a lower-
dimensional one.
Application of Linear Discriminants:
 Linear discriminants, particularly through LDA, are used in many fields, including:
• Pattern Recognition: Classifying images or text based on extracted features.
• Face Recognition: Reducing dimensionality in facial image datasets for
efficient classification.
• Medical Diagnostics: Classifying medical conditions based on a set of diagnostic features.
Linear Discriminants for Classification:
Linear discriminants are powerful tools for classification, particularly in situations
where the classes are linearly separable or approximately [Link] Discriminant Analysis
(LDA) helps both reduce dimensionality and increase the performance of classification
models by finding the optimal linear combinations of features that maximize class
separability.
Linear discriminants for classification refer to the use of linear combinations of features in a
dataset to distinguish between multiple [Link] main objective is to find a boundary (or
hyperplane) that can best separate the classes in the dataset based on their [Link] is
particularly useful when you have labeled data and wish to classify new, unseen data into one
of the predefined [Link] most common method for linear discriminant classification
is Linear Discriminant Analysis (LDA).LDA seeks to find a linear decision surface
(hyperplane in higher dimensions) that best separates different classes.
2. How are linear discriminants used for classification? 10 2 CO4

Answer:

Linear Discriminant Analysis (LDA) for Classification:

Linear Discriminant Analysis is a statistical method for classification that focuses on finding a
linear combination of features that best separates two or more classes. LDA assumes that:
Each class is normally distributed (Gaussian distribution). All classes share the same
covariance matrix (homoscedasticity).
Linear Discriminant Analysis (LDA) is a powerful and commonly used tool in the field of statistics
and machine learning.
It's all about classifying our data and understanding the factors that impact differing
categories. Let's dive into the A-Z of LDA, so grab your snorkels because we're diving deep, no
geek-speak guaranteed.
At its core, LDA is all about finding a linear combination of features that separates or characterizes
two or more classes of objects or events.

Assumptions in Linear Discriminant Analysis:

Applications of Linear Discriminants in Classification:

Face Recognition: LDA can be used to reduce the dimensionality of face image data and
improve the performance of classification models.
Medical Diagnosis: Classifying medical conditions based on a set of diagnostic features. Speech
Recognition: Identifying spoken words or phonemes by classifying sound features.
Email Filtering: Classifying emails as spam or not spam based on features such as word
frequencies.
Advantages of Linear Discriminants:

Simplicity: LDA provides a simple and interpretable model for classification.

Dimensionality Reduction: LDA is also used for reducing the number of features while
retaining important class-discriminatory information.
Efficiency: LDA is computationally efficient and works well in situations where the classes are
linearly separable.
Limitations:

Assumptions: LDA assumes that the data for each class is normally distributed and that the classes
have the same covariance matrix. If these assumptions are violated, LDA may not perform well.
Linear Separation: LDA works best when classes are linearly separable.
If the classes are not linearly separable, more complex methods like Quadratic Discriminant
Analysis (QDA) or support vector machines (SVM) might be more appropriate.

[Link] is the Perceptron classifier, and how does it make predictions? 10 2 CO4

Answer:

In Machine Learning and Artificial Intelligence, Perceptron is the most commonly used term
for all [Link] is the primary step to learn Machine Learning and Deep Learning technologies,
which consists of a set of weights, input values or scores, and a threshold.
Perceptron is a building block of an Artificial Neural Network. Initially, in the mid of 19th century,
Mr. Frank Rosenblatt invented the Perceptron for performing certain calculations to detect input data
capabilities or business intelligence.
Perceptron is a linear Machine Learning algorithm used for supervised learning for various binary
[Link] algorithm enables neurons to learn elements and processes them one by one during
[Link] in Machine Learning we will discuss in-depth knowledge of Perceptron and
its basic functions in brief.
A simple binary linear classifier called a perceptron generates predictions based on the weighted
average of the input [Link] on whether the weighted total exceeds a predetermined threshold, a
threshold function determines whether to output a 0 or a 1.
One of the earliest and most basic machine learning methods used for binary classification is the
[Link] Rosenblatt created it in the late 1950s, and it is a key component of more intricate
neural network topologies.
Perceptron is Machine Learning algorithm for supervised learning of various binary classification
[Link], Perceptron is also understood as an Artificial Neuron or neural network unit that
helps to detect certain input data computations in business intelligence.
Perceptron model is also treated as one of the best and simplest types of Artificial Neural networks.
However, it is a supervised learning algorithm of binary classifiers. Hence, we can consider it as a
single-layer neural network with four main parameters, i.e., input values, weights and Bias, net sum,
and an activation function.
Components of a Perceptron:

Input Features (x): Predictions are based on the characteristics or qualities of the input data, or input
features (x).
A number value is used to represent each feature. The two classes in binary classification are commonly
represented by the numbers 0 (negative class) and 1 (positive class).
Input Weights (w): Each input information has a weight (w), which establishes its significance when
formulating predictions.
The weights are numerical numbers as well and are either initialized to zeros or small random values.
Weighted Sum (\sum f(x) ): To calculate the weighted sum, use the dot product of the input features' (x)
weights and their associated features' (w) weights.
4. Describe the Perceptron learning algorithm in brief. 10 2 CO4

Answer:

Perceptron is a linear supervised machine learning algorithm. It is used for binary classification.
This article will introduce you to a very important binary classifier, the perceptron’s, which forms the basis
for the most popular machine learning models nowadays – the neural
networks.
Perceptron Learning Algorithm is also understood as an Artificial Neuron or neural network unit that
helps to detect certain input data computations in business intelligence. The
perceptron learning algorithm is treated as the most straightforward Artificial Neural
network. It is a supervised learning algorithm of binary classifiers. Hence, it is a single-layer neural network
with four main parameters, i.e., input values, weights and Bias, net sum, and an activation function.
There are four significant steps in a perceptron learning algorithm:
First, multiply all input values with corresponding weight values and then add them to determine the
weighted sum.
Mathematically, we can calculate the weighted sum as follows:
Add another essential term called bias 'b' to the weighted sum to improve the model performance.

Next, an activation function is applied to this weighed sum, producing a binary or a continuous-
valued output.

Next, the difference between this output and the actual target value is computed to get the error term, E,
generally in terms of mean squared error. The steps up to this form the forward propagation part of
the algorithm.

We optimize this error (loss function) using an optimization algorithm. Generally, some form
of gradient descent algorithm is used to find the optimal values of the hyperparameters
like learning rate, weight, Bias, etc. This step forms the backward propagation part of the
algorithm.

5. What is the main idea behind Support Vector Machines (SVMs)? 10 2 CO4

Answer:

➔ Support Vector Machine or SVM is one of the most popular Supervised Learning algorithms, which is used for
Classification as well as Regression problems. However, primarily, it is used for Classification problems in Machine
Learning.
➔ The goal of the SVM algorithm is to create the best line or decision boundary that can segregate n-dimensional
space into classes so that we can easily put the new data point in the correct category in the future. This best decision
boundary is called a hyperplane.
➔ SVM chooses the extreme points/vectors that help in creating the hyperplane.
➔ These extreme cases are called as support vectors, and hence algorithm is termed as Support Vector Machine.
Consider the below diagram in which there are two different categories that are classified using a decision boundary or
hyperplane:
Example: SVM can be understood with the example that we have used in the KNN classifier. Suppose we see a strange cat
that also has some features of dogs, so if we want a model that can accurately identify whether it is a cat or dog, so such a
model can be created by
using the SVM algorithm. -> We will first train our model with lots of images of cats and dogs so that it can learn about
different features of cats and dogs, and then we test it with this strange creature. So as support vector creates a decision
boundary between these two data (cat and dog) and choose extreme cases (support vectors), it will see the extreme case of
cat and dog. On the basis of the support vectors, it will classify it as a cat. Consider the below diagram

➔ SVM algorithm can be used for Face detection, image classification, text categorization, etc.
Types of SVM SVM can be of two types:
o Linear SVM: Linear SVM is used for linearly separable data, which means if a dataset can be classified into two
classes by using a single straight line, then such data is termed as linearly separable data, and classifier is used called as
Linear SVM classifier.
o Non-linear SVM: Non-Linear SVM is used for non-linearly separated data, which means if a dataset cannot be
classified by using a straight line, then such data is termed as non-linear data and classifier used is called as Non-linear
SVM classifier.
Hyperplane and Support Vectors in the SVM algorithm:
➔ Hyperplane: There can be multiple lines/decision boundaries to segregate the classes in n-dimensional space, but we
need to find out the best decision boundary that helps to classify the data points. This best boundary is known as the
hyperplane of SVM.
➔ The dimensions of the hyperplane depend on the features present in the dataset, which means if there are 2 features
(as shown in image), then hyperplane will be a straight line. And if there are 3 features, then hyperplane will be a 2-
dimension plane.
➔ We always create a hyperplane that has a maximum margin, which means the maximum distance between the data
[Link] Vectors:
➔ The data points or vectors that are the closest to the hyperplane and which affect the position of the hyperplane are
ermed as Support Vector. Since these vectors support the hyperplane, hence called a Support vector.

6. How do SVMs handle linearly non-separable cases? 10 2 CO4

Linearly Non-Separable Case, Non-linear SVM:

Linear SVM:
The working of the SVM algorithm can be understood by using an example. Suppose we have a dataset that has two tags
(green and blue), and the dataset has two features x1 and x2. We want a
classifier that can classify the pair(x1, x2) of coordinates in either green or blue. Consider the below

image:

So as it is 2-d space so by just using a straight line, we can easily separate these two classes. But there can be multiple lines
that can separate these classes. Consider the below image:

Hence, the SVM algorithm helps to find the best line or decision boundary; this best boundary or region is called as a
hyperplane.
-> SVM algorithm finds the closest point of the lines from both the classes. These points are called support vectors.
-> The distance between the vectors and the hyperplane is called as margin. And the goal of SVM is to maximize this
margin.
-> The hyperplane with maximum margin is called the optimal hyperplane
Non-Linear SVM: If data is linearly arranged, then we can separate it by using a straight line, but for non-linear data, we
cannot draw a single straight line. Consider the below image

So to separate these data points, we need to add one more dimension. For linear data, we have used two dimensions x and
y, so for non-linear data, we will add a third dimension z. It can be calculated
as:

By adding the third dimension, the sample space will become as below image:

So now, SVM will divide the datasets into classes in the following way. Consider the below image:
➔ Since we are in 3-d Space, hence it is looking like a plane parallel to the x-axis. If we convert it in 2d space with
z=1, then it will become as:

Hence, we get a circumference of radius 1 in case of non-linear data.

[Link] is the kernel trick, and why is it used in SVMs? 10 2 CO4

Answer:

The kernel trick is a powerful technique that enables SVMs to solve non-linear classification problems by implicitly
mapping the input data to a higher-dimensional feature space.
By doing so, it allows us to find a hyperplane that separates the different classes of data.
The kernel trick is a method used in SVMs to enable them to classify non-linear data using a linear classifier.
By applying a kernel function, SVMs can implicitly map input data into a higher-dimensional space where a linear
separator (hyperplane) can be used to divide the classes.
This mapping is computationally efficient because it avoids the direct calculation of the coordinates in this higher space.

8. How does logistic regression differ from linear regression in classification tasks? 10 2 CO4

Answer:
o Logistic regression is one of the most popular Machine Learning algorithms, which comes under the Supervised
Learning technique. It is used for predicting the categorical dependent variable using a given set of independent variables.
o Logistic regression predicts the output of a categorical dependent variable. Therefore, the outcome must be a
categorical or discrete value. It can be either Yes or No, 0 or 1, true or False, etc. but instead of giving the exact value as 0
and 1, it gives the probabilistic values which lie between 0 and 1.
o Logistic Regression is much similar to the Linear Regression except that how they are used. Linear Regression is
used for solving Regression problems, whereas Logistic regression is used for solving the classification problems.
o In Logistic regression, instead of fitting a regression line, we fit an "S" shaped logistic function, which predicts two
maximum values (0 or 1).
o The curve from the logistic function indicates the likelihood of something such as whether the cells are cancerous or
not, a mouse is obese or not based on its weight, etc.
o Logistic Regression is a significant machine learning algorithm because it has the ability to provide probabilities
and classify new data using continuous and discrete datasets.
o Logistic Regression can be used to classify the observations using different types of data and can easily determine
the most effective variables used for the classification. The below image is showing the logistic function:

Note: Logistic regression uses the concept of predictive modeling as regression; therefore, it is called logistic regression,
but is used to classify samples; Therefore, it falls under the classification algorithm.
Logistic Function (Sigmoid Function):
o The sigmoid function is a mathematical function used to map the predicted values to probabilities.
o It maps any real value into another value within a range of 0 and 1.
o The value of the logistic regression must be between 0 and 1, which cannot go beyond this limit, so it forms a curve
like the "S" form. The S-form curve is called the Sigmoid function or the logistic function.
o In logistic regression, we use the concept of the threshold value, which defines the probability of either 0 or 1. Such
as values above the threshold value tends to 1, and a value below the threshold values tends to 0.
Assumptions for Logistic Regression:
o The dependent variable must be categorical in nature.
o The independent variable should not have multi-collinearity.
Logistic Regression Equation:
The Logistic regression equation can be obtained from the Linear Regression equation. The mathematical steps to get
Logistic Regression equations are given below:
o We know the equation of the straight line can be written as:

o In Logistic Regression y can be between 0 and 1 only, so for this let's divide the above equation by (1-y):

o But we need range between -[infinity] to +[infinity], then take logarithm of the equation it will become:
o The equation for logistic regression is:

The above equation is the final equation for Logistic Regression.

Type of Logistic Regression:
On the basis of the categories, Logistic Regression can be classified into three types:
o Binomial: In binomial Logistic regression, there can be only two possible types of the dependent variables, such as
0 or 1, Pass or Fail, etc.
o Multinomial: In multinomial Logistic regression, there can be 3 or more possible unordered types of the dependent
variable, such as "cat", "dogs", or "sheep"
o Ordinal: In ordinal Logistic regression, there can be 3 or more possible ordered types of dependent variables, such
as "low", "Medium", or "High".
Linear Regression:
->Linear regression is one of the easiest and most popular Machine Learning algorithms. It is a statistical method that is
used for predictive analysis.
->Linear regression makes predictions for continuous/real or numeric variables such as sales, salary, age, product price, etc.
-> Linear regression is one of the easiest and most popular Machine Learning algorithms. It is a statistical method that is
used for predictive analysis.
-> Linear regression algorithm shows a linear relationship between a dependent (y) and one or more independent (y)
variables, hence called as linear regression.
-> Since linear regression shows the linear relationship, which means it finds how the value of the dependent variable is
changing according to the value of the independent variable.
->The linear regression model provides a sloped straight line representing the relationship between the variables. Consider
the below image:

Mathematically, we can represent a linear regression as: y= a0+a1x+ ε

Here,
Y=DependentVariable(TargetVariable) X=IndependentVariable(predictorVariable)
a0=interceptoftheline(Givesanadditionaldegreeoffreedom) a1=Linearregressioncoefficient(scalefactortoeachinputvalue). ε
= random error
The values for x and y variables are training datasets for Linear Regression model representation

9. What are Multi-Layer Perceptrons (MLPs), and why are they used? 10 2 CO4

Answer:

Multi-Layer Perceptrons (MLPs):

A Multi-Layer Perceptron (MLP) consists of fully connected dense layers that transform input data from one dimension to
another.
It is called “multi-layer” because it contains an input layer, one or more hidden layers, and an output layer.
The purpose of an MLP is to model complex relationships between inputs and outputs, making it a powerful tool for
various machine learning tasks.
a multilayer perceptron (MLP) is a name for a modern feedforward neural network consisting of fully connected neurons
with nonlinear activation functions, organized in layers, notable for being able to distinguish data that is not linearly
separable.
The key components of Multi-Layer Perceptron includes:
Input Layer: Each neuron (or node) in this layer corresponds to an input feature. For instance, if you have three input
features, the input layer will have three neurons.
Hidden Layers: An MLP can have any number of hidden layers, with each layer containing any number of nodes. These
layers process the information received from the input layer.
Output Layer: The output layer generates the final prediction or result. If there are multiple outputs, the output layer will
have a corresponding number of neurons.

Every connection in the diagram is a representation of the fully connected nature of an MLP. This means that every node in
one layer connects to every node in the next layer.
As the data moves through the network, each layer transforms it until the final output is generated in the output layer.
Advantages of Multi-Layer Perceptron
Versatility: MLPs can be applied to a variety of problems, both classification and regression.
Non-linearity: Thanks to activation functions, MLPs can model complex, non-linear relationships in data.
Parallel Computation: With the help of GPUs, MLPs can be trained quickly by taking advantage of parallel computing.
Disadvantages of Multi-Layer Perceptron
Computationally Expensive: MLPs can be slow to train, especially on large datasets with many layers.
Prone to Overfitting: Without proper regularization techniques, MLPs can overfit the training data, leading to poor
generalization.
Sensitivity to Data Scaling: MLPs require properly normalized or scaled data for optimal performance.

10. How does backpropagation train a Multi-Layer Perceptron? 10 2 CO4

Answer:

Backpropagation is a powerful and efficient algorithm for training MLPs.

It involves computing the gradient of the loss function with respect to each weight by propagating the error backward
through the network.
Backpropagation is a key algorithm used for training MLPs.
It involves adjusting the weights of the network by propagating the error from the output back through the layers to
minimize prediction error.
The Multi-Layer Perceptron (MLP) is a cornerstone in the field of artificial neural networks.
Its architecture and the backpropagation training algorithm have significantly influenced modern deep learning.
This comprehensive blog will explore the details of MLPs, the mechanics of backpropagation, and their profound impact
on artificial intelligence.
Backpropagation is a powerful and efficient algorithm for training MLPs. It involves computing the gradient of the loss
function with respect to each weight by propagating the error backward through the network.
Forward Pass
The forward pass is the first step in the backpropagation algorithm. During this step, the input data is passed through the
network, layer by layer, until it reaches the output layer. Each neuron computes a weighted sum of its inputs, applies an
activation function to this sum, and passes the result to the next layer. This process continues until the network produces an
output.
Error Calculation
Once the network’s output is obtained, the error is calculated using a loss function. The loss function measures the
difference between the predicted output and the actual target
values. Common loss functions include Mean Squared Error (MSE) for regression tasks and Cross-Entropy Loss for
classification tasks. The goal of training is to minimize this error.
Backward Pass
The backward pass is where the magic of backpropagation happens. During this step, the algorithm computes the gradient
of the loss function with respect to each weight in the network. This is done by applying the chain rule of calculus. The
chain rule allows the
algorithm to efficiently compute the gradient of the loss with respect to each weight by multiplying gradients along the
computational graph.
Gradient Calculation: The algorithm starts at the output layer and works its way backward through the network, layer by
layer. For each layer, it calculates the gradient of the loss with respect to the weights. This involves computing how much
each weight contributed to the overall error.

Error Propagation: The error is propagated backward through the network, from the output layer to the input layer. At each
layer, the algorithm updates the weights by subtracting a fraction of the gradient from the current weights. This fraction is
determined by the learning rate, a hyperparameter that controls the size of the weight updates.
Gradient Descent Step
The final step in the backpropagation algorithm is the Gradient Descent step. During this step, the weights are updated
based on the computed gradients. This involves taking a step in the direction that reduces the error. The size of this step is
determined by the learning
rate.
Key Aspects of Backpropagation
Automatic Differentiation (Autodiff): Backpropagation uses reverse-mode autodiff, which is fast and precise. This
technique is well-suited for functions with many variables (e.g., connection weights) and few outputs (e.g., one loss value).
Autodiff allows the algorithm to automatically compute the gradients needed for the Gradient Descent step.
Random Initialization: It is crucial to initialize the weights of the hidden layers randomly. If all weights are initialized to
the same value, the neurons in each layer will be identical, and
backpropagation will not be able to train them effectively. Random initialization breaks the symmetry and allows the
neurons to learn diverse features.
Activation Functions
Activation functions are a critical component of MLPs. They introduce non-linearity into the network, allowing it to model
complex functions. Without activation functions, an MLP would be equivalent to a single-layer linear model, regardless of
the number of layers.
Logistic Function (Sigmoid)

The logistic function, also known as the sigmoid function, maps any real-valued number to a value between 0 and 1.
It is S-shaped and has a well-defined nonzero derivative, which makes it suitable for use with the backpropagation
algorithm.
Hyperbolic Tangent Function (tanh)

The hyperbolic tangent function is similar to the logistic function but outputs values between -1 and 1.
This centering effect can help speed up convergence during training.
Rectified Linear Unit (ReLU)
The ReLU function is widely used in deep learning due to its simplicity and effectiveness. It outputs the input value if it is
positive; otherwise, it outputs zero.
Although ReLU is not differentiable at zero and has a zero gradient for negative inputs, it performs well in practice and
helps mitigate the vanishing gradient problem.
Unit No: V(Unit Name: Clustering)
1. What is clustering in the context of pattern recognition and data mining? 10 2 CO5

Answer:

Clustering is an unsupervised learning technique used in pattern recognition and data mining to group a set of objects into
clusters such that objects within the same cluster are more similar to each other and more dissimilar to objects in other
clusters. It aims to discover hidden structures or natural groupings in data without using predefined class labels.

In clustering, the idea is to analyse patterns or data points and organize them into meaningful groups based on a chosen
similarity or distance measure, such as Euclidean distance, Manhattan distance, cosine similarity, or correlation.

Clustering plays a crucial role in many real-world applications because it helps to:

1. Identify patterns and trends within large datasets.

2. Reduce data complexity by grouping similar items together.
3. Support decision-making by revealing hidden structures.
4. Serve as a preprocessing step for classification, compression, and anomaly detection.

Characteristics of Clustering

 Unsupervised: No prior labels or training data required.

 Similarity-based grouping: Relies on distance or similarity measures.
 Structure discovery: Finds inherent patterns in data.
 Cluster quality: Depends on compactness (within-cluster similarity) and separation (between-cluster difference).

How Clustering Works

Clustering typically involves the following steps:

1. Feature extraction: Identify attributes that describe each data point.

2. Similarity measurement: Choose a distance metric.
3. Clustering algorithm: Apply algorithms like K-Means, hierarchical clustering, DBSCAN, etc.
4. Cluster evaluation: Examine clusters using metrics like Silhouette Score or Davies–Bouldin Index.
5. Interpretation: Analyze final clusters for meaningful insights.

Types of Clustering

Clustering methods can be broadly classified into:

1. Partitional Clustering

Divides data into non-overlapping subsets (e.g., K-Means, K-Medoids).

2. Hierarchical Clustering

Builds a hierarchy of clusters in:

  Agglomerative (bottom-up)
 Divisive (top-down)
Represented using a dendrogram.

3. Density-Based Clustering

Clusters formed by dense regions (e.g., DBSCAN, OPTICS).

4. Soft/Fuzzy Clustering

A point can belong to multiple clusters with probabilities (e.g., Fuzzy C-Means).

Applications of Clustering

Clustering is widely used in:

 Image segmentation
 Market segmentation
 Document classification
 Anomaly detection
 Bioinformatics (gene expression analysis)
 Recommendation systems

Example

Suppose we have customer data based on their purchasing behavior. Clustering can automatically group customers into
segments such as:

 High spenders
 Occasional buyers
 Bargain seekers

These groups help businesses target marketing strategies effectively.

[Link] data partitioning in clustering. 10 1 CO5

Answer:

Partitioning of Data:

Data Partitioning is the technique of distributing data across multiple tables, disks, or sites in order to improve query
processing performance or increase database manageability.

Data partitioning is the process of dividing a large data set into smaller partitions, or segments. These partitions can be
stored, accessed, and managed separately.

Data partitioning can improve the performance, scalability, security, and availability of a database.

It can also make it easier to manage and process large amounts of data.
Data partitioning can improve the performance, scalability, security, and availability of a database.

It can also make it easier to manage and process large amounts of data.

Processing: Each processing node in a system performs an operation on a single partition of the data set.

Storage: Partitions can be stored across multiple tables, disks, or sites. Availability: If one node crashes, only a fraction of
the data is lost.

Scalability: As more nodes are added, the system capacity scales linearly.

[Link] is Matrix Factorization | Clustering of Patterns 10 2 CO5

Answer:

Matrix Factorization | Clustering of Patterns:

Matrix factorization is a powerful technique in data analysis and machine learning, particularly for identifying latent
features and patterns in data.

It's widely used in recommendation systems, dimensionality reduction, and clustering. The idea is to decompose a matrix
into two or more smaller matrices, capturing the hidden structure of the data.

Matrix Representation of Data:

Imagine you have a matrix A where rows represent entities (like users) and columns

represent features or items (like products). Each entry Aij can represent the interaction or relationship between the entity
and the item, such as user ratings for movies or purchase histories for products.

Factorization of the Matrix:

The goal is to decompose matrix A into two or more matrices, typically W and H (item features), such that:

A≈W×H

where:
• W is a matrix that captures the latent factors for each user (e.g., preferences or attributes).

• H is a matrix that captures the latent factors for each item (e.g., genres or characteristics).

Clustering of Patterns refers to the process of organizing a set of patterns (data points, objects, or items) into groups or
clusters, where patterns within each cluster are more similar to each other than to those in other clusters.

This technique is commonly used in data analysis, machine learning, and pattern

recognition to identify inherent structures and hidden relationships within datasets. Clustering of patterns is a vital
technique for identifying structure and trends in data.

By grouping similar patterns, it enables a deeper understanding of the data and can lead to valuable insights in various
fields, such as marketing, healthcare, bioinformatics, and machine learning.

When applied to clustering of patterns, matrix factorization can help identify clusters or groupings within the data based on
shared patterns or features. Here's how this works conceptually

[Link] is the difference between divisive and agglomerative clustering? 10 2 CO5

Answer:

Divisive Clustering is a hierarchical clustering technique that works by starting with all data points in a single cluster and
then recursively splitting this large cluster into smaller, more homogeneous clusters. This is the top-down approach, in
contrast to agglomerative clustering, which is a bottom-up approach that starts with individual data points and merges them
into larger clusters.

Divisive clustering is is useful for datasets with a clear hierarchical structure or when top-down analysis is needed.
The success of divisive clustering depends on choosing an appropriate splitting criterion to ensure meaningful and distinct
sub-clusters are formed at each stage.

The Process of Divisive Clustering:

Start with the whole dataset: Begin with a single cluster that contains all data points.

Evaluate the best way to split: The algorithm finds a way to split this cluster into two smaller clusters. This can be done
using methods like:

K-means clustering: Some divisive algorithms use the K-means algorithm to perform the split.

Maximizing dissimilarity: Dividing the cluster to maximize the dissimilarity between the resulting sub-clusters.

Recursively split: The splitting process continues for each sub-cluster created, applying the same logic until the stopping
condition is met.

Agglomerative Clustering:

Agglomerative Clustering is one of the most popular hierarchical clustering algorithms. Unlike divisive clustering, which
follows a top-down approach,

agglomerative clustering is a bottom-up approach where the algorithm starts with each data point as its own individual
cluster and progressively merges the closest clusters until all data points are grouped into a single cluster or until a stopping
condition is met.

Steps in Agglomerative Clustering:

Start with each data point as its own cluster: Initially, each data point is considered a separate cluster.

Compute the distance between all pairs of clusters: Use the chosen distance measure (e.g., Euclidean distance) to calculate
the pairwise distances between all clusters.

Merge the closest clusters: Identify the two clusters with the smallest distance and merge them into a single cluster.

Update the distance matrix: After merging two clusters, update the distance matrix to reflect the distances between the new
cluster and the remaining clusters. This update depends on the chosen linkage criterion.

Repeat: Continue this process of merging the closest clusters and updating the distance matrix until a stopping condition is
met.

[Link] the main objective of K-Means clustering. 10 2 CO5

Answer:

K-Means Clustering:

The main objective of K-Means clustering is to partition a given dataset into K distinct, non-overlapping clusters such that
the data points within each cluster are as similar as possible, while data points in different clusters are as dissimilar as
possible.

In other words, K-Means tries to minimize the total intra-cluster variance (or error), which is measured by the sum of
squared distances between each data point and the centroid of its cluster.

Mathematical Formulation

Let the dataset be X={x1,x2,...,xn}K clusters with centroids C1,C2,..., The objective function J of K-Means is:

J=i=1∑Kx∈Ci∑∣∣x−μi∣∣2

Where:

 CiC_iCi = cluster iii

∣∣x−μi∣∣2||x - \mu_i||^2∣∣x−μi∣∣2 = squared Euclidean distance between point xxx and cluster centroid
 μi\mu_iμi = centroid of cluster CiC_iCi


Goal: Minimize J → ensures that points are close to their cluster centroids.

 The K-Means algorithm is a simple algorithm capable of clustering this kind of dataset very quickly and efficiently,
often in just a few iterations.

 It was proposed by Stuart Lloyd at Bell Labs in 1957 as a technique for pulse-code modulation, but it was only published
outside of the company in 1982. In 1965, Edward

W. Forgy had published virtually the same algorithm, so K-Means is sometimes referred to as Lloyd–Forgy.
 K-Means Clustering is an unsupervised learning algorithm that is used to solve the clustering problems in machine
learning or data science.

 K-Means Clustering is an Unsupervised Learning which groups the unlabeled dataset into different clusters. Here K
defines the number of pre-defined clusters that need to be created in the process, as if K=2, there will be two clusters, and
for K=3, there will be three clusters, and so on.

 It is an iterative algorithm that divides the unlabeled dataset into k different clusters in such a way that each dataset
belongs only one group that has similar properties.

 It allows us to cluster the data into different groups and a convenient way to discover the categories of groups in the
unlabeled dataset on its own without the need for any training.

 It is a centroid-based algorithm, where each cluster is associated with a centroid. The main aim of this algorithm is to
minimize the sum of distances between the data point and their corresponding clusters.

 The algorithm takes the unlabeled dataset as input, divides the dataset into k-number of clusters, and repeats the process
until it does not find the best clusters. The value of k should be predetermined in this algorithm.

 The k-means clustering algorithm mainly performs two tasks:

o Determines the best value for K center points or centroids by an iterative process.

o Assigns each data point to its closest k-center. Those data points which are near to the particular k-center, create a
cluster.

Hence each cluster has datapoints with some commonalities, and it is away from other clusters. The below diagram
explains the working of the K-means Clustering Algorithm:

 The vast majority of the instances were clearly assigned to the appropriate cluster, but a few instances were probably
mislabeled (especially near the boundary between the top-left cluster and the central cluster).

 Indeed, the K-Means algorithm does not behave very well when the blobs have very different diameters because all it
cares about when assigning an instance to a cluster is the distance to the centroid.

 Instead of assigning each instance to a single cluster, which is called hard clustering, it can be useful to give each
instance a score per cluster, which is called soft clustering.
 The score can be the distance between the instance and the centroid; conversely, it can be a similarity score (or affinity),
such as the Gaussian Radial Basis Function.

 In the K-Means class, the transform () method measures the distance from each instance to every centroid:

[Link] is soft clustering? 10 2 CO5

Answer:

Soft Clustering:

Soft clustering, also known as fuzzy clustering, is a machine learning technique that groups data into clusters with varying
weights or probabilities. In soft clustering, data points can belong to multiple clusters, unlike in hard clustering, where each
data point is assigned to a single cluster.

Rather than assigning each input data point to a distinct cluster, it assigns a probability or likelihood of the data point being
in those clusters.

Soft Clustering is a clustering approach in which data points do not have a definitive, hard assignment to a single cluster.

Instead, each data point has a degree of membership or probability of belonging to each cluster, allowing for a fuzzy or
probabilistic assignment.

This contrasts with hard clustering methods (like K-means), where each data point is assigned to exactly one cluster.

In soft clustering, data points can be members of multiple clusters with varying degrees of membership, which makes it a
flexible approach for handling uncertainty, overlapping clusters, and cases where clear-cut group boundaries are difficult to
define.

Advantages of Soft Clustering:

1. Captures Uncertainty and Ambiguity:

o Soft clustering is more flexible than hard clustering methods, as it can model data points that might belong to
multiple clusters with different degrees of membership. This is particularly helpful when the data is noisy or when the
boundaries between clusters are not clear.

2. Overlapping Clusters:

o Soft clustering is useful when the clusters in the data overlap or have fuzzy boundaries, which is common in many
real-world scenarios like market segmentation, image segmentation, or medical diagnoses.

3. Probabilistic Interpretation:

o Probabilistic clustering methods (like GMM) provide a clear probabilistic interpretation, giving the likelihood that a
data point belongs to a specific cluster, which is useful in uncertainty modeling and decision-making.
4. More Flexible Models:

o Soft clustering algorithms can often model more complex cluster shapes, particularly when the assumption of
spherical clusters (as in K-means) does not hold.

Disadvantages of Soft Clustering:

1. Complexity:

o Soft clustering algorithms, especially probabilistic models like GMM, can be computationally expensive compared
to hard clustering methods like K-

means. The algorithms typically require iterative optimization techniques, which may take longer to converge, especially
for large datasets.

2. Initialization Sensitivity:

o Just like hard clustering methods, soft clustering algorithms can be sensitive to initial conditions (e.g., initial
centroids in Fuzzy C-means or the initial parameters in GMM). Poor initialization may lead to suboptimal solutions.

3. Difficulty in Interpreting Results:

o Soft clustering produces a membership degree for each data point, which can make interpretation more difficult
compared to the clear-cut assignments of hard clustering. It may be harder to assess how well the clustering has separated
the data.

4. Parameter Selection:

o Some soft clustering methods (e.g., GMM) require selecting additional parameters, such as the number of clusters
or the shape of the distributions. Selecting these parameters can be challenging, especially for high- dimensional data.

Applications of Soft Clustering:

• Customer Segmentation: Understanding customers who may belong to multiple segments (e.g., "high-income" and
"tech-savvy") simultaneously.

• Document Clustering: Clustering documents that may belong to multiple topics, such as a news article about both
politics and technology.

• Image Segmentation: Segmenting regions of an image that may belong to more than one object (e.g., overlapping
objects in a scene).

• Medical Diagnosis: Modeling cases where symptoms could indicate multiple potential diseases (e.g., fuzzy
classification of disease risk).

[Link] Fuzzy C-Means (FCM) clustering. 10 1 CO5

Answer:
Fuzzy Clustering is a type of clustering technique where data points do not belong strictly to one cluster but have degrees
of membership to multiple clusters.

This approach is often referred to as soft clustering, as opposed to hard clustering, where each data point belongs to exactly
one cluster.

In fuzzy clustering, each data point is assigned a membership value that reflects how much it belongs to each cluster.

These membership values typically range from 0 to 1, with the sum of membership values across all clusters for a given
data point being equal to 1.

This allows for more flexibility in modeling real-world data, where boundaries between clusters may be ambiguous or
overlapping.

Advantages of Fuzzy Clustering:

Handling Ambiguity:

Fuzzy clustering is ideal for situations where data points do not belong strictly to one group. For example, in customer
segmentation, a customer might exhibit characteristics of multiple segments, such as being both high-income and tech-
savvy.

Overlapping Clusters:

It allows for overlapping clusters, which can be beneficial in real-world applications where clear-cut boundaries between
clusters are rare.

Soft Memberships:

The probabilistic nature of fuzzy clustering (in FCM and other methods) gives a better representation of the data, especially
when uncertainty about cluster membership exists.

Better Modeling of Complex Data:

Fuzzy clustering can capture more complex data structures, such as when clusters are not spherical (which K-means
assumes) or when the boundaries between clusters are not well defined.

Disadvantages:

Computational Complexity:

Fuzzy clustering algorithms, especially FCM, are generally more computationally intensive than hard clustering algorithms
like K-means, particularly for large datasets.

Sensitivity to Initialization:

Fuzzy clustering, like K-means, is sensitive to the initial values of the centroids and membership matrix. Poor initialization
can lead to suboptimal results.

Applications of Fuzzy Clustering:

1. Customer Segmentation:
o In marketing, customers may belong to multiple segments based on their buying behavior, such as a customer who
is both a frequent buyer and a high- spender. Fuzzy clustering helps model such behavior.

2. Image Segmentation:

o In medical imaging or computer vision, regions in an image might belong to multiple objects or tissues. Fuzzy
clustering can segment these overlapping regions effectively.

3. Document Clustering:

o In text mining, documents can belong to multiple topics (e.g., a news article that discusses both politics and
technology). Fuzzy clustering helps in grouping documents based on topic relevance.

4. Bioinformatics:

o In gene expression analysis, different genes may be associated with multiple biological processes or pathways, and
fuzzy clustering helps capture this relationship.

5. Anomaly Detection:

o Fuzzy clustering can be used to detect outliers or anomalies in data, where a data point may not belong exclusively
to one cluster but may still have partial membership to several clusters.

[Link] is rough clustering? 10 2 CO5

Answer:

Rough Clustering is a type of clustering technique that combines ideas from both fuzzy clustering and rough set theory.

It is designed to handle uncertainty, imprecision, and vagueness in data, providing a more flexible approach to clustering
where data points can belong partially to multiple clusters but are also subject to certain boundaries or approximations.

The key idea behind rough clustering is to represent vague boundaries between clusters through the use of upper and lower
approximations—concepts borrowed from rough set theory.

It is particularly useful when dealing with datasets where clusters have unclear boundaries or there is significant overlap
between clusters.

Advantages of Rough Clustering:

Handling Uncertainty:

Rough clustering is effective at handling uncertainty in the data, particularly when the boundaries between clusters are not
well-defined or when data points may belong to multiple clusters.

Flexible Cluster Boundaries:

Rough clustering does not enforce hard boundaries between clusters, allowing for

overlapping clusters and indeterminate regions, making it ideal for datasets where clusters are not well separated.

Interpretation with Rough Set Theory:

 The use of rough set theory allows for clear interpretation of cluster boundaries, as data points in the boundary region are
explicitly identified, giving insight into how uncertain the clustering process is.

Data Exploration:

Rough clustering can help in data exploration, especially when the dataset includes noisy or ambiguous points that do not
clearly belong to any single cluster.

Disadvantages of Rough Clustering:

Complexity:

Rough clustering can be computationally more complex than traditional clustering methods, especially for large datasets, as
it requires calculating the upper and lower approximations for each data point.

Parameter Sensitivity:

The performance of rough clustering algorithms can be sensitive to the choice of

parameters, such as the size of the indiscernibility relation and how approximations are calculated.

Interpretation of Boundary Region:

While the boundary region is useful for identifying uncertain points, it can sometimes be difficult to interpret or may lead
to overly complex cluster definitions.

Inaccurate Boundaries:

If the assumptions behind the indiscernibility relation or approximation process are not well defined or do not fit the data
well, the resulting clusters may not align well with the true structure of the data.

Applications of Rough Clustering:

Medical Diagnosis:

In medical data analysis, especially for diseases with overlapping symptoms, rough clustering can help identify patients
who may belong to multiple disease groups with varying degrees of certainty.

Image Segmentation:

In image processing, rough clustering can be used for segmenting regions of an image that belong to multiple objects or
regions with unclear boundaries.

Market Segmentation:

In market research, rough clustering can be applied to segment consumers who may belong to multiple segments or exhibit
mixed behaviors that do not fit neatly into a single category.

Data Mining:

Rough clustering is useful in data mining for discovering patterns in data where boundaries between clusters are ambiguous
or not clearly defined.
Pattern Recognition:

Rough clustering can be used in pattern recognition tasks, particularly in situations where the classification boundaries are
not strictly defined and some uncertainty exists in the recognition process.

[Link] is the role of the Expectation-Maximization (EM) algorithm in clustering? 10 2 CO5

Answer:

The Expectation-Maximization (EM) algorithm is defined as the combination of various unsupervised machine learning
algorithms, which is used to determine the local

maximum likelihood estimates (MLE) or maximum a posteriori estimates (MAP) for unobservable variables in statistical
models.

Further, it is a technique to find maximum likelihood estimation when the latent variables are present. It is also referred to
as the latent variable model.

A latent variable model consists of both observable and unobservable variables where observable can be predicted while
unobserved are inferred from the observed variable. These unobservable variables are known as latent variables.

EM Algorithm

The EM algorithm is the combination of various unsupervised ML algorithms, such as the k- means clustering algorithm.

Being an iterative approach, it consists of two modes. In the first mode, we estimate the missing or latent variables. Hence
it is referred to as the Expectation/estimation step (E- step).

Further, the other mode is used to optimize the parameters of the models so that it can explain the data more clearly. The
second mode is known as the maximization-step or M- step.

o Expectation step (E - step): It involves the estimation (guess) of all missing values in the dataset so that after
completing this step, there should not be any missing value.

o Maximization step (M - step): This step involves the use of estimated data in the E- step and updating the
parameters.

o Repeat E-step and M-step until the convergence of the values occurs.

The primary goal of the EM algorithm is to use the available observed data of the dataset to estimate the missing data of the
latent variables and then use that data to update the values of the parameters in the M-step.
How Expectation-Maximization (EM) Algorithm Works:

The essence of the Expectation-Maximization algorithm is to use the available observed data of the dataset to estimate the
missing data and then use that data to update the values of the parameters. Let us understand the EM algorithm in detail.

1. Initialization:

• Initially, a set of initial values of the parameters are considered. A set of incomplete observed data is given to the
system with the assumption that the observed data comes from a specific model.

2. E-Step (Expectation Step): In this step, we use the observed data in order to estimate or guess the values of the
missing or incomplete data. It is basically used to update the variables.

• Compute the posterior probability or responsibility of each latent variable given the observed data and current
parameter estimates.

• Estimate the missing or incomplete data values using the current parameter estimates.

• Compute the log-likelihood of the observed data based on the current parameter estimates and estimated missing
data.

3. M-step (Maximization Step): In this step, we use the complete data generated in the preceding “Expectation” – step
in order to update the values of the parameters. It is basically used to update the hypothesis.

• Update the parameters of the model by maximizing the expected complete data log-likelihood obtained from the E-
step.

• This typically involves solving optimization problems to find the parameter values that maximize the log-
likelihood.

• The specific optimization technique used depends on the nature of the problem and the model being used.

4. Convergence: In this step, it is checked whether the values are converging or not, if yes, then stop otherwise repeat
step-2 and step-3 i.e. “Expectation” – step and “Maximization” – step until the convergence occurs.

• Check for convergence by comparing the change in log-likelihood or the parameter values between iterations.

• If the change is below a predefined threshold, stop and consider the algorithm converged.
• Otherwise, go back to the E-step and repeat the process until convergence is achieved.

10. State the basic idea of spectral clustering. 10 1 CO5

Answer:

Spectral clustering is a graph-based clustering technique that uses the eigenvalues (spectrum) of a similarity matrix derived
from the data to perform dimensionality reduction before clustering. It is particularly useful for detecting non-convex
clusters or clusters that are not easily separable in the original [Link] core idea of spectral clustering is:

1. Represent data points as a graph, where:

o Each node is a data point.
o Edges connect points that are similar (based on a similarity measure like Gaussian kernel or k-nearest
neighbors).
o Edge weights reflect the strength of similarity between points.
2. Construct a Laplacian matrix from the similarity graph.
3. Compute the eigenvectors corresponding to the smallest eigenvalues of the Laplacian matrix. These eigenvectors
represent a low-dimensional embedding of the original data, capturing the structure of clusters.
4. Apply a simple clustering algorithm like K-Means on these eigenvectors to form the final clusters.

Steps in Spectral Clustering

1. Construct Similarity Graph

o Use adjacency based on similarity (e.g., Gaussian similarity: wij=e−∣∣xi−xj∣∣22σ2)
2. Compute Laplacian Matrix
o L=D−W(Unnormalized Laplacian)
o D is the degree matrix, W is the similarity matrix
3. Eigen Decomposition
o Compute eigenvectors corresponding to the smallest k eigenvalues
4. Form Matrix U
o Stack these eigenvectors column-wise to form a new matrix U
5. Cluster Rows of U
o Treat each row of UUU as a point in a k-dimensional space and cluster them using K-Means

Advantages of Spectral Clustering

 Can find non-convex clusters (unlike K-Means).

 Works well with complex cluster structures.
 Does not assume clusters are spherical or equal size.
 Short Answer (2Marks) Question Bank

Unit No: I(Unit Name: Introduction to Machine Learning) M BL COs

1. Question: What is Machine Learning? 2 2 CO1

Answer:

Machine Learning is a field of artificial intelligence that enables computers to learn patterns from data and make
predictions or decisions without being explicitly programmed.

2. Question: Name the main paradigms of Machine Learning. 2 1 CO1

Answer:

The main paradigms of Machine Learning are:

1. Supervised Learning
2. Unsupervised Learning
3. Reinforcement Learning

3. Question: What is learning by rote? 2 2 CO1

Answer:

Learning by rote involves memorizing examples and repeating them without understanding patterns or generalization.

4. Question: Define reinforcement learning. 2 1 CO1

Answer: Reinforcement learning is a type of learning where an agent learns by interacting with the environment and
receives rewards or penalties to maximize cumulative reward.

5. Question: What are the types of data used in Machine Learning? 2 2 CO1

Answer:

Types of data include:

 Numerical data (continuous, discrete)

 Categorical data (nominal, ordinal)
 Text data
 Image/audio/video data

6. Question: What is feature engineering? 2 2 CO1

Answer:
Feature engineering is the process of selecting, transforming, or creating relevant features from raw data to improve the
performance of a Machine Learning model.

7. Question: What is model selection in Machine Learning? 2 2 CO1

Answer:

Model selection is the process of choosing the most suitable learning algorithm for a given problem based on data
characteristics and performance metrics.

8. Question: What are the stages in a typical Machine Learning workflow? 2 2 CO1

Answer:

Stages include:

1. Data Acquisition
2. Feature Engineering & Data Representation
3. Model Selection
4. Model Learning & Training
5. Model Evaluation
6. Model Prediction

9. Question: What is the purpose of model evaluation? 2 2 CO1

Answer:

Model evaluation measures the accuracy, performance, and generalization ability of a trained model on unseen data
using metrics like accuracy, precision, recall, or F1-score.

10. Question: What is the difference between supervised and unsupervised learning? 2 2 CO1

Answer:

 Supervised Learning: Model learns from labeled data.

 Unsupervised Learning: Model learns from unlabeled data by finding patterns or clusters.

Unit No: II(Unit Name: Nearest Neighbor-Based Models)

1. What is a proximity measure in nearest neighbor models? 2 2 CO2

Answer:

A proximity measure quantifies how similar or dissimilar two data points are, often using distance or similarity functions.

2. Name common distance measures used in nearest neighbor models. 2 2 CO2

Answer:
Common distance measures include:

 Euclidean distance
 Manhattan (City Block) distance
 Minkowski distance
 Mahalanobis distance

3. What is a non-metric similarity function? 2 2 CO2

Answer:

A non-metric similarity function measures similarity between objects but does not satisfy all metric properties like triangle
inequality or symmetry.

4. How is proximity computed between binary patterns? 2 2 CO2

Answer:

Proximity between binary patterns is often computed using matching coefficients such as:

 Simple matching coefficient (SMC)

 Jaccard coefficient

5. What is the main idea of the K-Nearest Neighbor (KNN) classifier? 2 2 CO2

Answer:

KNN classifies a data point based on the majority class of its K nearest neighbors in the feature space.

6. What is the Radius Distance Nearest Neighbor algorithm? 2 2 CO2

Answer:

It classifies points based on all training points within a specified radius, rather than a fixed number K of neighbors.

7. How does KNN regression work? 2 2 CO2

Answer:

KNN regression predicts the value of a continuous variable as the average (or weighted average) of the values of its K
nearest neighbors.

8. What factors affect the performance of KNN classifiers? 2 2 CO2

Answer:

Performance depends on:

 Choice of K
 Distance metric used
  Data scaling and normalization
 Noise and outliers in data

9. How is the performance of regression algorithms evaluated? 2 2 CO2

Answer:

Performance is measured using metrics like:

 Mean Squared Error (MSE)

 Root Mean Squared Error (RMSE)
 Mean Absolute Error (MAE)
 R-squared (R2R^2R2)

10. Give one advantage of nearest neighbor-based models. 2 1 CO2

Answer:

They are simple, non-parametric, and easy to implement, and can model complex decision boundaries without assuming a
specific distribution.

Unit No: III(Unit Name: Nearest Neighbor-Based Models)

1. What is a decision tree in machine learning? 2 2 CO3

Answer:

A decision tree is a tree-like model used for classification or regression, where internal nodes represent tests on features,
branches represent outcomes, and leaves represent class labels or values.

2. What are impurity measures in decision trees? 2 2 CO3

Answer:

Impurity measures quantify how mixed the classes are in a node. Common measures include:

 Gini Index
 Entropy (Information Gain)
 Misclassification Error

3. Name one property of decision trees. 2 1 CO3

Answer:

Decision trees are:

 Interpretable and easy to visualize

 Non-parametric (no assumptions about data distribution)
 Can handle both numerical and categorical data

4. How are decision trees used for regression? 2 2 CO3

Answer:

In regression trees, leaves represent predicted continuous values, usually calculated as the mean of the target variable for
the samples in that leaf.

5. What is the bias–variance trade-off in decision trees? 2 2 CO3

Answer:

 Deep trees: Low bias, high variance (may overfit)

 Shallow trees: High bias, low variance (may underfit)
The trade-off balances model complexity and generalization.

6. What are random forests? 2 2 CO3

Answer:

Random forests are ensembles of decision trees that combine multiple trees’ predictions using majority voting for
classification or average for regression, reducing overfitting and improving accuracy.

7. State Bayes’ theorem. 2 1 CO3

Answer:

Bayes’ theorem relates conditional probabilities:

P(C∣X)=P(X∣C)⋅P(C)/P(X)

Where C is the class and X is the observed data.

8. What is the main idea of a Bayes classifier? 2 2 CO3

Answer:

The Bayes classifier predicts the class with the highest posterior probability given the observed features, based on Bayes’
theorem.

9. What is a Naive Bayes Classifier (NBC)? 2 2 CO3

Answer:

NBC assumes conditional independence between features given the class and uses Bayes’ theorem to calculate the most
probable class, making it simple and efficient.

10. How is multi-class classification handled in a Bayes classifier? 2 2 CO3

Answer:

Each class’s posterior probability P(Ci∣X)P(C_i|X)P(Ci∣X) is calculated, and the class with the highest posterior is chosen.
This works naturally for any number of classes.

Unit No: IV(Unit Name: Linear Discriminants for Machine Learning)

1. What is a linear discriminant in machine learning? 2 2 CO4

Answer:

A linear discriminant is a linear function of features used to separate classes by finding a hyperplane that best discriminates
between them.

2. What is the main idea of the Perceptron classifier? 2 2 CO4

Answer:

The Perceptron is a linear binary classifier that assigns class labels based on which side of a decision hyperplane a data
point lies.

3. State the Perceptron learning algorithm. 2 1 CO4

Answer:

The Perceptron learning algorithm iteratively updates weights whenever a point is misclassified:

w←w+η(y−y^)x

where η is the learning rate, y is the true label, and x is the feature vector.

4. What is a Support Vector Machine (SVM)? 2 2 CO4

Answer:

SVM is a supervised learning algorithm that finds a hyperplane with maximum margin separating classes in a feature
space.

5. How does SVM handle linearly non-separable data? 2 2 CO4

Answer:

It uses soft margins with slack variables or maps data to a higher-dimensional space using kernel functions to achieve
separability.

6. What is the kernel trick in SVM? 2 2 CO4

Answer:

The kernel trick allows SVM to compute inner products in high-dimensional feature spaces without explicitly transforming
data, enabling non-linear classification efficiently.

7. What is logistic regression? 2 2 CO4

Answer:

Logistic regression models the probability of a binary outcome using the logistic (sigmoid) function applied to a linear
combination of input features.
8. What is the difference between linear and logistic regression? 2 2 CO4

Answer:

 Linear regression: Predicts continuous values.

 Logistic regression: Predicts probabilities for classification (binary or multi-class).

9. What is a multi-layer perceptron (MLP)? 2 2 CO4

Answer:

An MLP is a feedforward neural network with one or more hidden layers, capable of learning non-linear decision
boundaries.

10. What is backpropagation in MLP training? 2 2 CO4

Answer:

Backpropagation is a gradient-based algorithm that propagates the error backward through the network to update weights
and minimize the loss function.

Unit No: V(Unit Name: Clustering)

1. What is clustering in data mining? 2 2 CO5

Answer:

Clustering is an unsupervised learning technique that groups data points into clusters such that points in the same cluster
are similar, and points in different clusters are dissimilar.

2. What is the difference between partitional and hierarchical clustering? 2 2 CO5

Answer:

 Partitional clustering: Divides data into non-overlapping clusters directly (e.g., K-Means).
 Hierarchical clustering: Builds a tree of clusters using agglomerative (bottom-up) or divisive (top-down)
approaches.

3. What is divisive clustering? 2 2 CO5

Answer:

Divisive clustering is a top-down hierarchical method that starts with all points in one cluster and recursively splits them
into smaller clusters.

4. What is agglomerative clustering? 2 2 CO5

Answer:
Agglomerative clustering is a bottom-up hierarchical method where each point starts as its own cluster, and clusters are
iteratively merged based on similarity.

5. What is the main idea of K-Means clustering? 2 2 CO5

Answer:

K-Means divides data into K clusters by assigning points to the nearest centroid and updating centroids iteratively to
minimize intra-cluster variance.

6. What is soft clustering? 2 2 CO5

Answer:

In soft clustering, a data point can belong to multiple clusters with membership probabilities, unlike hard clustering where
each point belongs to only one cluster.

7. What is Fuzzy C-Means (FCM) clustering? 2 2 CO5

Answer:

FCM is a soft clustering algorithm where each point has degrees of membership to all clusters, and centroids are updated
based on weighted averages.

8. What is rough clustering? 2 2 CO5

Answer:

Rough clustering uses lower and upper approximations to handle uncertainty, allowing points to belong to boundary
regions of clusters.

9. What is the role of Expectation-Maximization (EM) in clustering? 2 2 CO5

Answer:

EM-based clustering estimates parameters of probabilistic models (like Gaussian mixtures) iteratively to maximize
likelihood of the data.

10. What is spectral clustering? 2 2 CO5

Answer:

Spectral clustering uses the eigenvalues (spectrum) of a similarity graph to reduce dimensionality and cluster data based on
graph structure, useful for non-convex clusters.