Q.

1 Explain Forward Chaining and backward chaining algorithm with the help of example
in artificial example.

1. Forward Chaining Algorithm (Artificial Intelligence)

Definition

Forward Chaining is a data-driven reasoning algorithm used in Artificial Intelligence.

It starts with known facts and repeatedly applies IF–THEN rules to infer new facts until a goal
is reached.

Algorithm Steps

1. Initialize the knowledge base with known facts.

2. Match facts with the IF part of rules.
3. Fire the rule whose conditions are satisfied.
4. Add the inferred fact to the knowledge base.
5. Repeat until the goal is achieved or no rule can be applied.

Artificial Intelligence Example (Forward Chaining)

Facts (Knowledge Base):

 Fever = Yes
 Cough = Yes

Rules:

 R1: IF Fever AND Cough THEN Flu

 R2: IF Flu THEN Treatment = Rest

Goal:

 Find the treatment

Reasoning Process:

1. Fever and Cough are known.

2. Rule R1 is applied → Flu is inferred.
3. Rule R2 is applied → Treatment = Rest is inferred.

Applications

 Expert systems
 Medical diagnosis systems
 Real-time decision systems
2. Backward Chaining Algorithm (Artificial Intelligence)

Definition

Backward Chaining is a goal-driven reasoning algorithm in AI.

It starts with a goal (hypothesis) and works backward to check whether the required facts exist.

Algorithm Steps

1. Start with the goal.

2. Find a rule that concludes the goal.
3. Check if the rule’s conditions are satisfied
4. If not known, treat them as sub-goals.
5. Continue until facts are verified or failure occurs.

Artificial Intelligence Example (Backward Chaining)

Goal:

 Treatment = Rest

Rules:

 R1: IF Flu THEN Treatment = Rest

 R2: IF Fever AND Cough THEN Flu

Facts:

 Fever = Yes
 Cough = Yes

Reasoning Process:

1. To achieve the goal (Treatment = Rest), Flu must be true.

2. To prove Flu, Fever and Cough must be true.
3. Fever and Cough are already true.
4. Hence Flu is true → Treatment = Rest is true.

Difference Between Forward and Backward Chaining

Feature Forward Chaining Backward Chaining

Reasoning type Data-driven Goal-driven
Starts with Known facts Goal
Direction Facts → Goal Goal → Facts
 Feature Forward Chaining Backward Chaining
Suitable for Monitoring systems Query systems

Q.2 Describe different approaches to knowledge representation.

Knowledge Representation in Artificial Intelligence

Knowledge Representation (KR) is a way of storing knowledge in a form that a computer

system can use to reason, learn, and make decisions.

AI uses different approaches to represent knowledge depending on the problem domain.

1. Logical Representation

Description

Knowledge is represented using formal logic (Propositional Logic and Predicate Logic).
Facts and rules are expressed as logical statements.

Example

 All humans are mortal

 Ram is a human
 Therefore, Ram is mortal

Advantages

 Clear and precise

 Supports mathematical reasoning

Disadvantages

 Difficult to represent uncertainty

 Complex for large systems

2. Semantic Network Representation

Description

Knowledge is represented as a graph with nodes (objects/concepts) and edges (relationships).

Example

 Node: Bird
  Node: Sparrow
 Relationship: Sparrow is a Bird

Advantages

 Easy to understand
 Good for hierarchical knowledge

Disadvantages

 Limited inference capability

 Not suitable for complex reasoning

3. Frame-Based Representation

Description

Knowledge is organized into frames (like objects or records).

Each frame contains slots and values.

Example

Frame: Dog

 Type: Animal
 Sound: Bark
 Legs: 4

Advantages

 Structured and organized

 Supports inheritance

Disadvantages

 Not flexible for dynamic knowledge

 Limited reasoning power

4. Production Rule Representation

Description

Knowledge is represented in the form of IF–THEN rules.

Used mainly in expert systems.

Example
IF fever AND cough
THEN disease = Flu

Advantages

 Simple and intuitive

 Easy to modify rules

Disadvantages

 Rule conflict may occur

 Difficult to manage large rule sets

5. Ontology-Based Representation

Description

Ontology represents knowledge using concepts, properties, and relationships in a formal way.
Common in Semantic Web and AI applications.

Example

 Class: Vehicle
 Subclass: Car
 Property: hasEngine

Advantages

 Shared and reusable knowledge

 Supports reasoning and consistency

Disadvantages

 Time-consuming to design
 Requires domain expertise

6. Probabilistic Representation

Description

Used when knowledge is uncertain or incomplete.

Includes Bayesian Networks and Markov Models.

Example

Probability of rain = 70%

Advantages

 Handles uncertainty effectively

 Useful in real-world problems

Disadvantages

 Computationally expensive
 Requires probability data

Comparison Summary Table

Approach Representation Style Best Used For

Logical Logic statements Formal reasoning
Semantic Network Graph structure Concept relationships
Frame-Based Slot-value structure Object knowledge
Production Rules IF–THEN rules Expert systems
Ontology Concepts & relations Semantic web
Probabilistic Probabilities Uncertain environments

Q.3 Explain WUMPUS World Environmental giving it space description explain how
percept sequences generated.

1. Wumpus World Environment

Definition

The Wumpus World is a classic Artificial Intelligence problem used to explain knowledge-
based agents, logical reasoning, and decision-making under uncertainty.

It consists of a 4 × 4 grid world in which an agent must find gold and return safely while
avoiding dangers.

Components of Wumpus World

 Agent – Starts at square (1,1)

 Wumpus – A monster that kills the agent
 Pits – Bottomless holes
 Gold – The objective
 Walls – Boundary of the grid

Environment Characteristics
  Partially observable – Agent cannot see the entire world
 Deterministic – Actions have predictable outcomes
 Sequential – Current action affects future actions
 Static – World does not change
 Discrete – Finite number of states and actions

2. State Space Description of Wumpus World

What is State Space?

The state space is the set of all possible configurations of the agent and environment.

State Description Includes:

A state is defined by:

 Agent position (x, y)

 Agent direction (North, South, East, West)
 Location of Wumpus
 Location of pits
 Location of gold
 Whether Wumpus is alive or dead
 Whether agent has the gold

Initial State

 Agent at (1,1)
 Agent facing East
 Gold somewhere in grid
 Wumpus alive
 Some pits present

Actions (Operators)

 Move Forward
 Turn Left
 Turn Right
 Grab
 Shoot
 Climb

Goal State

 Agent has gold

 Agent returns safely to (1,1)
 Agent climbs out
State Space Size

 Very large due to multiple combinations of pits, Wumpus, agent position, and orientation.

3. Percept Sequence Generation in AI (Wumpus World)

What is a Percept?

A percept is the information received by the agent from the environment at a given time.

Percepts in Wumpus World

At each square, the agent may perceive:

1. Stench – Wumpus is in an adjacent square

2. Breeze – A pit is in an adjacent square
3. Glitter – Gold is in the current square
4. Bump – Agent hits a wall
5. Scream – Wumpus has been killed

Percept Representation

A percept is represented as a 5-tuple:

<Stench, Breeze, Glitter, Bump, Scream>

Each value is either True or False.

Example of Percept Sequence Generation

Step 1: Agent at (1,1)

 No adjacent pits
 No Wumpus nearby
 No gold

Percept:

<False, False, False, False, False>

Step 2: Agent moves to (1,2)

 Pit nearby → Breeze

 No Wumpus → No stench

Percept:
<False, True, False, False, False>

Step 3: Agent moves to (2,2)

 Wumpus nearby → Stench

Percept:

<True, False, False, False, False>

Percept Sequence

The percept sequence is an ordered list of percepts received over time:

[<F,F,F,F,F>, <F,T,F,F,F>, <T,F,F,F,F>, ...]

4. Importance of Percept Sequences in AI

 Helps agent infer hidden information

 Used in logical reasoning
 Helps decide safe and unsafe squares
 Guides action selection
Q.4 Explain the Bayesian Networks.

Bayesian Networks (Bayes Nets)

Definition

A Bayesian Network is a probabilistic graphical model that represents a set of variables and
their conditional dependencies using a Directed Acyclic Graph (DAG).

Each node represents a random variable, and each edge represents a probabilistic dependency.

Components of a Bayesian Network

1. Nodes
o
Represent random variables
o
Example: Rain, Traffic, Accident
2. Directed Edges
o Show dependency between variables
o Direction indicates influence
3. Conditional Probability Table (CPT)
o Defines the probability of a node given its parents
o Example: P(Traffic | Rain)

Structure of Bayesian Network

 It is a Directed Acyclic Graph (DAG)

 No cycles are allowed
 Each node is conditionally independent of its non-descendants given its parents

Example of Bayesian Network

Scenario: Weather and Traffic

Variables:

 Rain (R)
 Accident (A)
 Traffic Jam (T)

Dependencies:

 Rain → Accident
 Accident → Traffic
 Rain → Traffic

Probability Example:
  P(Rain) = 0.3
 P(Accident | Rain) = 0.4
 P(Traffic | Accident, Rain) = 0.9

The network helps compute:

Probability of traffic jam when it is raining.

Working of Bayesian Networks

1. Evidence is observed (e.g., Rain = True)

2. Probabilities are updated using Bayes’ theorem
3. The network infers unknown probabilities

Bayes’ Theorem:

P(A∣B)= P(B∣A)P(A)/ P(B)

Inference in Bayesian Networks

Inference is the process of calculating:

 Posterior probabilities
 Given some evidence

Types of Inference:

 Predictive inference (cause → effect)

 Diagnostic inference (effect → cause)
 Intercausal inference

Advantages of Bayesian Networks

 Handles uncertainty effectively

 Represents causal relationships
 Combines prior knowledge with data
 Efficient reasoning

Disadvantages of Bayesian Networks

 Complex for large networks

 Requires accurate probability values
 Computationally expensive

Applications of Bayesian Networks

  Medical diagnosis systems
 Speech recognition
 Spam filtering
 Risk analysis
 Machine learning

Comparison with Rule-Based Systems

Feature Bayesian Network Rule-Based System

Uncertainty handling Yes No
Probabilistic Yes No
Reasoning Statistical Logical

Q.5 Explain modus ponen with suitable example.

Definition

Modus Ponens is a fundamental rule of inference used in logic and Artificial Intelligence.
It allows us to derive a conclusion from a conditional statement when its premise is true.

Logical Form

If P → Q
P
∴Q

Explanation

 If statement P implies Q is true

 And P is true
 Then Q must also be true

This rule is widely used in logical reasoning, expert systems, and AI inference engines.

Suitable Example (AI / Real-life)

Statement 1: If it is raining, then the ground is wet.

Statement 2: It is raining.

Conclusion:
➡ Therefore, the ground is wet.

This conclusion is derived using Modus Ponens.

Artificial Intelligence Example

Rule:
IF Fever = Yes → Disease = Flu

Fact:
Fever = Yes

Conclusion:
Disease = Flu

(Using Modus Ponens, the AI system infers the disease.)

Symbolic Example

Let:

 P = ―User enters correct password‖

 Q = ―System grants access‖

P→Q
P
∴Q

➡ System grants access.

Importance of Modus Ponens in AI

 Used in rule-based expert systems

 Helps in logical inference
 Forms the basis of automated reasoning

Q.6 Distinguish between propositional logic and first order predicate logic knowledge
representation mechanism.

1. Propositional Logic

Definition

Propositional Logic is a formal logic system in which knowledge is represented using

propositions that have only two truth values: True or False.
Each proposition represents a complete statement, without internal structure.

Characteristics

 Uses propositional symbols (P, Q, R)

  No concept of objects or relationships
 Knowledge is expressed using logical connectives
o AND (∧), OR (∨), NOT (¬), IMPLIES (→)

Example

 P: ―Ram is a human‖
 Q: ―Ram is mortal‖
 Rule: P → Q

Limitations

 Cannot represent general rules

 Cannot express relationships between objects
 Requires repetition for similar facts

2. First-Order Predicate Logic (FOPL)

Definition

First-Order Predicate Logic is an extension of propositional logic that represents knowledge

using predicates, objects, variables, and quantifiers, allowing description of properties and
relationships.

Characteristics

 Uses predicates (Human(x), Loves(x, y))

 Supports variables (x, y)
 Uses quantifiers
o Universal (∀) – ―for all‖
o Existential (∃) – ―there exists‖
 Much more expressive and compact

Example

∀x (Human(x) → Mortal(x))
Human(Ram)
∴ Mortal(Ram)

Detailed Comparison Table

Aspect Propositional Logic First-Order Predicate Logic

Represents knowledge as atomic Represents knowledge using predicates

Definition
propositions and quantifiers
 Aspect Propositional Logic First-Order Predicate Logic

Basic unit Proposition Predicate

Truth values True / False True / False

Objects Not supported Supported

Relationships Cannot represent Can represent

Variables Not allowed Allowed

Quantifiers (∀, ∃) Not available Available

Expressiveness Low Very high

Knowledge
Poor (repetition required) High (general rules)
compactness

Reasoning power Limited Powerful

Example P→Q ∀x (Bird(x) → Fly(x))

Domain size
Small domains Large, real-world domains
handling

Inference complexity Simple More complex

Use in AI Simple rule systems Knowledge-based systems

Knowledge Representation Capability

Propositional Logic Example Limitation

To represent:

―All humans are mortal‖

We must write:

Human(Ram) → Mortal(Ram)
Human(Sita) → Mortal(Sita)
Human(John) → Mortal(John)
This becomes inefficient as the domain grows.

Predicate Logic Advantage

Same knowledge expressed as:

∀x (Human(x) → Mortal(x))

✔ Compact
✔ General
✔ Powerful

Applications in Artificial Intelligence

Logic Type Applications

Propositional Logic Simple expert systems, digital circuits

Predicate Logic AI reasoning systems, NLP, planning, theorem proving

Advantages & Disadvantages

Propositional Logic

Advantages

 Simple to understand
 Easy inference

Disadvantages

 Cannot represent complex knowledge

 Not scalable

First-Order Predicate Logic

Advantages

 Rich and expressive

 Represents real-world knowledge effectively

Disadvantages

 Computationally expensive
  More complex syntax

Q.7 Compare Machine Learning with traditional programming. Discuss types of Machine
Learning with suitable examples.

1. Traditional Programming

Definition:
In traditional programming, the programmer explicitly writes rules and logic.
The computer follows these rules to produce output.

Flow:

Input + Program (Rules) → Output

Example:
A program to check whether a number is even or odd:

IF number % 2 == 0 → Even
ELSE → Odd

Here, rules are fixed and written by humans.

2. Machine Learning

Definition:
Machine Learning is a subset of AI where the system learns patterns from data and makes
decisions without being explicitly programmed.

Flow:

Input Data + Output → Learning Algorithm → Model

New Input + Model → Output

Example:
An email spam filter learns from thousands of emails labeled spam or not spam.

Comparison Table: Machine Learning vs Traditional Programming

Aspect Traditional Programming Machine Learning

Rule creation Written manually by programmer Learned automatically from data
Input Data Data
Output Output Model
Adaptability Not adaptive Self-improving
 Aspect Traditional Programming Machine Learning
Handling complexity Difficult Efficient
Data dependency Low High
Accuracy improvement Manual changes needed Improves with more data
Use cases Calculations, control systems Prediction, classification
Example Calculator Recommendation system

Part B: Types of Machine Learning

Machine Learning is mainly classified into three types:

1. Supervised Learning

Definition

Supervised Learning uses labeled data, where both input and correct output are known.

The algorithm learns a mapping from input → output.

Types

 Classification – Output is a category

 Regression – Output is a continuous value

Example 1: Classification

Problem: Email spam detection

 Input: Email text

 Output: Spam / Not Spam

Algorithms:

 Decision Tree
 Naïve Bayes
 Support Vector Machine

Example 2: Regression

Problem: House price prediction

 Input: Area, location, rooms

 Output: Price
Algorithm:

 Linear Regression

Applications

 Medical diagnosis
 Face recognition
 Credit scoring

2. Unsupervised Learning

Definition

Unsupervised Learning uses unlabeled data.

The system finds hidden patterns or structures in data.

Common Tasks

 Clustering
 Association rule mining

Example 1: Clustering

Problem: Customer segmentation

 Input: Purchase history

 Output: Customer groups

Algorithm:

 K-Means

Example 2: Association

Problem: Market basket analysis

 Finds items frequently bought together

 Example: Bread → Butter

Algorithm:

 Apriori

Applications
  Recommendation systems
 Anomaly detection
 Data exploration

3. Reinforcement Learning

Definition

Reinforcement Learning is based on reward and punishment.

An agent learns by interacting with the environment.

Key Components

 Agent
 Environment
 Actions
 Rewards

Example

Problem: Game playing (Chess, Ludo, Video games)

 Correct move → Reward

 Wrong move → Penalty

Algorithm:

 Q-Learning
 Deep Q Networks

Applications

 Robotics
 Self-driving cars
 Game AI

Summary Table: Types of Machine Learning

Type Data Feedback Example

Supervised Labeled Yes Spam detection
Unsupervised Unlabeled No Customer clustering
Reinforcement No fixed dataset Reward-based Game playing

Advantages of Machine Learning over Traditional Programming

  Handles large and complex data
 Learns automatically
 Improves performance over time
 Suitable for real-world problems

Q.8 What are various Statistical Learning Approaches

1. Regression Methods

Description:
Regression models the relationship between input variables and a continuous output variable.

Types:

 Linear Regression
 Multiple Linear Regression
 Polynomial Regression

Example:
Predicting house prices based on area, location, and number of rooms.

Applications:
Forecasting, trend analysis

2. Classification Methods

Description:
Classification assigns data into predefined categories.

Common Algorithms:

 Logistic Regression
 Naïve Bayes
 k-Nearest Neighbors (k-NN)
 Support Vector Machine (SVM)

Example:
Email spam detection (Spam / Not Spam)

Applications:
Medical diagnosis, fraud detection

3. Bayesian Learning Methods

Description:
Uses Bayes’ theorem to update probabilities as new evidence is available.
Techniques:

 Bayesian Networks
 Naïve Bayes Classifier

Example:
Disease diagnosis based on symptoms

Applications:
Risk analysis, expert systems

4. Instance-Based Learning

Description:
Learns by storing training instances and making predictions based on similarity.

Algorithm:

 k-Nearest Neighbors (k-NN)

Example:
Handwritten digit recognition

Applications:
Pattern recognition

5. Clustering Methods

Description:
Groups data into clusters based on similarity without labeled outputs.

Algorithms:

 K-Means
 Hierarchical Clustering

Example:
Customer segmentation

Applications:
Market analysis, data mining

6. Dimensionality Reduction Techniques

Description:
Reduces the number of features while retaining important information.
Techniques:

 Principal Component Analysis (PCA)

 Linear Discriminant Analysis (LDA)

Example:
Reducing image data dimensions

Applications:
Data visualization, noise reduction

7. Density Estimation Methods

Description:
Estimates the probability distribution of data.

Techniques:

 Gaussian Mixture Models (GMM)

 Kernel Density Estimation (KDE)

Example:
Anomaly detection in network traffic

Applications:
Outlier detection

8. Ensemble Learning Methods

Description:
Combines multiple models to improve performance.

Techniques:

 Bagging
 Boosting (AdaBoost)
 Random Forest

Example:
Credit risk prediction

Applications:
High-accuracy prediction systems

Summary Table
 Approach Task Example
Regression Prediction House price
Classification Decision making Spam filter
Bayesian Learning Probabilistic reasoning Medical diagnosis
Instance-Based Similarity matching Digit recognition
Clustering Pattern discovery Customer groups
Dimensionality Reduction Feature reduction Image compression
Density Estimation Probability modeling Anomaly detection
Ensemble Learning Accuracy improvement Risk prediction

Q.9 Explain different data formats used in Machine Learning.

In Machine Learning, data format refers to the structure and representation of data used for
training and testing models. Different ML problems require different data formats.

1. Structured Data

Description

Structured data is organized in tabular form with rows and columns.

Each column represents a feature, and each row represents a record.

Examples

 CSV files
 Excel sheets
 SQL tables

Example Dataset

Age Income Purchased

25 30000 No
40 60000 Yes

Usage

 Regression
 Classification

Algorithms Used

 Linear Regression
  Decision Tree
 Logistic Regression

2. Semi-Structured Data

Description

Semi-structured data does not follow a strict table structure but contains tags or keys.

Examples

 JSON
 XML
 HTML

Example (JSON)

{
"name": "Ram",
"age": 30,
"salary": 50000
}

Usage

 Web data
 APIs

Algorithms Used

 Tree-based models
 Deep Learning models

3. Unstructured Data

Description

Unstructured data has no predefined structure.

It forms the majority of real-world data.

Examples

 Text documents
 Images
 Audio
 Video
Usage

 Natural Language Processing (NLP)

 Computer Vision

Algorithms Used

 Neural Networks
 CNNs
 RNNs

4. Numerical Data

Description

Numerical data consists of numbers and can be directly processed by ML models.

Types

 Continuous (height, weight)

 Discrete (number of students)

Example

 Temperature = 35.5°C
 Marks = 85

Usage

 Regression
 Clustering

5. Categorical Data

Description

Categorical data represents labels or categories.

Types

 Nominal (Gender, Color)

 Ordinal (Low, Medium, High)

Example

 Gender: Male / Female

  Rating: 1–5

Processing

 Label Encoding
 One-Hot Encoding

6. Time-Series Data

Description

Time-series data is collected over time.

Example

Date Sales
01-01-2024 200
02-01-2024 250

Usage

 Stock prediction
 Weather forecasting

Algorithms Used

 ARIMA
 LSTM

7. Text Data

Description

Text data consists of words and sentences.

Examples

 Emails
 Reviews
 Tweets

Processing

 Tokenization
 Stop-word removal
  TF-IDF

Usage

 Sentiment analysis
 Chatbots

8. Image Data

Description

Image data is represented as pixel values.

Format

 JPEG
 PNG
 BMP

Representation

 2D or 3D matrices

Usage

 Face recognition
 Medical imaging

9. Audio Data

Description

Audio data represents sound signals.

Formats

 WAV
 MP3

Features

 Frequency
 Amplitude

Usage
  Speech recognition
 Voice assistants

Summary Table

Data Format Example ML Application

Structured CSV Classification
Semi-Structured JSON Web analytics
Unstructured Text, Image NLP, Vision
Numerical Temperature Regression
Categorical Gender Classification
Time-Series Stock price Forecasting
Text Reviews Sentiment analysis
Image Photos Face detection
Audio Voice Speech recognition

Q.10 What is Machine Learning? Explain applications of Machine Learning in data science.

Definition

Machine Learning (ML) is a branch of Artificial Intelligence (AI) that enables computer
systems to learn from data, identify patterns, and make predictions or decisions without being
explicitly programmed.

Instead of writing fixed rules, ML algorithms build models from historical data and improve
their performance as more data becomes available.

Simple Example

A spam email filter learns from past emails labeled spam or not spam and automatically
classifies new emails.

Key Features of Machine Learning

 Learns from data

 Improves with experience
 Handles large and complex datasets
 Makes predictions or decisions automatically
Applications of Machine Learning in Data Science

Machine Learning is the core tool of Data Science, used to analyze data, extract insights, and
build predictive systems.

1. Predictive Analytics

Explanation

ML models predict future outcomes based on historical data.

Example

 Sales forecasting
 Stock price prediction

Algorithms Used

 Linear Regression
 Time-series models

2. Classification and Decision Making

Explanation

ML classifies data into predefined categories.

Example

 Email spam detection

 Loan approval (Approve / Reject)

Algorithms Used

 Logistic Regression
 Decision Trees
 SVM

3. Recommendation Systems

Explanation

ML analyzes user behavior to suggest products, movies, or content.

Example
  Netflix movie recommendations
 Amazon product suggestions

Algorithms Used

 Collaborative filtering
 Matrix factorization

4. Customer Segmentation

Explanation

ML groups customers based on behavior and preferences.

Example

 Marketing campaigns
 Personalized offers

Algorithms Used

 K-Means clustering

5. Fraud Detection

Explanation

ML detects unusual patterns in transactions.

Example

 Credit card fraud detection

 Insurance fraud

Algorithms Used

 Anomaly detection
 Random Forest

6. Natural Language Processing (NLP)

Explanation

ML enables machines to understand and process human language.

Example
  Sentiment analysis
 Chatbots

Algorithms Used

 Naïve Bayes
 Deep Learning models

7. Image and Video Analysis

Explanation

ML processes images and videos to identify objects and patterns.

Example

 Face recognition
 Medical image diagnosis

Algorithms Used

 Convolutional Neural Networks (CNNs)

8. Healthcare Analytics

Explanation

ML helps in disease prediction and diagnosis.

Example

 Predicting diabetes
 Cancer detection

9. Business Intelligence and Decision Support

Explanation

ML converts raw data into actionable insights.

Example

 Demand forecasting
 Risk analysis

Summary Table
 Application Data Science Use

Predictive Analytics Forecast future trends

Classification Decision making

Recommendation Personalization

Clustering Customer grouping

Fraud Detection Risk prevention

NLP Text analysis

Image Analysis Pattern recognition

Healthcare Diagnosis support

Q.11 What is decision trees. Explain in detail?

Definition

A Decision Tree is a popular supervised machine learning algorithm used for classification
and regression tasks. It models decisions and their possible consequences in a tree-like
structure, helping to make predictions based on input features.

Structure of a Decision Tree

 Root Node:
The top node representing the entire dataset, which is split based on a feature.
 Internal Nodes (Decision Nodes):
Nodes where the data is split based on a condition on one of the features.
 Leaf Nodes (Terminal Nodes):
Final nodes that represent the output/class label (in classification) or continuous value (in
regression).

How Does a Decision Tree Work?

 The algorithm starts at the root node.

 It chooses the best feature to split the data based on a criterion.
 The dataset is divided into subsets based on this feature.
 This process continues recursively for each subset until:
o All data points belong to the same class, or
 o There are no more features to split on, or
o Some stopping criterion is met (like max depth).

Important Concepts

1. Splitting Criterion

 The quality of a split is measured to find the best feature to split the data.
 Common criteria include:

a. Information Gain (based on Entropy):

Measures how well a feature separates the data into classes.

 Entropy measures impurity or disorder in a dataset.

 Information Gain = Entropy(before split) − Weighted Entropy(after split)

b. Gini Index:
Measures the probability of misclassification in a dataset.

 Lower Gini index means better purity.

2. Entropy Formula

Entropy(S)=−∑i=1npilog⁡2piEntropy(S) = - \sum_{i=1}^n p_i \log_2 p_iEntropy(S)=−i=1∑npi

log2pi

where pip_ipi is the proportion of class iii in set SSS.

Example of Decision Tree

Suppose we want to classify whether a person will play tennis based on Weather (Sunny,
Overcast, Rainy) and Humidity (High, Normal).

Weather Humidity Play Tennis

Sunny High No
Sunny Normal Yes
Overcast High Yes
Rainy Normal Yes
Rainy High No

Steps:

1. Calculate entropy for the target variable.

2. Calculate information gain for Weather and Humidity.
 3. Choose the feature with the highest information gain to split.
4. Repeat for subsets until leaves are pure or stopping criteria met.

Advantages of Decision Trees

 Easy to understand and interpret (like flowcharts).

 Requires little data preprocessing.
 Can handle both numerical and categorical data.
 Can model nonlinear relationships.

Disadvantages of Decision Trees

 Prone to overfitting, especially with deep trees.

 Small changes in data can lead to different trees (unstable).
 Biased towards features with more levels.
 Less accurate compared to ensemble methods (Random Forest, Boosting).

Applications of Decision Trees

 Medical diagnosis
 Credit scoring
 Customer segmentation
 Fraud detection
 Marketing decision making

Summary Table

Aspect Description
Type Supervised learning (Classification/Regression)
Output Class labels or continuous values
Structure Root, internal nodes, leaf nodes
Splitting Criteria Information Gain, Gini Index
Advantages Simple, interpretable, handles various data types
Disadvantages Overfitting, instability, biased splits
Common Algorithms ID3, C4.5, CART

Q.12 Differentiate between overfitting and underfitting.

In machine learning, overfitting and underfitting are common problems that affect the
performance of models on new, unseen data. Both relate to how well a model generalizes from
the training data to real-world data.
1. Overfitting

Definition

Overfitting occurs when a machine learning model learns the training data too well, including
the noise and outliers. It fits the training data very closely, capturing even minor fluctuations,
which harms its ability to generalize to new data.

Characteristics

 High accuracy on training data

 Poor accuracy on test or validation data
 Model is too complex (e.g., too many parameters, very deep trees)

Causes

 Excessively complex models (deep decision trees, high-degree polynomials)

 Insufficient training data
 Too many features relative to the number of training samples
 Lack of regularization

Consequences

 Model captures noise as if it were a true pattern

 Poor performance on unseen data (low generalization)
 Unstable predictions

Solutions

 Use simpler models

 Apply regularization techniques (L1, L2)
 Use cross-validation to tune hyperparameters
 Prune decision trees or limit model complexity
 Increase training data size

2. Underfitting

Definition

Underfitting occurs when a machine learning model is too simple to capture the underlying
pattern in the data. The model cannot fit the training data well, resulting in poor performance
even on the training set.

Characteristics

 Low accuracy on training data

  Low accuracy on test data
 Model is too simple (e.g., linear model for non-linear data)

Causes

 Overly simple models

 Insufficient features or poor feature selection
 Inadequate training (too few iterations, too much regularization)

Consequences

 Model fails to learn important patterns

 Both training and test errors are high
 Systematic bias in predictions

Solutions

 Use more complex models

 Add relevant features or perform feature engineering
 Reduce regularization if over-applied
 Train longer or improve training procedures

Comparison Table

Aspect Overfitting Underfitting

Model Complexity Too complex Too simple

Training Accuracy Very high Low

Test Accuracy Low Low

Error on Training
Low High
Data

Error on Test Data High High

Model too simple, insufficient

Cause Learning noise, too many parameters
learning

Generalization Poor Poor

Solution Simplify model, regularize, get more Increase complexity, add features
 Aspect Overfitting Underfitting

data

Visual Explanation

 Overfitting: The model curve passes through almost every training point, including
noise, leading to a jagged, overly complex function.
 Underfitting: The model curve is too simple (e.g., a straight line for non-linear data),
failing to capture the data trend.

Summary

 Overfitting happens when the model memorizes the training data and fails to generalize.
 Underfitting happens when the model fails to learn the patterns in the data, both on
training and new data.

Balancing the two is crucial for building an effective machine learning model.

Q.13 Write a detail note naive Bayes linear models?

Definition

Naive Bayes is a probabilistic classifier based on Bayes’ theorem with the “naive”
assumption that features are conditionally independent given the class label. Despite this
simplifying assumption, it performs remarkably well in many practical applications.

Bayes’ Theorem

Bayes’ theorem relates the conditional and marginal probabilities of random events:

P(C∣X)=P(X∣C)×P(C)P(X)P(C|X) = \frac{P(X|C) \times P(C)}{P(X)}P(C∣X)=P(X)P(X∣C)×P(C)

 P(C∣X)P(C|X)P(C∣X): Posterior probability of class CCC given features XXX.

 P(X∣C)P(X|C)P(X∣C): Likelihood of features XXX given class CCC.
 P(C)P(C)P(C): Prior probability of class CCC.
 P(X)P(X)P(X): Evidence (probability of features).

Naive Bayes Classifier

The classifier assigns a class CkC_kCk to an input X=(x1,x2,...,xn)X = (x_1, x_2, ..., x_n)X=(x1
,x2,...,xn) by maximizing the posterior probability:
C^=arg⁡max⁡CkP(Ck)∏i=1nP(xi∣Ck)\hat{C} = \arg\max_{C_k} P(C_k) \prod_{i=1}^n P(x_i |
C_k)C^=argCkmaxP(Ck)i=1∏nP(xi∣Ck)

Here, the naive assumption is that each feature xix_ixi is independent given class CkC_kCk, so
likelihood factorizes.

Types of Naive Bayes Classifiers

 Gaussian Naive Bayes: Assumes features follow a normal distribution (for continuous
data).
 Multinomial Naive Bayes: For discrete counts, e.g., text classification with word
frequencies.
 Bernoulli Naive Bayes: For binary/boolean features.

Advantages of Naive Bayes

 Simple and fast to train and predict.

 Works well with high-dimensional data.
 Requires relatively small training data.
 Performs well for text classification and spam filtering.

Limitations

 Strong assumption of feature independence (rarely true in real data).

 Can perform poorly if features are highly correlated.

Applications

 Email spam detection

 Sentiment analysis
 Document classification
 Medical diagnosis

2. Linear Models

Definition

Linear models are a class of models that assume a linear relationship between input features
and the output. They are widely used for both regression and classification tasks.

Common Linear Models

a) Linear Regression

Used for predicting a continuous output variable based on linear combination of input features:
y=β0+β1x1+β2x2+⋯+βnxn+ϵy = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \cdots + \beta_n x_n +
\epsilony=β0+β1x1+β2x2+⋯+βnxn+ϵ

Where:

 yyy: Dependent variable

 xix_ixi: Independent variables (features)
 βi\beta_iβi: Coefficients (weights)
 ϵ\epsilonϵ: Error term

b) Logistic Regression

Used for binary classification problems, models the probability of the target class using the
logistic function:

P(y=1∣X)=11+e−(β0+β1x1+⋯+βnxn)P(y=1|X) = \frac{1}{1 + e^{-(\beta_0 + \beta_1 x_1 +

\cdots + \beta_n x_n)}}P(y=1∣X)=1+e−(β0+β1x1+⋯+βnxn)1

It estimates the log-odds as a linear function of inputs.

Training Linear Models

 Objective is to minimize a loss function (e.g., mean squared error for regression, log-loss
for classification).
 Optimization techniques like Gradient Descent are used to find the best parameters
β\betaβ.

Advantages of Linear Models

 Simple and interpretable

 Computationally efficient
 Good baseline models
 Work well when data relationships are approximately linear

Limitations

 Cannot capture complex, non-linear relationships

 Sensitive to outliers
 May underfit complex data

Applications

 Predicting house prices

 Customer churn prediction
 Credit scoring
  Medical risk assessment

Relationship Between Naive Bayes and Linear Models

 Naive Bayes can be interpreted as a linear classifier under certain conditions.

 Both models make simplifying assumptions: Naive Bayes assumes feature independence;
linear models assume linear separability.
 Naive Bayes works well with small data and high-dimensional data, while linear models
can be more flexible with feature engineering.

Summary Table

Aspect Naive Bayes Linear Models

Type Probabilistic classifier Regression and classification

Features independent given

Assumption Linear relationship between input and output
class

Continuous (regression) or class probabilities

Output Class probabilities
(logistic regression)

Model
Simple, fast Simple, interpretable
Complexity

Data Handles small and high-

Needs sufficient data for good fit
Requirements dimensional data

Text classification, spam

Use Cases Regression, binary classification
detection

Feature independence
Limitations Cannot capture non-linear patterns
assumption

Q.14 What is support vector machine. Discuss in detail?

Definition

Support Vector Machine (SVM) is a powerful supervised machine learning algorithm used
mainly for classification and regression tasks. It works by finding the optimal hyperplane that
best separates data points of different classes in a high-dimensional space.
Core Idea of SVM

 Given labeled training data, SVM tries to find a decision boundary (hyperplane) that
maximizes the margin between two classes.
 The margin is the distance between the hyperplane and the nearest data points from each
class, called support vectors.
 Maximizing the margin helps improve the model's ability to generalize to unseen data.

Components of SVM

1. Hyperplane

 In a 2D space, a hyperplane is a line that separates the classes.

 In higher dimensions, it becomes a plane or hyperplane.

2. Margin

 Margin is the gap between the hyperplane and the closest points from each class.
 SVM finds the hyperplane with the maximum margin.

3. Support Vectors

 Support vectors are the data points that lie closest to the decision boundary.
 These points directly influence the position and orientation of the hyperplane.

Working of SVM

1. Linearly Separable Case:

o If data is linearly separable, SVM finds a hyperplane that perfectly separates the
two classes with the maximum margin.
2. Non-Linearly Separable Case:
o When data is not linearly separable, SVM uses:
 Soft Margin: Allows some misclassifications to avoid overfitting.
 Kernel Trick: Transforms data into a higher-dimensional space where it
becomes linearly separable.

Kernel Trick

 Kernels enable SVM to handle non-linear classification by implicitly mapping input

features into higher-dimensional spaces without computing the coordinates explicitly.
 Common kernel functions include:
o Linear Kernel: For linearly separable data.
o Polynomial Kernel: For polynomial decision boundaries.
o Radial Basis Function (RBF) Kernel / Gaussian Kernel: For complex non-
linear boundaries.
o Sigmoid Kernel: Similar to neural networks activation.
Mathematical Formulation

 SVM solves the optimization problem:

min⁡w,b12∥w∥2\min_{\mathbf{w}, b} \frac{1}{2} \|\mathbf{w}\|^2w,bmin21∥w∥2

subject to

yi(w⋅xi+b)≥1,∀iy_i(\mathbf{w} \cdot \mathbf{x}_i + b) \geq 1, \quad \forall iyi(w⋅xi+b)≥1,∀i

where

 w\mathbf{w}w is the weight vector (normal to the hyperplane)

 bbb is the bias term
 yiy_iyi are class labels (+1 or -1)
 xi\mathbf{x}_ixi are input feature vectors

Advantages of SVM

 Effective in high-dimensional spaces and when the number of features exceeds the
number of samples.
 Works well with clear margin of separation.
 Uses only support vectors, making it memory efficient.
 Flexible through kernel functions for linear and non-linear problems.

Disadvantages of SVM

 Not suitable for very large datasets because of high training time complexity.
 Choosing the right kernel and parameters (like C and gamma) can be difficult.
 Less effective on noisy data with overlapping classes.
 SVM outputs are not probabilistic by default (though extensions exist).

Applications of SVM

 Text classification (e.g., spam detection)

 Image recognition (e.g., face detection)
 Bioinformatics (e.g., protein classification)
 Handwriting recognition
 Fault detection in engineering systems

Summary Table

Aspect Description
Type Supervised learning (classification/regression)
Decision Boundary Hyperplane maximizing margin
 Aspect Description
Key Components Support vectors, margin, kernel functions
Handles Linearly and non-linearly separable data
Kernel Types Linear, Polynomial, RBF, Sigmoid
Advantages Effective in high dimensions, flexible kernels
Disadvantages Computationally intensive, parameter tuning needed
Common Applications Text, image, bioinformatics, handwriting

Q.15 Explain artificial neural network based on perception concept with diagram.

The Perceptron is the simplest type of artificial neural network and is the fundamental building
block of more complex networks. It was introduced by Frank Rosenblatt in 1958.

 It is a binary classifier that decides whether an input belongs to one class or another.
 It mimics the behavior of a biological neuron.

Structure of a Perceptron

A perceptron consists of:

1. Input Layer:
Receives multiple input signals x1,x2,...,xnx_1, x_2, ..., x_nx1,x2,...,xn.
2. Weights:
Each input is multiplied by a corresponding weight w1,w2,...,wnw_1, w_2, ..., w_nw1,w2
,...,wn that represents the importance of the input.
3. Summation Function:
Calculates the weighted sum of inputs:

S=∑i=1nwixi+bS = \sum_{i=1}^n w_i x_i + bS=i=1∑nwixi+b

where bbb is the bias term.

4. Activation Function:
Applies a function to the sum to produce the output.
For a simple perceptron, a step function is used:

output={1if S≥00if S<0output = \begin{cases} 1 & \text{if } S \geq 0 \\ 0 & \text{if } S <
0 \end{cases}output={10if S≥0if S<0

Working of a Perceptron

 The inputs are multiplied by their weights.

 The weighted inputs are summed with the bias.
  The activation function determines the output (usually binary).
 The perceptron learns by adjusting weights to minimize classification error on the
training set.

Diagram of a Perceptron

x1 ---- w1 ----\
\
x2 ---- w2 ------> [ Σ (weighted sum) ] --> Activation --> Output (0 or 1)
/
... /
b (bias)

 Inputs x1,x2,...,xnx_1, x_2, ..., x_nx1,x2,...,xn are multiplied by weights

w1,w2,...,wnw_1, w_2, ..., w_nw1,w2,...,wn.
 The weighted sum plus bias bbb is passed to the activation function.
 The output is generated based on the activation.

From Perceptron to Artificial Neural Network (ANN)

 An ANN is a network of interconnected perceptrons (neurons) arranged in layers:

o Input Layer: Receives raw input data.
o Hidden Layer(s): Perform intermediate computations.
o Output Layer: Produces the final output.
 The perceptron is the basic unit; combining many perceptrons with nonlinear activation
functions creates powerful networks that can solve complex problems.

Summary

Term Explanation
Perceptron Basic binary classifier neuron
Inputs Features from dataset
Weights Importance of each input
Bias Threshold adjuster
Summation Weighted sum of inputs + bias
Activation Step function for output
Output Binary classification (0 or 1)

Applications of Perceptron

 Simple binary classification problems

 Foundation for modern deep neural networks
 Pattern recognition
Q.16 Describe mutli-layer neural network. Explain why back propagation algorithm is
required.

A Multi-Layer Neural Network (MLNN), also called a Multi-Layer Perceptron (MLP), is an

extension of the simple perceptron. It consists of multiple layers of neurons arranged in a
sequence:

 Input Layer: Receives raw input features.

 One or more Hidden Layers: Intermediate layers that process inputs using weighted
connections and nonlinear activation functions.
 Output Layer: Produces final predictions or classifications.

Structure of MLNN

Input Layer → Hidden Layer(s) → Output Layer

 Each neuron in a layer is connected to every neuron in the next layer.

 Neurons perform a weighted sum of inputs, add a bias, and apply a nonlinear activation
function (like sigmoid, ReLU).

Why Use Multi-Layer Networks?

 Single-layer perceptrons can only solve linearly separable problems.

 MLNNs can model complex, non-linear relationships in data.
 Adding hidden layers allows the network to learn hierarchical feature representations.

Components of MLNN

Component Description

Neurons Basic units performing computations

Weights Parameters controlling strength of connections

Bias Offset to activation function input

Activation Function Introduces non-linearity (e.g., sigmoid, ReLU)

Layers Input, Hidden (one or more), Output

Need for Backpropagation Algorithm

Problem in Training MLNN

  To train an MLNN, we need to adjust the weights and biases to minimize the error
between predicted output and actual output.
 Unlike single-layer perceptrons, which have a straightforward learning rule, multi-layer
networks require a way to efficiently compute gradients of the error with respect to
each weight in all layers.
 This is complex because the error depends on weights across multiple layers connected
non-linearly.

What is Backpropagation?

Backpropagation is a supervised learning algorithm used to train multi-layer neural

networks. It is an application of the chain rule of calculus to efficiently compute the gradient of
the loss function with respect to each weight.

How Backpropagation Works (Step-by-step):

1. Forward Pass:
o Input data passes through the network layer-by-layer.
o Outputs of each neuron are computed using current weights.
o The network produces a prediction (output).
2. Calculate Error:
o Compute the difference between predicted output and actual target using a loss
function (e.g., Mean Squared Error).
3. Backward Pass:
o The error is propagated backward through the network.
o Gradients of the loss function with respect to weights are calculated using the
chain rule.
o This tells how much each weight contributed to the error.
4. Update Weights:
o Weights are updated using Gradient Descent or similar optimization techniques:

w←w−η∂E∂ww \leftarrow w - \eta \frac{\partial E}{\partial w}w←w−η∂w∂E

where η\etaη is the learning rate, and ∂E∂w\frac{\partial E}{\partial w}∂w∂E is the
gradient of error w.r.t weight.

5. Repeat:
o Steps 1–4 are repeated for many iterations (epochs) until error converges or is
sufficiently low.

Mathematical Summary

 Let output of neuron jjj in layer lll be aj(l)a_j^{(l)}aj(l).

 Error EEE is a function of output.
 Using chain rule, compute gradient:
∂E∂wij(l)=δj(l)×ai(l−1)\frac{\partial E}{\partial w_{ij}^{(l)}} = \delta_j^{(l)} \times a_i^{(l-
1)}∂wij(l)∂E=δj(l)×ai(l−1)

where δj(l)\delta_j^{(l)}δj(l) is the error term for neuron jjj in layer lll.

Importance of Backpropagation

 Allows efficient training of deep networks with many layers.

 Computes gradients without redundant calculations.
 Enables networks to learn complex functions.
 Is the backbone of most modern deep learning.

Summary Table

Aspect Description

MLNN Network with multiple layers of neurons

Purpose of
Efficiently calculate gradients for all weights
Backpropagation

Forward pass → Error calculation → Backward pass → Weight

Working
update

Optimization method Gradient Descent or variants (Adam, RMSProp)

Benefit Enables learning of complex, nonlinear patterns

Diagram of Multi-Layer Neural Network and Backpropagation

Input Layer --> Hidden Layer(s) --> Output Layer

| | |
Forward pass --> Calculate error --> Backward pass
(Adjust weights via gradient descent)

Q.17 What is Functional Link Artificial Neural Network (FLANN)? Explain its merits over
other ANNs

Functional Link Artificial Neural Network (FLANN) is a type of single-layer feedforward

neural network that enhances the input features using functional expansions before feeding
them into the network. It was introduced to improve the network's ability to model nonlinear
relationships without adding hidden layers.
Key Idea

 Instead of having multiple hidden layers (as in traditional multilayer ANNs), FLANN
expands the input vector using nonlinear functions (like trigonometric, polynomial,
Chebyshev, or power series).
 This expansion generates a higher-dimensional input space where patterns become
more linearly separable.
 The network then uses a simple linear model on this expanded input.

Structure of FLANN

 Input Layer: Original input features.

 Functional Expansion Block: Transforms input features into an expanded set using
nonlinear functions.
 Output Layer: Single-layer perceptron that performs linear mapping on the expanded
inputs.

Working of FLANN

1. Original input vector X=[x1,x2,...,xn]X = [x_1, x_2, ..., x_n]X=[x1,x2,...,xn] is

transformed to an expanded vector X′=[ϕ1(x),ϕ2(x),...,ϕm(x)]X' = [\phi_1(x), \phi_2(x),
..., \phi_m(x)]X′=[ϕ1(x),ϕ2(x),...,ϕm(x)], where ϕi(x)\phi_i(x)ϕi(x) are nonlinear
functional expansions.
2. Weights are applied to the expanded inputs.
3. The output is computed as a weighted sum of these expanded inputs.
4. Training involves adjusting the weights using algorithms like least mean squares (LMS)
or gradient descent.

Merits of FLANN over Other ANNs

Merit Explanation
FLANN uses a single-layer network, avoiding complex multilayer
Simpler Architecture
structures. Easier to design and train.
Due to the absence of hidden layers, training converges faster
Faster Training
compared to multilayer ANNs.
Effective Nonlinear Functional expansion maps input data into a higher-dimensional
Modeling nonlinear space, enabling linear separability without hidden layers.
Lower Computational Less computational resources are needed as there are fewer weights
Cost and simpler topology.
Training is less prone to getting stuck in local minima compared to
Avoids Local Minima
multilayer backpropagation networks.
Good for Function FLANN can approximate complex nonlinear functions effectively
Approximation with fewer parameters.
Easier Implementation Simpler learning algorithms (like LMS) can be used, simplifying
 Merit Explanation
coding and deployment.
Suitable for Real-Time Faster training and simpler computations make FLANN suitable for
Applications real-time systems.

Comparison with Traditional Multilayer ANNs

Feature FLANN Multilayer ANN

Single-layer with functional Multiple layers (input, hidden,
Architecture
expansion output)
Training Time Faster Slower due to backpropagation
Computational Complexity Lower Higher due to many weights
Ability to Model Achieved by multiple nonlinear
Achieved by input expansion
Nonlinearity layers
Higher, especially with many
Risk of Overfitting Lower, due to simpler model
layers
Implementation
Simpler More complex
Complexity

Applications of FLANN

 Pattern recognition
 Signal processing
 Time series prediction
 Function approximation
 Control systems

Q.18 Explain Single Layer Neural Network and What is Activation Function? Explain with
a suitable example

A Single Layer Neural Network is the simplest form of an artificial neural network consisting
of:

 An input layer of neurons that receives input features.

 An output layer of neurons that produces the network’s output.

Unlike multi-layer networks, this network contains only one layer of weights connecting inputs
directly to outputs — hence the name ―single layer.‖ It has no hidden layers.

Structure:

 Inputs: x1,x2,...,xnx_1, x_2, ..., x_nx1,x2,...,xn

 Weights: w1,w2,...,wnw_1, w_2, ..., w_nw1,w2,...,wn associated with each input
  Bias: A constant term bbb added to the weighted sum
 Output: Computed based on the weighted sum and an activation function

Working of a Single Layer Neural Network

1. Weighted Sum Calculation:

Each input is multiplied by its corresponding weight, and the products are summed along
with the bias:

S=∑i=1nwixi+bS = \sum_{i=1}^n w_i x_i + bS=i=1∑nwixi+b

2. Activation Function Application:

The sum SSS is passed through an activation function to produce the final output.

Limitations of Single Layer Neural Networks

 Can only solve linearly separable problems (e.g., AND, OR logic gates).
 Cannot solve problems that require modeling complex, non-linear relationships (e.g.,
XOR problem).
 Lack the ability to learn hierarchical features since there are no hidden layers.

What is an Activation Function?

Definition

An Activation Function is a mathematical function applied to the output of a neuron’s weighted

sum before passing it forward. It determines whether the neuron should be ―activated‖ or not,
introducing non-linearity into the network.

Without an activation function, the neural network would behave like a linear regression model,
regardless of the number of layers.

Types of Activation Functions

1. Step Function (Binary Threshold)

 Outputs 1 if the input exceeds a threshold; otherwise, outputs 0.

 Used in the simplest perceptron models.

output={1,if S≥00,if S<0output = \begin{cases} 1, & \text{if } S \geq 0 \\ 0, & \text{if } S < 0
\end{cases}output={1,0,if S≥0if S<0

2. Sigmoid Function

 Outputs values between 0 and 1, useful for probability interpretation.

σ(S)=11+e−S\sigma(S) = \frac{1}{1 + e^{-S}}σ(S)=1+e−S1

3. Hyperbolic Tangent (tanh)

 Outputs values between -1 and 1, centered around zero.

tanh⁡(S)=eS−e−SeS+e−S\tanh(S) = \frac{e^S - e^{-S}}{e^S + e^{-S}}tanh(S)=eS+e−SeS−e−S

4. ReLU (Rectified Linear Unit)

 Outputs zero if input is negative; otherwise, outputs the input.

ReLU(S)=max⁡(0,S)ReLU(S) = \max(0, S)ReLU(S)=max(0,S)

Why Activation Functions are Important

 Non-linearity: Enables the network to learn complex patterns.

 Control output: Limits output range, making training more stable.
 Enables deep networks: Without activation functions, stacking layers wouldn’t add
power to the model.

Example: Single Layer Neural Network for AND Gate

Input x1x_1x1 Input x2x_2x2 Output (AND)

0 0 0

0 1 0

1 0 0

1 1 1

Step 1: Assign weights and bias

Choose weights w1=1w_1 = 1w1=1, w2=1w_2 = 1w2=1, and bias b=−1.5b = -1.5b=−1.5.

Step 2: Calculate weighted sum

S=w1x1+w2x2+bS = w_1 x_1 + w_2 x_2 + bS=w1x1+w2x2+b

Step 3: Apply step activation function

output={1if S≥00if S<0output = \begin{cases} 1 & \text{if } S \geq 0 \\ 0 & \text{if } S < 0
\end{cases}output={10if S≥0if S<0

Step 4: Check for all inputs

x1x_1x1 x2x_2x2 S=w1x1+w2x2+bS = w_1x_1 + w_2x_2 + bS=w1x1+w2x2+b Output

0 0 0 + 0 - 1.5 = -1.5 0

0 1 0 + 1 - 1.5 = -0.5 0

1 0 1 + 0 - 1.5 = -0.5 0

1 1 1 + 1 - 1.5 = 0.5 1

The network correctly models the AND function.

Q.19 Explain Geometric Model and Probabilistic Model with suitable examples.

Definition

The Geometric Model represents data as points in a geometric (feature) space. Classification
is performed by drawing geometric boundaries such as lines, planes, or curves to separate
different classes.

Each data instance is treated as a point in an n-dimensional space, where each dimension
corresponds to a feature.

Key Idea

 Objects belonging to different classes occupy different regions in the feature space.
 A classifier finds a decision boundary that separates these regions.
 Decision boundaries can be:
o Linear (line, plane, hyperplane)
o Non-linear (curves, complex surfaces)

Examples of Geometric Models

1. Perceptron

 Uses a straight line (in 2D) or hyperplane (in higher dimensions) to separate classes.
 Works only for linearly separable data.

2. Support Vector Machine (SVM)

 Finds an optimal hyperplane with maximum margin between classes.

 Can create non-linear boundaries using kernel functions.

3. k-Nearest Neighbors (k-NN)

 Classifies a point based on geometric distance (Euclidean, Manhattan) from neighbors.

Example of Geometric Model

Consider classifying fruits using:

 Feature 1: Weight
 Feature 2: Sweetness

Each fruit is a point on a 2D graph.

 Apples and oranges form clusters in different regions.

 A straight line or curve is drawn to separate them.
 New fruit is classified based on which side of the boundary it falls.

Advantages of Geometric Model

 Simple and intuitive visualization

 Works well for spatial and distance-based problems
 Efficient for linearly separable data

Limitations

 Performs poorly with overlapping data

 Not suitable when uncertainty or probability is important
 Sensitive to noise and outliers

2. Probabilistic Model

Definition

The Probabilistic Model represents uncertainty using probability theory. It assumes that data
is generated by underlying probability distributions and makes predictions based on likelihoods
and posterior probabilities.
Key Idea

 Each class has a probability distribution.

 Classification is done by calculating:

P(Class∣Data)P(Class | Data)P(Class∣Data)

 The class with the highest probability is chosen.

Core Concept: Bayes’ Theorem

P(C∣X)=P(X∣C) P(C)P(X)P(C|X) = \frac{P(X|C)\,P(C)}{P(X)}P(C∣X)=P(X)P(X∣C)P(C)

Where:

 P(C∣X)P(C|X)P(C∣X): Posterior probability

 P(X∣C)P(X|C)P(X∣C): Likelihood
 P(C)P(C)P(C): Prior probability
 P(X)P(X)P(X): Evidence

Examples of Probabilistic Models

1. Naive Bayes Classifier

 Assumes features are conditionally independent.

 Widely used in text classification and spam detection.

2. Gaussian Mixture Model (GMM)

 Models data as a mixture of multiple Gaussian distributions.

 Used in clustering and density estimation.

3. Hidden Markov Model (HMM)

 Used for sequential data like speech recognition.

Example of Probabilistic Model

Email Spam Detection

 Feature: Presence of words like ―free‖, ―offer‖.

 Calculate probability that an email is spam given these words.
 If:
P(Spam∣Email)>P(Not Spam∣Email)P(\text{Spam} | \text{Email}) > P(\text{Not Spam} |
\text{Email})P(Spam∣Email)>P(Not Spam∣Email)

→ classify as spam.

Advantages of Probabilistic Model

 Handles uncertainty and noise effectively

 Provides confidence levels for predictions
 Works well with overlapping classes
 Strong theoretical foundation

Limitations

 Requires assumptions about data distributions

 Computation can be complex
 Performance depends on correctness of probability estimates

Comparison Between Geometric and Probabilistic Models

Aspect Geometric Model Probabilistic Model

Data Representation Points in feature space Probability distributions

Decision Method Geometric boundaries Maximum probability

Handles Uncertainty No Yes

Examples Perceptron, SVM, k-NN Naive Bayes, GMM, HMM

Overlapping Data Poor handling Good handling

Output Class label Class + probability

Q.20 Discuss the recent trends in various learning techniques of machine learning.

Machine Learning continues to grow rapidly. Traditional supervised and unsupervised learning
are now being expanded and blended with advanced techniques to deal with big data,
complexity, real-world uncertainty, and limited labeled data. The key trends include:

1. Deep Learning (DL)

Overview
Deep Learning uses neural networks with many layers to automatically learn hierarchical
feature representations from raw data.

Recent Advances

 Convolutional Neural Networks (CNNs): Highly successful in image recognition,

object detection, medical imaging, etc.
 Recurrent Neural Networks (RNNs)/LSTM/GRU: Used for sequence modeling —
language, speech, time-series.
 Transformers: Groundbreaking models (e.g., BERT, GPT) outperform previous
methods in NLP and are expanding into vision and reasoning.

Key Innovations

 Self-Attention Mechanism: Learns relationships within data without sequence order

limitations.
 Pre-trained Models: Trained on massive datasets and fine-tuned for specific tasks,
reducing data needs.
 Generative Models (GANs, VAEs): Learn to generate realistic data — images, music,
even human-like text.

Applications

 Computer Vision, Natural Language Processing (NLP), speech recognition, autonomous

driving, game AI.

2. Reinforcement Learning (RL)

Overview

RL focuses on learning through interaction — agents take actions to maximize cumulative

reward.

Recent Trends

 Deep Reinforcement Learning: Combines RL with Deep Learning (e.g., Deep Q-

Networks).
 Multi-Agent RL: Multiple autonomous agents learning and interacting.
 Model-Based RL: Learning an internal model of the environment for better planning.

Breakthroughs

 AlphaGo/AlphaZero — mastering strategic games.

 Robotics — continuous control and manipulation learning.

Applications
Autonomous vehicles, robotics, recommendation systems, resource allocation.

3. Transfer Learning and Fine-Tuning

Overview

Transfer learning uses a pre-trained model on a large dataset and adapts it for a specific task
with far less labeled data.

Why It’s Trending

 Reduces data scarcity problems.

 Speeds up training.
 Improves performance when labeled data is rare.

Examples

 Fine-tuning BERT for sentiment analysis.

 Using ImageNet-trained CNN models for medical image classification.

4. Semi-Supervised and Self-Supervised Learning

Semi-Supervised Learning

Blends labeled and unlabeled data to build better models when labeling is expensive.

Self-Supervised Learning

Learns hidden structure from unlabeled data by creating pretext tasks:

 Predict masked words (BERT)

 Predict missing parts of images (MAE, SimCLR)

Impact

 Dramatically reduces reliance on human labeling.

 Pushed state-of-the-art in NLP and vision.

5. Meta-Learning (Learning to Learn)

Overview

Meta-learning trains models that adapt quickly to new tasks with minimal data.

Key Techniques
  Model-Agnostic Meta-Learning (MAML)
 Optimization and metric-based meta-learners

Why Important

Useful for few-shot or zero-shot learning — learning with very little or no labeled examples.

Applications

Personalized AI, medical diagnosis, adaptive robotics.

6. Federated Learning

Overview

Training ML models across distributed devices (e.g., mobile phones) while keeping data local
for privacy.

Why It Matters

 Protects user privacy

 Reduces central data storage
 Enables collaborative learning from distributed edge devices

Applications

Predictive keyboards, healthcare, finance.

7. Explainable AI (XAI)

Overview

ML systems increasingly need to be interpretable and transparent, especially in critical

domains.

Techniques

 SHAP, LIME, Integrated Gradients

 Rule extraction, attention visualization

Why Trending

Regulations (GDPR), ethical AI, fairness, accountability.

8. Probabilistic & Bayesian Learning

Overview

Integrates probability distributions for handling uncertainty and confidence estimation.

Recent Uses

 Bayesian neural networks

 Uncertainty quantification for safety-critical systems

Applications

Medical prognosis, anomaly detection, risk modeling.

9. Graph Neural Networks (GNNs)

Overview

Designed for graph-structured data (nodes + edges).

Why It’s Useful

Many real datasets are naturally graphical: social networks, molecules, transport networks.

Applications

Molecule property prediction, recommendation systems, fraud detection.

10. Ensemble and Hybrid Models

Overview

Combining multiple models to improve robustness and accuracy:

 Random Forests, Gradient Boosting (XGBoost, LightGBM)

 Hybrid DL + probabilistic components

Recent Trends

Boosted decision trees + neural embeddings for tabular data.

11. Edge and TinyML

Trend

Deploying ML models on resource-limited devices (microcontrollers, IoT).

Techniques

Model compression, quantization, pruning.

Why It’s Relevant

Real-time inference without cloud dependence, lower latency and privacy protection.

12. Ethical and Responsible AI

Focus Areas

 Bias mitigation
 Fairness
 Data governance
 Privacy preservation

Why Trending

AI systems increasingly operate in sensitive domains: healthcare, hiring, finance.

Summary of Trends and Why They Matter

Trend Key Benefit Why It’s Important

Deep Learning Learns complex patterns Powers vision & language models

Reinforcement Learning Decision optimization Suits autonomous agents

Transfer Learning Saves labeled data Adaptability to tasks

Self/Semi-Supervised Uses unlabeled data Scalability

Meta-Learning Fast task adaptation Few-shot learning

Federated Learning Privacy-preserving Decentralized ML

Explainable AI Transparency Trust & compliance

Bayesian Learning Uncertainty modeling Safety and reliability

Graph Neural Networks Relation learning Networked data tasks

Ensemble Models Improved accuracy Robust predictions

Edge/TinyML Low-power inference Real-world deployment

Ethical AI Responsible decisions Trustworthy systems

Q.21 Explain learning techniques in ML and explain in detail.

Machine Learning (ML) is a branch of Artificial Intelligence that enables computers to learn
from data and improve performance without being explicitly programmed. The method by
which a machine learns from data is called a learning technique. Based on the nature of
learning and availability of labeled data, ML techniques are broadly classified into different
types.

1. Supervised Learning

Definition

Supervised Learning is a learning technique where the model is trained using labeled data.
Each training example consists of an input and a corresponding correct output (label).

Working

 The model learns a mapping from input to output.

 During training, predicted outputs are compared with actual outputs.
 Errors are minimized using optimization techniques.

Types

 Classification: Output is categorical

Example: Email spam detection, disease diagnosis
 Regression: Output is continuous
Example: House price prediction, temperature forecasting

Common Algorithms

 Linear Regression
 Logistic Regression
 Decision Trees
 Support Vector Machine (SVM)
 k-Nearest Neighbors (k-NN)
 Naive Bayes

Advantages
  High accuracy when sufficient labeled data is available
 Easy to evaluate performance

Disadvantages

 Requires large labeled datasets

 Labeling is time-consuming and costly

2. Unsupervised Learning

Definition

Unsupervised Learning works with unlabeled data. The system tries to discover hidden
patterns, structures, or relationships in the data.

Working

 No predefined output labels

 Algorithms group or organize data based on similarity or structure

Types

 Clustering: Grouping similar data points

 Association Rule Mining: Discovering relationships between variables
 Dimensionality Reduction: Reducing number of features

Common Algorithms

 K-Means Clustering
 Hierarchical Clustering
 DBSCAN
 Principal Component Analysis (PCA)
 Apriori Algorithm

Examples

 Customer segmentation
 Market basket analysis
 Image compression

Advantages

 No labeled data required

 Useful for exploratory data analysis

Disadvantages
  Difficult to evaluate results
 Interpretation may be subjective

3. Semi-Supervised Learning

Definition

Semi-Supervised Learning uses a small amount of labeled data and a large amount of
unlabeled data to train models.

Working

 Initial model is trained on labeled data

 Model predictions are used to label unlabeled data iteratively

Examples

 Web page classification

 Speech recognition systems

Advantages

 Reduces labeling cost

 Improves accuracy compared to unsupervised learning

Disadvantages

 Risk of propagating incorrect labels

4. Reinforcement Learning

Definition

Reinforcement Learning (RL) is a learning technique where an agent learns by interacting

with an environment and receives rewards or penalties based on its actions.

Components

 Agent – learner or decision-maker

 Environment – everything the agent interacts with
 Action – moves taken by the agent
 Reward – feedback from environment
 Policy – strategy followed by agent

Working
  Agent selects actions
 Receives reward or penalty
 Updates policy to maximize cumulative reward

Common Algorithms

 Q-Learning
 SARSA
 Deep Q-Networks (DQN)

Applications

 Game playing (Chess, Go)

 Robotics
 Autonomous vehicles

Advantages

 Suitable for sequential decision problems

 Learns optimal strategies

Disadvantages

 Requires large training time

 Computationally expensive

5. Self-Supervised Learning

Definition

A subset of unsupervised learning where the system automatically generates labels from data
itself.

Examples

 Masked word prediction in NLP (BERT)

 Image rotation prediction

Importance

 Reduces dependency on human labeling

 Enables large-scale learning

6. Online and Batch Learning

Batch Learning
  Model trained using entire dataset at once
 Suitable for static datasets

Online Learning

 Model learns continuously as new data arrives

 Suitable for streaming data

Comparison of Learning Techniques

Learning Technique Labeled Data Interaction Example Applications

Supervised Yes No Spam detection
Unsupervised No No Customer segmentation
Semi-Supervised Partial No Speech recognition
Reinforcement No Yes Robotics
Self-Supervised Auto No NLP pretraining

Q.22 How PCA is used for dimensionality reduction in machine learning?

Principal Component Analysis (PCA) is an unsupervised statistical technique that

transforms a high-dimensional dataset into a lower-dimensional space by creating new features
called principal components.

 These components are:

o Linear combinations of original features
o Uncorrelated with each other
o Ordered by the amount of variance they capture

Why Dimensionality Reduction is Needed

High-dimensional data causes problems such as:

 Curse of dimensionality
 High computational cost
 Overfitting
 Difficulty in visualization
 Redundant and correlated features

PCA helps by:

 Removing redundancy
 Retaining maximum information
 Improving model performance and speed
How PCA Performs Dimensionality Reduction (Step-by-Step)

Step 1: Standardize the Data

PCA is sensitive to scale, so features are standardized:

z=x−μσz = \frac{x - \mu}{\sigma}z=σx−μ

This ensures each feature has:

 Mean = 0
 Standard deviation = 1

Step 2: Compute the Covariance Matrix

The covariance matrix shows how features vary together:

Covariance Matrix=1n−1XTX\text{Covariance Matrix} = \frac{1}{n-1} X^T

XCovariance Matrix=n−11XTX

 Large covariance → strong relationship

 PCA aims to remove correlated dimensions

Step 3: Compute Eigenvalues and Eigenvectors

 Eigenvectors → directions of maximum variance (principal components)

 Eigenvalues → amount of variance in each direction

Step 4: Sort Eigenvectors by Eigenvalues

 Eigenvectors are ranked in descending order of eigenvalues

 First principal component (PC1) captures maximum variance
 Next PCs capture decreasing variance

Step 5: Select Top k Principal Components

 Choose top k eigenvectors that explain most variance

 This reduces dimensionality from n → k

Example:

 Original features = 100

 Selected components = 10
 Dimensionality reduced by 90%

Step 6: Project Data onto New Feature Space

Original data is projected onto the selected principal components:

Xreduced=X⋅WX_{reduced} = X \cdot WXreduced=X⋅W

where WWW contains selected eigenvectors.

Simple Example

Suppose we have student data with:

 Height
 Weight
 Age

Height and weight are highly correlated.

Using PCA:

 PCA creates new features:

o PC1 = combination of height and weight
o PC2 = age
 One component may be dropped if it explains very little variance

Result:

 Data reduced from 3 features → 2 features

 Minimal information loss

Explained Variance Ratio

PCA provides explained variance ratio:

 Indicates how much variance each component retains

Example:

Component Variance (%)

PC1 70%
PC2 20%
PC3 10%

If PC1 + PC2 = 90%, PC3 can be discarded.

Benefits of PCA for Dimensionality Reduction

  Reduces data size
 Removes correlated features
 Improves training speed
 Reduces overfitting
 Helps data visualization (2D/3D)

Limitations of PCA

 Components are not easily interpretable

 Assumes linear relationships
 Sensitive to outliers
 Does not consider class labels

Applications of PCA

 Image compression
 Noise reduction
 Feature extraction
 Visualization of high-dimensional data
 Preprocessing step for ML algorithms

Summary

Aspect Description
Type Unsupervised learning
Goal Reduce dimensions with minimal information loss
Method Variance maximization
Output Principal components
Used Before Classification, clustering, regression

Q.23 Explain the concept of Back Propagation in ANN with example.

Backpropagation (Backward Propagation of Errors) is a supervised learning algorithm

used to train multi-layer neural networks.
It works by calculating the error at the output layer and propagating it backward through
the network to update the weights.

The main objective of backpropagation is to minimize the error between the actual output and
the predicted output.

2. Why Backpropagation is Required?

 Multi-layer neural networks contain hidden layers.

  Error cannot be directly assigned to hidden layer neurons.
 Backpropagation uses calculus (chain rule) to distribute the error backward.
 It allows the network to learn complex nonlinear relationships.

3. Basic Components Involved

 Input Layer – receives input values

 Hidden Layer(s) – processes inputs
 Output Layer – produces prediction
 Weights – strength of connections
 Bias – threshold adjustment
 Activation Function – introduces non-linearity
 Loss Function – measures error

4. Working of Backpropagation Algorithm

Backpropagation works in two main phases:

Phase 1: Forward Propagation

1. Input values are fed to the network.

2. Weighted sum is calculated:

z=∑wx+bz = \sum wx + bz=∑wx+b

3. Activation function is applied.

4. Output is produced at the output layer.

Phase 2: Backward Propagation

1. Error is calculated using a loss function:

Error=(Target−Output)2Error = (Target - Output)^2Error=(Target−Output)2

2. Error is propagated backward from output to hidden layers.

3. Gradients are calculated using the chain rule.
4. Weights are updated using Gradient Descent:

wnew=wold−η∂Error∂ww_{new} = w_{old} - \eta \frac{\partial Error}{\partial w}wnew

=wold−η∂w∂Error

where η\etaη is the learning rate.

5. Backpropagation Algorithm Steps

1. Initialize weights and bias randomly.

 2. Perform forward pass.
3. Compute error.
4. Compute gradients of error.
5. Update weights and bias.
6. Repeat for multiple epochs until error is minimized.

6. Simple Numerical Example

Problem:

Train a neural network to learn the AND logic gate.

x₁ x₂ Target
1 1 1

Step 1: Initial Values

 Weights:
w1=0.5,w2=0.5w_1 = 0.5, w_2 = 0.5w1=0.5,w2=0.5
 Bias:
b=−0.7b = -0.7b=−0.7
 Learning Rate:
η=0.1\eta = 0.1η=0.1
 Activation Function: Sigmoid

Step 2: Forward Pass

Weighted sum:

z=(1×0.5)+(1×0.5)−0.7=0.3z = (1 \times 0.5) + (1 \times 0.5) - 0.7 =

0.3z=(1×0.5)+(1×0.5)−0.7=0.3

Apply sigmoid:

Output=11+e−0.3≈0.574Output = \frac{1}{1 + e^{-0.3}} \approx 0.574Output=1+e−0.31≈0.574

Step 3: Error Calculation

Error=(Target−Output)2=(1−0.574)2=0.181Error = (Target - Output)^2 = (1 - 0.574)^2 =

0.181Error=(Target−Output)2=(1−0.574)2=0.181

Step 4: Backward Pass (Weight Update Concept)

 Calculate gradient of error

 Update weights using gradient descent
 Weights move in direction that reduces error
(Exact derivative calculations are usually omitted in exams, concept is sufficient.)

Step 5: Repeat Training

The process is repeated for all input combinations until the output becomes close to the target
value.

7. Diagram of Backpropagation

Input Layer → Hidden Layer → Output Layer

↓ ↓ ↓
Forward Propagation
↑ ↑ ↑
Backward Error Propagation

8. Advantages of Backpropagation

 Enables training of deep neural networks

 Can learn complex non-linear patterns
 Widely used and well-understood
 Efficient gradient computation

9. Limitations of Backpropagation

 Slow convergence for large networks

 May get stuck in local minima
 Requires differentiable activation functions
 Sensitive to learning rate selection

10. Applications of Backpropagation

 Image recognition
 Speech recognition
 Handwriting recognition
 Medical diagnosis
 Stock market prediction

11. Summary

Aspect Description
Type Supervised learning
Purpose Minimize prediction error
Key Idea Backward error correction
Method Gradient descent + chain rule
 Aspect Description
Used In Multi-layer neural networks

Q.24 What is K mean clustering? Explain with example.

K-Means clustering is an unsupervised machine learning algorithm used to group data into
K distinct clusters based on similarity.
Each cluster is represented by the mean (centroid) of the data points belonging to that cluster.

 ―K‖ → number of clusters (chosen in advance)

 ―Means‖ → average of data points in a cluster

The goal of K-Means is to minimize intra-cluster distance and maximize inter-cluster

distance.

2. Key Idea

Data points that are close to each other are grouped into the same cluster.
Closeness is usually measured using Euclidean distance.

3. Algorithm of K-Means Clustering

Step-by-Step Procedure

1. Choose the number of clusters K.

2. Randomly initialize K centroids.
3. Assign each data point to the nearest centroid.
4. Recalculate the centroid of each cluster (mean of all points in that cluster).
5. Repeat steps 3 and 4 until:
o Centroids do not change, or
o Maximum iterations are reached.

4. Mathematical Representation

Distance Calculation (Euclidean Distance):

d=(x1−x2)2+(y1−y2)2d = \sqrt{(x_1 - x_2)^2 + (y_1 - y_2)^2}d=(x1−x2)2+(y1−y2)2

New Centroid Calculation:

Centroid=1n∑i=1nxiCentroid = \frac{1}{n} \sum_{i=1}^{n} x_iCentroid=n1i=1∑nxi

5. Example of K-Means Clustering

Example: Student Marks Clustering

Suppose we have students’ marks in Mathematics and Science:

Student Math Science

A 85 80
B 88 82
C 30 35
D 32 30
E 60 62
F 58 65

Let K = 3 (High, Medium, Low performers).

Step 1: Initialize Centroids (Randomly)

 C1 → (85, 80)
 C2 → (30, 35)
 C3 → (60, 62)

Step 2: Assign Points to Nearest Centroid

Student Assigned Cluster

A C1
B C1
C C2
D C2
E C3
F C3

Step 3: Recalculate Centroids

 New C1 = Mean of A & B → (86.5, 81)

 New C2 = Mean of C & D → (31, 32.5)
 New C3 = Mean of E & F → (59, 63.5)

Step 4: Repeat

Since cluster assignments do not change, the algorithm stops.

Final Clusters

 Cluster 1: High performers

  Cluster 2: Low performers
 Cluster 3: Medium performers

6. Diagram Representation (Conceptual)

● ● Cluster 1
× (Centroid)

● ● Cluster 3
×

● ● Cluster 2
×

(● = data points, × = centroid)

7. Advantages of K-Means

 Simple and easy to implement

 Computationally efficient
 Works well for large datasets
 Fast convergence

8. Limitations of K-Means

 Value of K must be chosen beforehand

 Sensitive to initial centroid selection
 Not suitable for non-spherical clusters
 Affected by outliers

9. Applications of K-Means Clustering

 Customer segmentation
 Image compression
 Document clustering
 Market research
 Pattern recognition

10. Summary Table

Feature Description
Type Unsupervised learning
Output Clustered data
Distance Metric Euclidean (commonly)
Goal Minimize within-cluster variance
Q.25 What are building blocks of neural network, elaborate?

A Neural Network is inspired by the human brain and is composed of several fundamental
components called building blocks. These components work together to process input data,
learn patterns, and generate outputs.

1. Neurons (Processing Units)

 A neuron (also called a node or unit) is the basic computational element of a neural
network.
 It receives input signals, processes them, and produces an output.

Function of a Neuron:

z=∑i=1nwixi+bz = \sum_{i=1}^{n} w_i x_i + bz=i=1∑nwixi+b

Where:

 xix_ixi → input
 wiw_iwi → weight
 bbb → bias
 zzz → weighted sum

The neuron then applies an activation function to produce the final output.

2. Layers of Neural Network

Neural networks are organized into layers:

a) Input Layer

 Accepts raw input features.

 No computation is performed.
 Each neuron represents one feature.

b) Hidden Layer(s)

 Perform intermediate processing.

 Extract patterns and relationships.
 A network may have one or multiple hidden layers.

c) Output Layer

 Produces the final prediction.

  Number of neurons depends on the problem type:
o 1 neuron → binary classification
o Multiple neurons → multi-class classification or regression

3. Weights

 Weights determine the strength of connections between neurons.

 Learning occurs by updating weights during training.
 Larger weights → stronger influence of the input.

4. Bias

 Bias allows the model to shift the activation function.

 Helps the network fit data better.
 Without bias, outputs may always pass through the origin.

5. Activation Function

Activation functions introduce non-linearity into the network.

Common Activation Functions:

Function Formula Use

Step 0 or 1 Perceptron
Sigmoid 11+e−x\frac{1}{1+e^{-x}}1+e−x1 Binary classification
ReLU max(0, x) Deep learning
Tanh tanh⁡(x)\tanh(x)tanh(x) Hidden layers
Softmax Probabilities Multi-class output

6. Loss (Cost) Function

 Measures how far the predicted output is from the actual output.
 Guides learning by providing an error signal.

Examples:

 Mean Squared Error (MSE)

 Cross-Entropy Loss

7. Learning Algorithm

 Adjusts weights and bias to minimize loss.

 Backpropagation is the most common learning algorithm.
 Uses gradient descent to update parameters.
8. Learning Rate

 Controls the step size during weight updates.

 Small learning rate → slow learning
 Large learning rate → may overshoot optimal solution

9. Optimizer

 Improves speed and stability of learning.

 Examples:
o Gradient Descent
o Adam
o RMSProp
o Momentum

10. Epochs and Batches

 Epoch: One complete pass of the dataset through the network.

 Batch: Subset of data processed at once.

11. Network Architecture

 Arrangement of layers and neurons.

 Determines model complexity.
 Examples:
o Single-layer network
o Multi-layer perceptron
o Deep neural network

12. Regularization Techniques

 Prevent overfitting.
 Examples:
o L1/L2 regularization
o Dropout
o Early stopping

Summary Table

Component Description
Neuron Basic processing unit
Weights Strength of connections
Bias Threshold adjustment
Activation Introduces non-linearity
 Component Description
Layers Structure of network
Loss Measures prediction error
Optimizer Updates weights
Learning Rate Step size of updates

Q.26 Write a short note on Recurrent neural n/w & convolutional neural n/w.

A Recurrent Neural Network (RNN) is a type of artificial neural network designed to process
sequential and time-dependent data. Unlike feedforward neural networks, RNNs have
feedback connections, which allow information to persist over time. This makes RNNs suitable
for tasks where context and order of data matter.

2. Key Concept

The defining feature of RNNs is memory.

An RNN maintains a hidden state that captures information from previous inputs and uses it to
influence the current output.

Mathematically:

ht=f(Whht−1+Wxxt+b)h_t = f(W_h h_{t-1} + W_x x_t + b)ht=f(Whht−1+Wxxt+b)

yt=g(Wyht)y_t = g(W_y h_t)yt=g(Wyht)

Where:

 xtx_txt = input at time t

 hth_tht = hidden state
 yty_tyt = output
 f,gf, gf,g = activation functions

3. Architecture of RNN

 Input layer
 Hidden layer with recurrent connections
 Output layer

The same weights are shared across all time steps, making the model efficient for sequence
learning.

4. Working of RNN

1. Input is fed one element at a time.

 2. Hidden state stores information from previous inputs.
3. Output depends on current input and past hidden state.
4. Training is done using Backpropagation Through Time (BPTT).

5. Types of RNN

 Simple RNN
 LSTM (Long Short-Term Memory) – handles long-term dependencies
 GRU (Gated Recurrent Unit) – simplified version of LSTM

6. Advantages of RNN

 Can handle variable-length input sequences

 Maintains temporal information
 Suitable for sequential data

7. Limitations of RNN

 Vanishing and exploding gradient problem

 Difficult to learn long-term dependencies
 Computationally expensive for long sequences

8. Applications of RNN

 Speech recognition
 Language translation
 Text generation
 Time series forecasting
 Stock price prediction

Convolutional Neural Network (CNN)

1. Introduction

A Convolutional Neural Network (CNN) is a specialized neural network mainly used for
image processing, computer vision, and pattern recognition. CNNs automatically extract
spatial features from input data using convolution operations.

2. Key Concept

CNNs work on the principle of local connectivity and weight sharing, which reduces the
number of parameters and improves efficiency.

3. Architecture of CNN
a) Convolution Layer

 Applies filters (kernels) to input

 Extracts features like edges, corners, textures

b) Activation Layer

 Introduces non-linearity
 Commonly uses ReLU (Rectified Linear Unit)

c) Pooling Layer

 Reduces spatial dimensions

 Types: Max pooling, Average pooling

d) Fully Connected Layer

 Performs classification based on extracted features

4. Working of CNN

1. Input image is passed through convolution layers.

2. Feature maps are generated.
3. Pooling reduces dimensionality.
4. Fully connected layers produce final output.

5. Advantages of CNN

 Automatic feature extraction

 Fewer parameters than fully connected networks
 High accuracy for image-based tasks
 Translation invariance

6. Limitations of CNN

 Requires large training data

 Computationally intensive
 Less effective for sequential data

7. Applications of CNN

 Image classification
 Face recognition
 Object detection
 Medical image analysis
 Autonomous vehicles
Comparison between RNN and CNN

Feature RNN CNN

Data type Sequential data Spatial data

Memory Yes (hidden state) No

Main use Time-series, NLP Images, videos

Key operation Recurrence Convolution

Examples Speech, text Vision tasks

Q.27 What do you mean by resolution and unification explain with example.

[Link] in Artificial Intelligence

�Meaning of Resolution

Resolution is a rule of inference used in predicate logic and propositional logic to prove
whether a given statement is true or false by refutation (proof by contradiction).

In simple words:
Resolution tries to derive a contradiction (FALSE) from the given knowledge base.
If contradiction is found → the statement is proved TRUE.

�Why Resolution is Used in AI?

 Used in automated theorem proving

 Used in expert systems
 Helps machines reason logically
 Works well with first-order logic

�Resolution Principle

If we have two clauses:

 Clause 1: (A ∨ B)
 Clause 2: (¬B ∨ C)

Then by resolving on B, we get:

  (A ∨ C)

This new clause is called the resolvent.

�Example of Resolution (Propositional Logic)

Given clauses:

1. (P ∨ Q)
2. (¬Q)

Apply resolution:

 Resolve Q and ¬Q
 Result: P

✔ So, P is proved true

�Resolution Example (AI Context)

Knowledge Base:

1. All humans are mortal

→ ¬Human(x) ∨ Mortal(x)
2. Socrates is a human
→ Human(Socrates)

Goal: Prove Socrates is mortal

Negate goal:
¬Mortal(Socrates)

Now resolve step-by-step:

 From (¬Human(x) ∨ Mortal(x)) and Human(Socrates)

→ Mortal(Socrates)
 Mortal(Socrates) and ¬Mortal(Socrates)
→ Contradiction (FALSE)

✔ Hence, ocrates is mortal is proved.

2. Unification in Artificial Intelligence

�Meaning of Unification
Unification is the process of making two logical expressions identical by substituting
variables with constants or other variables.

In short:
Unification answers the question:
“Can these two expressions match?”

�Why Unification is Important?

 Essential for resolution

 Used in predicate logic
 Used in logic programming (PROLOG)
 Helps AI systems match rules with facts

�Example of Unification

Expressions:

 Loves(x, IceCream)
 Loves(Archana, y)

After unification:

 x = Archana
 y = IceCream

✔ Now both expressions become:

Loves(Archana, IceCream)

�Another Example

Expressions:

 Parent(x, y)
 Parent(John, Mary)

Unification gives:

 x = John
 y = Mary

✔ Expressions match successfully.

�Failure of Unification

Expressions:
  Likes(x, Apple)
 Likes(Banana, y)

Cannot unify because:

 Apple ≠ Banana

3. Relationship Between Resolution and Unification

Resolution Unification
Logical inference rule Matching process
Proves statements Helps match predicates
Used to derive new clauses Used before applying resolution
Works on clauses Works on variables

Unification is used inside resolution to match predicates correctly.

4. Combined Example (Resolution + Unification)

Knowledge Base:

1. ¬Student(x) ∨ Intelligent(x)
2. Student(Archana)

Goal: Intelligent(Archana)

Steps:

 Unify x = Archana
 Apply resolution:
o From clause 1 and clause 2 → Intelligent(Archana)

Q.28 Explain Kalman Filter in detail.

[Link]

The Kalman Filter is an optimal recursive algorithm used to estimate the state of a dynamic
system from a series of noisy and incomplete measurements.

In simple words:
It continuously predicts and corrects values to get the best possible estimate, even when data
is noisy.
It is widely used in:

 Artificial Intelligence
 Robotics
 Autonomous vehicles
 GPS navigation
 Signal processing
 Control systems

[Link] Kalman Filter is Needed

Real-world measurements are:

 Noisy
 Incomplete
 Uncertain

The Kalman Filter:

 Combines model prediction + sensor measurements

 Minimizes estimation error
 Works in real time
 Is computationally efficient

[Link] Idea (Intuition)

Kalman Filter works in two repeating steps:

�Step 1: Prediction

―Where do I think the system is now?‖

Uses the previous state and system model.

�Step 2: Update (Correction)

―What do my measurements say?‖

Corrects the prediction using new sensor data.

This cycle repeats continuously.

4. Mathematical Model of Kalman Filter

Kalman Filter assumes the system is linear and Gaussian.

(A) State Equation (Process Model)

xk=Axk−1+Buk+wkx_k = A x_{k-1} + B u_k + w_kxk=Axk−1+Buk+wk

Where:

 xkx_kxk = state vector at time k

 AAA = state transition matrix
 BBB = control input matrix
 uku_kuk = control vector
 wkw_kwk = process noise (Gaussian)

(B) Measurement Equation

zk=Hxk+vkz_k = H x_k + v_kzk=Hxk+vk

Where:

 zkz_kzk = measurement vector

 HHH = observation matrix
 vkv_kvk = measurement noise (Gaussian)

[Link] Filter Algorithm Steps

Step 1: Prediction

Predicted State Estimate

x^k−=Ax^k−1+Buk\hat{x}_k^{-} = A \hat{x}_{k-1} + B u_kx^k−=Ax^k−1+Buk

Predicted Error Covariance

Pk−=APk−1AT+QP_k^{-} = A P_{k-1} A^T + QPk−=APk−1AT+Q

Where:

 PPP = error covariance matrix

 QQQ = process noise covariance

Step 2: Update (Correction)

Kalman Gain
Kk=Pk−HT(HPk−HT+R)−1K_k = P_k^{-} H^T (H P_k^{-} H^T + R)^{-1}Kk=Pk−HT(HPk−
HT+R)−1

Kalman Gain decides:

  How much to trust prediction
 How much to trust measurement

Updated State Estimate

x^k=x^k−+Kk(zk−Hx^k−)\hat{x}_k = \hat{x}_k^{-} + K_k (z_k - H \hat{x}_k^{-})x^k=x^k−
+Kk(zk−Hx^k−)

Updated Error Covariance

Pk=(I−KkH)Pk−P_k = (I - K_k H) P_k^{-}Pk=(I−KkH)Pk−

6. Simple Example (Real-World)

�Example: Estimating the Position of a Moving Car

 Prediction from motion model:

Car is at 100 m
 GPS measurement (noisy):
Shows 105 m

Kalman Filter:

 Trusts prediction if GPS is noisy

 Trusts GPS if model is uncertain
 Combines both → best estimate ≈ 102 m

✔ This happens continuously at each time step.

�Assumptions of Kalman Filter

 System is linear
 Noise is Gaussian
 Noise has zero mean
 Model parameters are known

�Types of Kalman Filters

Type Used For

Standard Kalman Filter Linear systems

Extended Kalman Filter (EKF) Non-linear systems (linearized)

 Type Used For

Unscented Kalman Filter (UKF) Highly non-linear systems

Adaptive Kalman Filter Time-varying noise

[Link]

✔ Optimal estimator
✔ Real-time operation
✔ Low computational cost
✔ Handles noisy data efficiently

�Limitations

Assumes linearity
Assumes Gaussian noise
Sensitive to incorrect model parameters

1. Applications in AI

 Robot localization & mapping

 Sensor fusion
 Object tracking
 Speech recognition
 Financial time-series prediction

Q.29 Explain the following. i) Linear regression ii) Logistic Regression

i) Linear Regression

[Link]

Linear Regression is a supervised machine learning algorithm used to predict a continuous

numerical value by establishing a linear relationship between:

 Independent variable(s) (X)

 Dependent variable (Y)

It assumes that the change in output is proportional to the change in input.

[Link] Model
Simple Linear Regression

Y=β0+β1X+ϵY = \beta_0 + \beta_1 X + \epsilonY=β0+β1X+ϵ

Where:

 YYY = dependent variable (output)

 XXX = independent variable (input)
 β0\beta_0β0 = intercept
 β1\beta_1β1 = slope (coefficient)
 ϵ\epsilonϵ = error term

Multiple Linear Regression

Y=β0+β1X1+β2X2+⋯+βnXn+ϵY = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \dots + \beta_n X_n

+ \epsilonY=β0+β1X1+β2X2+⋯+βnXn+ϵ

[Link] of Linear Regression

Step 1: Collect training data

Step 2: Fit the best-fit line

Step 3: Minimize error using Least Squares Method

Cost Function (Mean Squared Error)

J(β)=1n∑i=1n(Yi−Y^i)2J(\beta) = \frac{1}{n} \sum_{i=1}^{n} (Y_i - \hat{Y}_i)^2J(β)=n1
i=1∑n(Yi−Y^i)2

The objective is to minimize the error between predicted and actual values.

[Link]

Predicting house price based on area ([Link]):

 Area = 1000 [Link]

 Price = 50 lakhs

The model learns the relationship and predicts prices for new areas.

[Link]

 Linear relationship between X and Y

 Errors are independent
 Homoscedasticity (constant variance)
  No multicollinearity
 Errors are normally distributed

[Link]

✔ Simple and easy to interpret

✔ Fast computation
✔ Works well with linearly related data

[Link]

Cannot model non-linear relationships

Sensitive to outliers
Assumptions must be satisfied

[Link]

 Sales forecasting
 Price prediction
 Risk assessment
 Trend analysis

ii) Logistic Regression

[Link]

Logistic Regression is a supervised classification algorithm used to predict categorical

outcomes, mainly binary classes (0 or 1).

Despite its name, it is used for classification, not regression.

[Link] Function (Sigmoid Function)

σ(z)=11+e−z\sigma(z) = \frac{1}{1 + e^{-z}}σ(z)=1+e−z1

Where:

z=β0+β1Xz = \beta_0 + \beta_1 Xz=β0+β1X

The sigmoid function converts values into probabilities between 0 and 1.

[Link] Model
P(Y=1∣X)=11+e−(β0+β1X)P(Y=1|X) = \frac{1}{1 + e^{-(\beta_0 + \beta_1
X)}}P(Y=1∣X)=1+e−(β0+β1X)1

 Output close to 1 → Class 1

 Output close to 0 → Class 0

[Link] Boundary

 If probability ≥ 0.5 → Class 1

 If probability < 0.5 → Class 0

This threshold can be adjusted based on the problem.

[Link] Function (Log Loss)

J(β)=−1n∑[ylog⁡(y^)+(1−y)log⁡(1−y^)]J(\beta) = -\frac{1}{n} \sum [y \log(\hat{y}) + (1-y)

\log(1-\hat{y})]J(β)=−n1∑[ylog(y^)+(1−y)log(1−y^)]

Used instead of MSE because MSE performs poorly for classification.

[Link]

Predicting whether an email is spam or not spam:

 Output: 1 → Spam
 Output: 0 → Not spam

[Link] of Logistic Regression

Type Description

Binary Two classes

Multinomial More than two classes

Ordinal Ordered classes

[Link]

 Linear relationship between independent variables and log-odds

 Independent observations
 No multicollinearity
 Large sample size preferred
[Link]

✔ Outputs probability values

✔ Easy to implement and interpret
✔ Efficient for binary classification

�Limitations

Not suitable for complex non-linear problems

Sensitive to outliers
Requires feature scaling

[Link] Differences (Exam-Friendly Table)

Feature Linear Regression Logistic Regression

Output Continuous Categorical

Function Linear Sigmoid

Problem Type Regression Classification

Cost Function MSE Log Loss

Output Range (-∞, +∞) (0, 1)

Q.30 Differentiate between Lasso Regression and Ridge Regression.

[Link] (Common Background)

Both Lasso Regression and Ridge Regression are regularization techniques used in linear
regression to:

 Prevent overfitting
 Handle multicollinearity
 Improve model generalization

They work by adding a penalty term to the loss function.

[Link] Function (Key Difference)

Ridge Regression (L2 Regularization)

J(β)=∑(yi−y^i)2+λ∑βj2J(\beta) = \sum (y_i - \hat{y}_i)^2 + \lambda \sum \beta_j^2J(β)=∑(yi
−y^i)2+λ∑βj2

 Penalizes square of coefficients

 Shrinks coefficients close to zero, but never exactly zero

Lasso Regression (L1 Regularization)

J(β)=∑(yi−y^i)2+λ∑∣βj∣J(\beta) = \sum (y_i - \hat{y}_i)^2 + \lambda \sum |\beta_j|J(β)=∑(yi−y^

i)2+λ∑∣βj∣

 Penalizes absolute value of coefficients

 Can shrink coefficients exactly to zero

[Link] on Coefficients

�Ridge Regression

 Reduces magnitude of coefficients

 Keeps all features
 No feature elimination

�Lasso Regression

 Forces some coefficients to become zero

 Performs automatic feature selection
 Produces a sparse model

[Link] of Regularization Parameter (λ)

 Controls strength of penalty

 Larger λ → stronger regularization
 λ = 0 → ordinary linear regression

λ Value Effect
Small λ Weak regularization
Large λ Strong shrinkage

[Link] Interpretation (Intuition)

 Ridge Regression: Circular constraint region

 Lasso Regression: Diamond-shaped constraint region

Because of sharp corners in Lasso, solutions often lie on axes → coefficients become zero.
[Link]

Suppose we predict house price using:

 Area
 Bedrooms
 Distance from city
 Age of house

Ridge Regression:

 Uses all variables

 Reduces impact of less important features

Lasso Regression:

 May completely remove age of house

 Keeps only most relevant features

[Link] to Use Which?

✔�Use Ridge Regression when:

 Many features are important

 Multicollinearity exists
 Want stable coefficients

✔�Use Lasso Regression when:

 Feature selection is needed

 Dataset has many irrelevant features
 Want simpler, interpretable model

[Link] and Limitations

Ridge Regression

Advantages

 Handles multicollinearity well

 Reduces model variance

Limitations

 No feature selection
 Less interpretable
Lasso Regression

Advantages

 Performs feature selection

 Produces simpler models

Limitations

 Unstable when features are highly correlated

 May select one feature and ignore others arbitrarily

[Link] Net (Related Concept)

Elastic Net combines both penalties:

J(β)=∑(yi−y^i)2+λ1∑∣βj∣+λ2∑βj2J(\beta) = \sum (y_i - \hat{y}_i)^2 + \lambda_1 \sum |\beta_j|

+ \lambda_2 \sum \beta_j^2J(β)=∑(yi−y^i)2+λ1∑∣βj∣+λ2∑βj2

Used when we want benefits of both Lasso and Ridge.

�Exam-Friendly Comparison Table

Feature Lasso Regression Ridge Regression

Regularization Type L1 L2
Penalty Term (\sum \beta
Feature Selection Yes No
Coefficients Can become zero Never zero
Model Sparsity Sparse model Dense model
Multicollinearity May be unstable Handles well
Interpretability High Moderate
Computational Cost Higher Lower

Q.31Explain the following Evaluation Metrics: i) MAE ii) RMSE iii) R2

Evaluation Metrics

Evaluation metrics are used to measure the performance of regression models by comparing
actual values with predicted values.

i) Mean Absolute Error (MAE)

[Link]

Mean Absolute Error (MAE) measures the average of the absolute differences between
actual values and predicted values.

In simple terms:
It tells us how far predictions are from actual values on average, without considering
direction (+ or −).

[Link]

MAE=1n∑i=1n∣yi−y^i∣MAE = \frac{1}{n} \sum_{i=1}^{n} |y_i - \hat{y}_i|MAE=n1i=1∑n∣yi

−y^i∣

Where:

 yiy_iyi = actual value

 y^i\hat{y}_iy^i = predicted value
 nnn = number of observations

[Link]

| Actual (y) | Predicted (ŷ) | |y − ŷ| |

|----|----|----|
| 100 | 90 | 10 |
| 200 | 210 | 10 |
| 300 | 290 | 10 |

MAE=10+10+103=10MAE = \frac{10 + 10 + 10}{3} = 10MAE=310+10+10=10

✔ On average, predictions are 10 units away from actual values.

4. Interpretation

 Lower MAE → Better model

 MAE = 0 → Perfect prediction
 Unit of MAE is same as target variable

[Link]

✔ Easy to understand
✔ Less sensitive to outliers
✔ Direct interpretation

6. Limitations
 Does not penalize large errors strongly
Absolute value is not differentiable at zero

ii) Root Mean Square Error (RMSE)

1. Definition

Root Mean Square Error (RMSE) measures the square root of the average of squared
differences between actual and predicted values.

RMSE penalizes large errors more heavily than MAE.

2�Formula

RMSE=1n∑i=1n(yi−y^i)2RMSE = \sqrt{\frac{1}{n} \sum_{i=1}^{n} (y_i -

\hat{y}_i)^2}RMSE=n1i=1∑n(yi−y^i)2

3�Example

Actual (y) Predicted (ŷ) (y − ŷ)²

100 90 100

200 210 100

300 260 1600

RMSE=100+100+16003=600≈24.49RMSE = \sqrt{\frac{100 + 100 + 1600}{3}} = \sqrt{600}

\approx 24.49RMSE=3100+100+1600=600≈24.49

[Link]

 Lower RMSE → Better model

 Penalizes large errors
 Same unit as output variable

[Link]

✔ Differentiable (useful for optimization)

✔ Penalizes large errors
✔ Widely used in practice

[Link]
 Highly sensitive to outliers
Harder to interpret than MAE

iii) R² Score (Coefficient of Determination)

[Link]

R² (R-squared) measures how well the model explains the variability in the dependent
variable.

It shows the goodness of fit of a regression model.

2. Formula

R2=1−∑(yi−y^i)2∑(yi−yˉ)2R^2 = 1 - \frac{\sum (y_i - \hat{y}_i)^2}{\sum (y_i -

\bar{y})^2}R2=1−∑(yi−yˉ)2∑(yi−y^i)2

Where:

 yˉ\bar{y}yˉ = mean of actual values

[Link] of R² Values

R² Value Meaning

1 Perfect fit

0 Model predicts mean only

<0 Worse than mean prediction

Example:

 R² = 0.85 → Model explains 85% variance in data

[Link]

If:

 Total variance = 1000

 Unexplained variance = 200

R2=1−2001000=0.8R^2 = 1 - \frac{200}{1000} = 0.8R2=1−1000200=0.8

✔ 80% of the variation is explained by the model.

[Link]

✔ Easy comparison of models

✔ Unitless metric
✔ Indicates goodness of fit

[Link]

Can increase even if irrelevant features are added

Not reliable for non-linear models
Does not show prediction error magnitude

�Comparison Table (Exam-Friendly)

Metric MAE RMSE R²

Measures Avg absolute error Root mean squared error Variance explained

Penalizes large errors No Yes No

Unit Same as output Same as output Unitless

Sensitivity to outliers Low High Medium

Best Used When Equal error weight Large errors matter Model comparison