DEEP LEARNING
( UNIT 1 ) INTRODUCTION :
Introduction to machine learning-
1. Artificial Intelligence (AI)
Artificial Intelligence is a branch of computer science that deals with the design of intelligent
agents or systems capable of perceiving their environment, reasoning, learning, and taking
👉
actions to achieve specific goals, similar to human intelligence.
In simple words: AI is the umbrella field that aims to make machines think and act like humans.
Real-World Examples : Chatbots – AI understands user queries and gives replies.
2. Machine Learning (ML)
Machine Learning is is defined as a discipline of and subset of artificial intelligence (AI) that
provides machines the ability to automatically learn and improve from data and past experiences
with minimal human intervention.
It focuses on algorithms that can identify patterns, make predictions, and improve performance
👉
over time.
In simple words: ML is how computers learn from data to make decisions or predictions.
Real-World Examples : Netflix / YouTube / Amazon Recommendations – Suggests movies/products based
on your past behavior.
3. Deep Learning (DL)
Deep Learning is a subset of Machine Learning that uses artificial neural networks with multiple
layers to learn hierarchical representations of data.
It is highly effective for image processing, speech recognition, and natural language
👉
understanding, requiring large datasets and high computational power.
In simple words: DL is ML with neural networks that handle complex data like images, voice,
and text.
Real-World Examples : Self-Driving Cars (Tesla, Waymo) – DL processes camera and sensor data to detect
lanes, pedestrians, and vehicles.
⚡ Hierarchy (as in curriculum):
AI (broad field) → ML (subset of AI) → DL (subset of ML).
Aspect Artificial Intelligence (AI) Machine Learning (ML) Deep Learning (DL)
A subset of AI that enables A subset of ML that uses neural
Broad field of creating smart
machines to learn from data networks with many layers to
Definition systems that can simulate
and improve over time learn complex patterns from
human intelligence.
without explicit programming large datasets.
To create intelligent systems To enable machines to learn To achieve highly accurate
Goal that can think, reason, and act from data and make predictions and automate feature
like humans. predictions/decisions. extraction.
Very broad – includes
Narrower – focused mainly Even narrower – focused on
reasoning, problem-solving,
Scope on learning and prediction neural networks and high-level
decision-making, ML, DL,
from data. feature extraction.
robotics, etc.
May need heavy human Requires some human Minimal intervention; system
Human
intervention (rules, knowledge- guidance (feature extraction, automatically learns features
Intervention
based systems). algorithm selection). from raw data.
Data Works even with small data Needs structured data for Needs huge datasets to perform
Requirement (rule-based). good performance. well.
Hardware Can work on normal Can run on CPUs but benefits Requires powerful GPUs/TPUs
Needs computers. from GPUs. and high computational power.
Depends on complexity of Moderate training time Very high training time due to
Time to Train
rules, not always heavy. depending on dataset size. large neural networks.
May not always be very Good accuracy for simpler Very high accuracy for complex
Accuracy
accurate (depends on rules). problems. tasks (images, speech, NLP).
CNN (Convolutional Neural
Expert systems, search Regression, decision trees,
Techniques Networks), RNN (Recurrent
algorithms, logic-based clustering, random forests,
Used Neural Networks), LSTM,
reasoning. etc.
Transformers.
Self-driving cars, facial
Email spam filter, fraud
Chess-playing AI, chatbots, recognition, voice assistants
Examples detection, movie
smart assistants, robotics. (speech-to-text), medical image
recommendation.
analysis.
TYPES OF ML :
1) SUPERVISED LEARNING
Supervised learning. Is the types of machine learning In which machines Are trained using Well
“labelled ” Training data. And on basis of that data. Machines predict the output on new/unseen
data.
The labelled data means some input data is already Tagged with the correct output.
Key Idea: Learn from examples (teacher supervises).
Examples (for exam):
• Predicting house prices (input = features like size, location; output = price).
• Email spam classification (spam / not spam).
• Handwritten digit recognition (MNIST dataset)
Algorithms: Linear Regression, Logistic Regression, Decision Trees, Support Vector Machines
(SVM), Neural Networks.
Types of supervised learning.
(i) Classification : A classification problem is when the output variable Is a category Such as red or
blue, Disease and no disease .Yes-No, Male-female, True false Etc.
(ii) Regression.: Our regression problem is when The output variable is a real value such as
forecasting sales, weather forecasting .
2) UNSUPERVISED LEARNING
Unsupervised learning is a type of machine learning In which Models are Trained using an
unlabeled data set. And allowed to act on that data without any supervision. The goal of
unsupervised learning is to find the underlying structure of dataset , Group that data According to
similarities and represent that dataset in a compressed format.
Types of unsupervised learning
i) Clustering
Clustering is an unsupervised learning problem where the goal is to group data into clusters
(groups) such that data points in the same group are more similar to each other than to those in
👉
other groups.
Example: Grouping customers based on their buying behavior (like frequent buyers, discount
seekers, luxury buyers).
In simple words: Clustering means finding natural groups in data without having predefined labels.
(ii) Association
Association rule learning is an unsupervised learning technique used to find relationships or
dependencies between different items in a dataset.
It discovers interesting rules or patterns like “if a customer buys item A, they are also likely to buy
👉
item B”. This helps businesses make strategies and reduce cost.
Example: In supermarkets, if customers buy bread, they also often buy butter.
In simple words: Association means finding which items or events occur together frequently in
data.
⚡ Exam-Ready Summary (AKTU style):
• Clustering: Finding natural groups in data (e.g., customer segmentation).
• Association: Finding rules or relations between data items (e.g., bread → butter).
3) SEMI SUPERVISED LEARNING
• In semi-supervised learning, we use a small amount of labeled data and a large amount of
unlabeled data.
• The process is:
1. First, similar data points are grouped using an unsupervised learning algorithm (like
clustering).
👉
2. Then, the small labeled dataset is used to assign labels to the unlabeled data.
In short: It combines supervised + unsupervised methods to make better use of data.
Assumptions in Semi-Supervised Learning
1. Continuity Assumption
• Points that are close to each other in the data space are likely to have the same label.
• Example: Two students with very similar marks are likely to belong to the same grade
group.
2. Cluster Assumption
• Data can be grouped into clusters, and all points in the same cluster probably share
the same output label.
• Example: Customers grouped in a cluster based on shopping habits are likely to buy
similar products.
3. Manifold Assumption
• Data exists on a lower-dimensional space within the higher-dimensional input.
• Example: Although an image has thousands of pixels, the essential information (like
shape of an object) lies in a much simpler, lower-dimensional representation.
⚡ Exam-Ready Simple Definition:
Semi-supervised learning uses few labeled + many unlabeled data, assumes nearby points,
clusters, and lower-dimensional structures can help label unknown data more accurately.
Examples (for exam writing):
• Fraud Detection: Few transactions labeled as “fraud” or “not fraud” + huge unlabeled data.
• Web Content Classification: Few webpages are labeled (sports, news, entertainment) +
many unlabeled webpages.
• Speech Recognition: Few labeled voice samples + huge amount of unlabeled speech data.
What is Reinforcement Learning?
Reinforcement Learning (RL) is a feedback-based machine learning technique where an agent
learns to behave in an environment by performing actions and observing the results.
• For each good action, the agent receives a positive reward.
👉
• For each bad action, the agent receives a negative reward (penalty).
In short: RL is learning by trial and error with feedback.
Main Elements of an RL System
1. Agent (Learner): The system that makes decisions.
2. Environment: The external system the agent interacts with.
3. Policy: The strategy the agent uses to decide actions.
4. Reward Signal: The feedback (positive or negative) received after taking an action.
How it Works (Flow):
• Agent takes action → interacts with environment.
• Environment gives reward/penalty → agent updates knowledge.
• Process repeats until the agent learns the best behavior (optimal policy).
🌍 Real-World Examples of Reinforcement Learning
1. Self-Driving Cars → The car (agent) learns to drive by getting rewards for safe driving and
penalties for mistakes like hitting obstacles.
2. Robotics → A robot learns to walk or pick objects by trial and error.
3. Game Playing (Chess, Go, Video Games) → The agent learns strategies by winning
(reward) or losing (penalty).
4. Recommendation Systems → YouTube/Netflix recommend content, get positive feedback if
you watch, negative if you skip.
⚡ Exam-ready one-liner definition:
Reinforcement Learning is a machine learning approach where an agent learns optimal behavior through trial-
and-error interactions with an environment, guided by rewards and penalties.
Linear Model
• A linear model means the decision boundary (line or plane) is a straight line (in 2D), or a
flat plane (in higher dimensions).
👉
• The prediction is made using a linear combination of input features.
Example: For two features (x1, x2), a linear model looks like:
y = w1x1 + w2x2 + b
• In SVM, this line (or hyperplane) separates the data into different classes.
•
SVM (Support Vector Machine)
SVM or Support Vector Machine is a linear model for classification and regression problems. It can
solve linear and non-linear problems and work well for many practical problems.
It tries to classify data by finding a hyperplane that maximizes the margin between the classes in the
training data. Hence, SVM is an example of a large margin classifier.
The idea of SVM is simple: The algorithm creates a line or a hyperplane which separates the data
into classes.
Real-World Examples of SVM
1. Email Spam Detection: Classify emails into spam and not spam.
2. Face Recognition: Distinguish between different persons.
3. Medical Diagnosis: Classify tumors as benign or malignant.
4. Handwriting Recognition: Identify digits from 0–9.
According to the SVM algorithm we find the points closest to the line from both the classes. These
points are called support vectors.
We compute the distance between the line and the support vectors. This distance is called the
margin. Our goal is to maximize the margin. The hyperplane for which the margin is maximum is
the optimal hyperplane.
Thus, SVM tries to make a decision boundary in such a way that the separation between the two
classes is as wide as possible.
SVM KERNELS
• SVM can work well in non-linear data cases using kernel trick.
• The function of the kernel trick is to map the low-dimensional input space and transform it
into a higher dimensional space.
• In simple words, kernels convert non-separable problems into separable problems by adding
more dimensions to it.
• It makes SVM more powerful, flexible and accurate.
🔹 Types of SVM Kernels (in context of [Link] Curriculum)
Support Vector Machine (SVM) uses kernel functions to transform data into higher dimensions
where it becomes linearly separable.
1. Linear Kernel
• Formula:
• It is simply the dot product of input vectors.
• Used when data is linearly separable.
• Example: Spam vs. Non-spam classification with simple word counts.
2. Polynomial Kernel
• Formula:
• Here, dd is the polynomial degree.
• Captures non-linear relations by considering interactions between features.
• Example: Handwritten digit recognition.
3. Gaussian Kernel (RBF Kernel)
• Formula:
• Also called Radial Basis Function (RBF).
• Very popular because it handles highly non-linear data well.
• Example: Face recognition.
4. Exponential Kernel
• Formula:
• Similar to Gaussian but uses L1 distance instead of squared L2.
• Good for cases where sharp changes are expected.
• Example: Time series anomaly detection.
5. Laplace Kernel
• Formula:
• It is basically a special case of Exponential kernel.
• Robust to noise, works well with sparse data.
• Example: Text classification.
📌 Summary Table (easy for exam memory)
Kernel Formula Use Case
Linear xTy+cx^T y + c Linearly separable data
Polynomial (xTy+c)d(x^T y + c)^d Non-linear, feature interaction
Gaussian (RBF) exp(−∥x−y∥2/2σ2)\exp(-\|x-y\|^2 / 2\sigma^2) Complex non-linear problems
Exponential exp(−∥x−y∥/σ)\exp(-\|x-y\| / \sigma) Sudden/abrupt changes (time series)
Laplace exp(−∥x−y∥/σ)\exp(-\|x-y\| / \sigma) Sparse/noisy data
Linear Regression ([Link] Curriculum)
1. Introduction
Linear Regression is a statistical method used to model the relationship between a dependent
variable (output) and one or more independent variables (inputs). It works by fitting a linear
equation to the observed data.
The main goal is to find the best-fitting line that minimizes the difference between the observed
values (actual data) and the predicted values given by the model.
2. Mathematical Representation
For a simple linear regression with one independent variable:
y=mx+by = mx + b
Where:
• y → Dependent variable (output)
• x → Independent variable (input)
• m → Slope of the line (shows rate of change)
• b → Intercept (value of y when x=0x = 0)
3. Objective
• To minimize the sum of squared errors (SSE) between observed and predicted values.
• Find the line of best fit.
4. Real-World Examples
1. Business – Predicting sales based on advertising expenditure.
• Independent variable (xx): Amount spent on ads
• Dependent variable (yy): Sales revenue
2. Healthcare – Estimating patient recovery time based on dosage of medicine.
• Independent variable (xx): Dosage of drug
• Dependent variable (yy): Recovery time (in days)
3. Education – Predicting student exam scores based on hours of study.
• Independent variable (xx): Hours studied
• Dependent variable (yy): Exam score
4. Engineering – Forecasting energy consumption based on temperature.
• Independent variable (xx): Temperature
• Dependent variable (yy): Electricity usage
5. Visualization
• Data Points: Plotted on a graph (scatter plot).
• Regression Line: A straight line that best fits the points.
Perfect 👍Let’s restructure the given content into a curriculum-style explanation (AKTU [Link]
module style) with real-world examples:
Logistic Regression ([Link] Curriculum – AKTU)
1. Introduction
Logistic Regression is a supervised machine learning algorithm primarily used for classification
tasks. Unlike Linear Regression, which predicts continuous values, Logistic Regression predicts the
probability that an instance belongs to a particular class (e.g., Yes/No, 0/1, True/False).
2. Key Concepts
• Classification Goal: To determine whether an input belongs to a given class or not.
• Output Nature: Instead of exact values, Logistic Regression produces probabilistic values
between 0 and 1.
• Decision-Making: If probability ≥ 0.5 → class 1 (Yes/True), else class 0 (No/False).
3. Mathematical Representation
The logistic function (sigmoid) is used:
Where:
• P(y=1∣x) → Probability that the instance belongs to class 1
• m, b → Coefficients (similar to Linear Regression)
• e → Euler’s number
4. Difference from Linear Regression
• Linear Regression → Used for regression (continuous values prediction).
• Logistic Regression → Used for classification (categorical values prediction).
5. Real-World Examples
1. Email Filtering
• Input (x): Words in an email
• Output (y): Spam (1) or Not Spam (0)
2. Medical Diagnosis
• Input (x): Patient’s symptoms/test results
• Output (y): Disease Present (1) or Not (0)
3. Banking – Loan Default Prediction
• Input (x): Applicant’s credit score, income
• Output (y): Default (1) or No Default (0)
4. Industry – Defect Detection in Manufacturing
• Input (x): Sensor readings
• Output (y): Defective product (1) or Non-defective (0)
6. Advantages
• Simple and efficient for binary classification.
• Provides probability scores, not just hard decisions.
• Easy to implement and interpret.
Linear Regression vs Logistic Regression
Feature Linear Regression Logistic Regression
Type of Problem Regression (continuous value prediction) Classification (categorical value prediction)
Predicts continuous values (can range
Output Nature Predicts probability values (between 0 and 1)
from −∞-∞ to +∞+∞)
Applications student marks, forecasting sales detection, credit risk prediction
Linearity Fits a straight line to data points Fits a sigmoid curve (S-shaped)
Continuous (e.g., salary, temperature, Binary/Categorical (e.g., spam/not spam,
Target Variable
sales) disease/no disease)
Error
Uses Least Squares Method (SSE) Uses Maximum Likelihood Estimation (MLE)
Minimization
Range of Output Any real number 0 to 1 (interpreted as probability)
Decision Uses threshold (commonly 0.5) to classify
Not applicable
Boundary outcomes
Example Predicting house prices, predicting Email spam detection, medical diagnosis, fraud