Machine Learning

• Data: Vital ingredient in AI and ML.

• Converted to a form which is efficient for processing and transfer.
• It can be structured and unstructured.
• Data is collected to be measured, reported, visualized and analyzed.
• Types of data:
• 1. Visual
• 2. Textual
• 3. Numerical
Contd..
• 1. Visual data: made of images. Example: Computer vision.

• 2. Textual data: gathered via camera or scanners or digital documents.

• Organized into linguistically relevant characters, words, sentences and
concepts.
• Example: NLP

• 3. Numerical: Made up of figures and measurements.

• Gathered by machines, sensors or people.
• Example: Driver analysis.
Contd..
• Dataset: A collection of related data points: with an order.

• Types:
• 1. Training dataset: used during the process of training a machine
learning algorithm
• 2. Test dataset: Unlabelled data used to check that a ML model is able
to perform its assigned task.
Contd..
Contd..
• Overfitting and Underfitting are two crucial concepts in
machine learning and are the prevalent causes for the
poor performance of a machine learning model.
• 1. An underfit model has poor performance on the training
data and will result in unreliable predictions.(The size of
the training dataset used is not enough)

• 2. When a model performs very well for training data but

has poor performance with test data (new data), it is
known as overfitting. (Data used for training is not cleaned
and contains noise (garbage values) in it)
Modelling in ML
• Model: A program that analyses datasets to find patterns and make
predictions.
• Modelling: The process of development and implementation of ML
model.
• AI modelling: Replicates human intelligence and is effective when
receive data points.
Discriminative Modelling
• Discriminative models are a type of machine learning model that
are used to classify data into different groups.
• They are also known as conditional models.
• Discriminative models are algorithms designed to directly model
and learn the boundary between different classes or categories in a
dataset.
• They work by estimating the conditional probability distribution
of the output variable given the input features, focusing on
discerning the decision boundary that separates distinct classes,
enabling classification or prediction tasks.
• Discriminative models aim to capture the relationship between
the input features and the output labels
Importance of Discriminative
models
• 1. Discriminative models play a crucial role in various
machine learning applications due to their effectiveness
in classification and regression tasks.
• 2. Their significance lies in their ability to directly model
the decision boundary between different classes,
allowing for precise predictions and classifications
based on input features.
• 3. Discriminative models are particularly adept at
capturing complex relationships within data, enabling
accurate classification.
• 4. This efficiency is vital in scenarios where the primary
goal is to assign inputs to specific categories or predict
numerical values with high precision.
Contd..
• 5. Discriminative models are versatile and can be
applied to a wide range of datasets, making them
valuable in diverse domains.
• 6. They can handle complex, high-dimensional data and
adapt well to different types of input features, making
them suitable for various real-world applications.
Characteristics of Discriminative
models
• Handles complex data: Can handle high-dimensional
data and adapt to different input features
• Learns decision boundaries: Can use large amounts of
labeled data to learn complex decision boundaries
• Estimates probabilities: Can estimate the probability
that an instance belongs to a class
• Used for supervised learning: Most discriminative
models are used for supervised machine learning
Applications of Discriminative
models
• Discriminative models are useful for a variety of applications, including:
• Natural language processing,
• Image recognition,
• Healthcare analytics.
• Discriminative models often yield high predictive accuracy, enabling
companies to make more informed decisions. Whether it's predicting
customer preferences, fraud detection, or demand forecasting, the precision
offered by discriminative models can lead to better outcomes and resource
allocation.
• In industries like e-commerce, advertising, and recommendation systems,
discriminative models excel at personalization. By understanding and
exploiting patterns in user behavior, these models empower companies to
tailor their products, services, and content to individual preferences,
enhancing the overall user experience.
Gradient Descent algorithm
• Gradient:
• 1. The degree to which a road, etc. goes up or down
OR
• 2. A part sloping upward or downward.
OR
• 3. Change in the value of a quantity (such as temperature,
pressure, or concentration) with change in a given variable,
especially per unit distance in a specified direction.
• Descent:
• 1. a movement down to a lower place
• OR
• 2. the act or process of descending from a higher to a
lower level, rank, or state. (begin our descent down the
mountain).

• Lines have a positive gradient as they travel in an

upwards direction from left to right and a line traveling
in a downward direction from left to right has a negative
gradient.
Gradient descent
• Gradient descent is an optimization
algorithm commonly used to train machine learning
models and neural networks.
• It trains machine learning models by minimizing errors
between predicted and actual results.
• Once machine learning models are optimized for
accuracy, they can be powerful tools for AI and
computer science applications.
Gradient descent
• It is a widely used algorithm in ML to find the set of
parameters that minimize the cost function of a model
and for finding the local minimum of a function.[The
local minimum is a point within an interval at
which the function has a minimum value.]
• The algorithm works by iteratively adjusting the
parameters in the direction of the negative gradient of
the cost function.
• The cost function represents the discrepancy between
the predicted output of the model and the actual
output.
Cost function
• It is a function that measures the performance of a
model for any given data.
• Cost function quantifies the error between predicted
and expected values and presents it as a single real
number.
• After making a hypothesis with initial parameters, we
calculate the Cost function.
• Intending to reduce the cost function, we modify the
parameters by using the Gradient descent algorithm
over the given data.
Contd..
• Gradient descent can be applied to various machine-
learning algorithms, including
• 1. Linear regression
• 2. Logistic regression
• 3. Neural networks
• 4. Support Vector machines
Example of Gradient Descent
Algorithm
• Let’s say you are playing a game in which the players
are at the top of a mountain and asked to reach the
lowest point of the mountain.
• Additionally, they are blindfolded.
• So, what approach do you think would make you reach
the lake?
• The best way is to observe the ground and find where
the land descends.
• From that position, step in the descending direction and
iterate this process until we reach the lowest point.
• Finding the lowest point in a hilly landscape.
• The goal of the gradient descent algorithm is to
minimize the given function (say, cost function). To
achieve this goal, it performs two steps iteratively:
[Link] the gradient (slope), the first-order
derivative of the function at that point
[Link] a step (move) in the direction opposite to
the gradient. The opposite direction of the slope
increases from the current point by at at that point
Modelling technique
• Prediction modelling:
• This is the development of models which can forecast future events,
trends or patterns based on historical data.
• Business use these models to make informed decisions for future
endeavours.
• This technique utilizes statistical and data science methods.
• These is used in marketing, banking, finance, healthcare and weather
forecast.
Contd..
• It involves building a mathematical model that takes
relevant input variables and generates a predicted
output variable.
• A Predictive model is concerned with minimizing the
error of a model or making the most accurate
predictions possible.
• Predictive Machine learning algorithms are used to train
and improve these models to help you make better
decisions.
• Predictive modeling is used in many industries and
applications and can solve a wide range of issues, such
as fraud detection, customer segmentation, disease
Process

Get data Prepare data Train model Test data Refine

Probabilistic models
• A Probabilistic model in machine learning is a mathematical
representation of a real-world process that incorporates uncertain or
random variables.
• Probabilistic interpretation is used for making predictions based on
fundamental principles of probability and statistics.
• The goal of probabilistic modeling is to estimate the probabilities of
the possible outcomes of a system based on data or prior knowledge.
• Probabilistic models are used in a variety of machine learning tasks
such as classification, regression, clustering, and dimensionality
reduction.
• Example, In the case of tossing a fair coin, the probability of getting a
Head is 1/2.
Regression
• Regression analysis is a fundamental concept in the field of ML.
• It falls under supervised learning wherein the algorithm is trained with both
input features and output labels.
• It helps in establishing a relationship among the variables by estimating
how one variable affects the other.
• Regression in machine learning consists of mathematical methods that
allow data scientists to predict a continuous outcome (y target variable)
based on the value of one or more predictor variables (x).
• Linear regression is probably the most popular form of regression analysis
because of its ease of use in predicting and forecasting.
• Regression in machine learning is a technique used to capture the
relationships between independent and dependent variables, with the main
purpose of predicting an outcome.
What are regression models
used for?
• Machine learning regression models are mainly used in
predictive analytics to forecast trends and predict outcomes.
• Regression models are trained to understand the relationship
between different independent variables and an outcome.
• The model can therefore understand the many different
factors which may lead to a desired outcome.
• The resulting models can be used in a range of ways and in a
variety of settings.
• Outcomes can be predicted from new and unseen data,
market fluctuations can be predicted and accounted for, and
campaigns can be tested by tweaking different independent
variables.
Common uses for machine learning
regression models include:
• Forecasting continuous outcomes like house prices,
stock prices, or sales.
• Predicting the success of future retail sales or marketing
campaigns to ensure resources are used effectively.
• Predicting customer or user trends, such as on
streaming services or e-commerce websites.
• Predicting interest rates or stock prices from a variety of
factors.
Types of Regression
• 1. Linear Regression
• 2. Logistic regression
Linear Regression
• A Supervised Learning algorithm.
• One of the most well-known and well-understood algos used in Stats and
ML.
• Make predictions about future events based on past data.
• LR makes predictions for continuous or real or numerical variables such as
Sales
Salary
Age
Product price
Contd..
• LR shows a linear relationship between a dependent(Y) and one
independent variable (X).
• It finds how the dependent variable's value changes according to the
independent variable's value.
• LR predicts a dependent variable value (Y) based on a given
independent variable (X).
• Linear regression uses the relationship between the data points to
draw a straight line through all of them.
• This line can be used to predict future values.
Contd..
• The independent variable is also the predictor or
explanatory variable that remains unchanged due to the
change in other variables.
• However, the dependent variable changes with
fluctuations in the independent variable.
• The regression model predicts the value of the
dependent variable, which is the response or outcome
variable being analyzed or studied.
Contd..
• Thus, linear regression is a supervised learning
algorithm that simulates a mathematical relationship
between variables and makes predictions for continuous
or numeric variables such as sales, salary, age, product
price, etc.
• This analysis method is advantageous when at least two
variables are available in the data, as observed in stock
market forecasting, portfolio management, scientific
analysis, etc.
Contd..
• The relation is usually a straight line that best fits the
different data points as close as possible.
• The output is of a continuous form, i.e., numerical value.
• For example, the output could be revenue or sales in
currency, the number of products sold, etc.
• For example, predicting the price of a house by
analyzing sales data of that region.
• In the above machine learning example in linear
regression, the independent variable can be single or
multiple.
Contd..
• In the above figure,
• X-axis = Independent variable
• Y-axis = Output / dependent variable
• Line of regression = Best fit line for a model

• Here, a line is plotted for the given data points that suitably
fit all the issues.
• Hence, it is called the ‘best fit line.’
• The goal of the linear regression algorithm is to find this best
fit line seen in the above figure.
Contd..
• Mathematically these slant lines follow the following
equation,
• Y = m*X + b
• Where X = dependent variable (target)
• Y = independent variable
• m = slope of the line (slope is defined as the ‘rise’ over
the ‘run’)
For example, suppose we want to predict a car's fuel efficiency in miles per gallon based on how heavy the car is, and we have the following dataset:

Pounds in 1000s (feature) Miles per gallon (label)

3.5 18

3.69 15

3.44 18

3.43 16

4.34 15

4.42 14

2.37 24
If we plotted these points, we'd get the following graph:

Car heaviness (in pounds) versus miles per gallon rating. As a car gets heavier, its
miles per gallon rating generally decreases.
We could create our own model by drawing a best fit line through the points:

A best fit line drawn through the data from the previous figure.
Contd..
• The line of best fit in linear regression is a straight line
that shows the relationship between variables in a
dataset. It's also known as a trendline.
The types of linear regression models include:
• 1. Univariate linear Regression: When each observation or
data point corresponds to a single variable.
• Example: Suppose the heights of seven students in a class
are to be recorded. There is only one variable, which is height,
and is not dealing with any cause or relationship.

• 2. Multivariate linear Regression: It involves two different

variables and the analysis of this type of data focuses on
understanding the relationship between these two variables.
• Example: Suppose ice cream sales and temperature are the
two variables of bivariate data. The relationship between the
two is directly proportional to each other and thus related
because as the temperature increases, the sales also
increase.
Logistic Regression
• Logistic regression is a supervised learning algorithm that
uses logistic functions to predict the probability of a
binary outcome.
• A statistical method used for building machine learning models where
the dependent variable is dichotomous ie binary: which means a
variable that has 2 outputs,
• For example, A person will survive this accident or not,
The student will pass this exam or not.
• The outcome can either be yes or no (possible outputs).
• This regression technique is similar to linear regression
and can be used to predict classification problems’. We can
say that it is also regression but with targets y=(0,1), i.e., it is
classification.
Contd..
• Logistic Regression is basically a predictive model analysis
technique where the target variables (output) are discrete
values for a given set of features or input (X).
• For example whether someone is covid-19 positive (1) or
negative (0).
• It is a very powerful yet simple classification algorithm in
machine learning borrowed from statistics algorithms.
• Around 60% of the world’s classification problems can be
solved by using the logistic regression algorithm.
Contd..
• Used to describe data and relationship between one dependent and
one or more independent variables.
• It can work with both numerical and categorical data,
making it versatile for various applications.
• Logistic regression models the data using the sigmoid function to
estimate the probability of an event occurring, such as voted or didn’t
vote, based on a given data set of independent variables.
• Since the outcome is a probability, the dependent variable is bounded
between 0 and 1.
Contd..
• Use: Perfect for classification tasks, especially binary outcomes (yes/no,
true/false), or a percentage.

• How it works: The logistic function estimates the probability that an instance
belongs to a particular class.

• Example: Predicting whether an email is spam or not based on features like

sender, subject, and content.

• When to use: When dealing with classification problems, like spam detection
or medical diagnosis.
Contd..
• Logistic regression is used to find the probability of event=success and
event=failure.
• We should use logistic regression when the dependent variable is
binary (0/1, teue/false, yes/no) in nature.
• Here the value of Y ranges from 0 to 1 and it can be represented by
following equation:
• Odds=p/(1-p)= probability of an event occurrence/not
• ln(odds)=ln(p/(1-p))
• logit(p)=ln(p/1-p))=b0+b1x1+b2x2+b3x3…+bkxk
Logistic model

Log (Likelihood)
sex(0/1)
Diet score(0-15)
Age group(0/1)
 Logistic Function - Sigmoid Function

• The sigmoid or logistic function is essential for

converting predicted values into probabilities in logistic
regression.
• This function maps any real number to a value between
0 and 1, ensuring that predictions remain within this
probability range.
• Its "S" shaped curve helps translate raw scores into a
more interpretable format.
• A threshold value is used in logistic regression to make
decisions based on these probabilities.
Contd..
• For instance, if the predicted probability is above a
certain threshold, such as 0.5, the result is 1. If it’s
below, it’s classified as 0.
• This approach allows for clear and actionable outcomes,
such as determining whether a customer will purchase a
product or a patient has a particular condition based on
the probability calculated by the sigmoid function.
• The sigmoid curve is the visualization like intuition, acting as a
boundary, guiding logistic regression on differentiating and deciding
between choices.
Key points:
• It predicts the output of categorical dependent variables. Therefore
the outcome must be categorical or discrete value.

• It can be either Yes or NO, 0 or 1, True or False, etc. but instead of

giving the exact value as 0 or 1, it gives the probabilistic values which
lie between 0 and 1.

• In logistic function, instead of fitting a regression line, we fit an “S”

shaped logistic function, which predicts two minimum values (0 or 1).
Example
• At a buffet, you are picking for a full-bellied friend.
• Holding up options like chocolate cake or strawberry ice cream, they
give a thumbs-up for “Yes” or a thumbs-down for “No”.
• Over time, you start predicting flavors they will pick next time based
on their past choices.
• Logistic regression works similarly.
• It observes patterns in data and makes educated guesses about
whether something will or won’t happen, just like you deduce your
friend’s choice.
 In this equation:
f(x) represents the output of the sigmoid function.
x is the input, which can take any real value.
e is the base of the natural logarithm, approximately equal to 2.71828.
The sigmoid function takes any real number as input and outputs a value between 0 and 1. It approaches 0 as the input becomes negative and 1
as the input becomes positive.
When the input is 0, the sigmoid function returns 0.5.
Assumption in a Logistic Regression Algorithm

• In a binary logistic regression, the dependent variable

must be binary
• Only meaningful variables should be included
• The independent variables should be independent of
each other. This means the model should have little or
no multicollinearity
• Logistic regression requires quite large sample sizes
Support Vector Machine (SVM)
• A way for machines to put things into groups based on their features-
like a seesaw separates things based on weight.
• One of the most popular supervised learning models.
• Used for classification as well as regression problems.
• However, primarily, it is used for Classification problems in Machine
Learning.
• Works best on smaller datasets.
Contd..
• Used for image classification, spam detection, handwriting
identification, face detection, and anomaly detection.
• The goal of this algo is to create the best line or decision boundary
that can segregate n-dimensional space into the correct category. This
best decision boundary is called a hyperplane.
• The hyperplane tries to make the margin between the closest points
of different classes to be as maximum as possible.
• SVM chooses the extreme points/vectors that help create the hyperplane.
• These extreme cases are called support vectors and hence algo is termed
as SVM.
• The basic idea behind SVM is to find the possible boundary that separates
the different classes in the dataset.
• This boundary is called hyperplane and the goal is to find the hyperplane
that maximizes the margin. I.e., the distance between the hyperplane and
the nearest data points from each other.
• Technically, the primary objective of the SVM algorithm is to identify a
hyperplane that distinguishably segregates the data points of different
classes.
• The hyperplane is localized in such a manner that the largest margin
separates the classes under consideration.
Example
• Imagine you are at a house party.
• There is an IPL match today and the new Shahrukh Khan movie is
dropping out on OTT at the exact same time.
• And you, the host, with one TV, are trying to keep the movie buffs
away from the cricket fans.
• You are trying to find a no-man’s land between both.
• That’s SVM in ML.
• It finds the best “no-man’s land” between two groups, making them
as far as possible.
Contd..
• It’s like the ultimate party planner for your data.
• Hence there are 2 clusters of data points: “Movie buffs” and “Cricket
fans“.
• The SVM separates them and draws the dashed line “no-man’s land”
between them.
• If we plot the data points in the 2-D graph, it is a straight line, but if we
have more dimensions, we call this decision boundary a “hyperplane”.
• The points closest to the line are called Support Vectors.
• The best hyperplane is that plane with the maximum distance from
both the classes- this is the main aim of SVM.
How Does Support Vector
Machine Work?
• Let’s understand the working of SVM using an example.
Suppose we have a dataset that has two classes (green
and blue). We want to classify that the new data point
as either blue or green.
• To classify these points, we can have many decision
boundaries, but the question is which is the best and
how do we find it?
• NOTE: Since we are plotting the data points in a 2-
dimensional graph we call this decision boundary
a straight line but if we have more dimensions, we call
this decision boundary a “hyperplane”
• The best hyperplane is that plane with the maximum
distance from both classes, which is the main aim of
SVM.
• This is done by finding different hyperplanes that
classify the labels in the best way.
• Then, it will choose the one farthest from the data
points or the one with a maximum margin.
Types of SVM
• SVM is of two types:
• 1. Linear SVM: used for linearly separable data, which means if a
dataset can be classified into 2 classes by using a straight line, such
data is termed as linearly separable data, and the classifier is called
linear SVM classifier.
• 2. Non-linear SVM: used for non-linearly separated data where if a
dataset can not be classified by using a straight line.
Kernel function in SVM
• It is a method used to take data as input and transform it into the
required form of processing data.
• Kernel is a set of mathematical functions used in SVM as
hyperparameters providing the window to manipulate the data.
• The choice of kernel function can significantly impact the
performance of the SVM model.
• 1. Nonlinear separability: allows to find a hyperplane that can
separate the classes that were previously inseparable in the original
feature space.
Contd..
• 2. Efficiency: Makes SVM work efficiently in high-dimensional feature
spaces as a kernel function. Only needs to calculate the similarity
between the input data points.
• 3. Flexibility: Can be used with a wide range of input data types,
including continuous, discrete, and categorical data.

• Some kernel functions are linear, polynomial and sigmoid.