20CS254

FOUNDATIONS OF
DATA SCIENCE

Dr. P. Perumal
Academic Year 2025-2026
Module 2
 MODULE II
RELATION TO STATISTICS
Representations of relations between
data and linear algebraic operations on
matrices
ML models Leveraging Linear
Algebra
 Contd..

 Machines or computers only understand numbers. And these

numbers need to be represented and processed in a way that
lets machines solve problems by learning from the data instead
of learning from predefined instructions (as in the case of
programming).
 Contd..

 All types of programming use mathematics at some level.

Machine learning involves programming data to learn the
function that best describes the data.

 The problem (or process) of finding the best parameters of

a function using data is called model training in ML.
 Contd..

 Therefore, in a nutshell, machine learning is programming

to optimize for the best possible solution – and we need
math to understand how that problem is solved.

 first step towards learning Math for ML is to learn linear

algebra.
Linear Algebra

 Linear Algebra is the mathematical foundation that solves the problem of

representing data as well as computations in machine learning models.

 It is the math of arrays — technically referred to as vectors, matrices

and tensors.

 In the ML context, all major phases of developing a model have linear

algebra running behind the scenes.
Common Areas of Application — Linear
Algebra in Action
Data Representation

 The fuel of ML models, that is data, needs to be converted into

arrays before you can feed it into your models. The computations
performed on these arrays include operations like matrix
multiplication (dot product). This further returns the output that
is also represented as a transformed matrix/tensor of numbers.
Word embeddings

 representing large-dimensional data (think of a huge

number of variables in your data) with a smaller
dimensional vector.
Contd..
 Contd..

 Natural Language Processing (NLP) deals with textual data.

Dealing with text means comprehending the meaning of a
large corpus of words. Each word represents a different
meaning which might be similar to another word. Vector
embeddings in linear algebra allow us to represent these
words more efficiently.
Eigenvectors (SVD)

 Finally, concepts like eigenvectors allow us to reduce the

number of features or dimensions of the data while keeping
the essence of all of them using something called principal
component analysis.
 Contd..

 Linear algebra basically deals with vectors and matrices

(different shapes of arrays) and operations on these arrays.
In NumPy, vectors are basically a 1-dimensional array of
numbers but geometrically, they have both magnitude and
direction.
 Contd…

 Our data can be represented using a vector. In the figure above,

one row in this data is represented by a feature vector which has
3 elements or components representing 3 different dimensions. N-
entries in a vector makes it n-dimensional vector space and in this
case, we can see 3-dimensions.
 Deep Learning — Tensors Flowing
Through a Neural Network

 We can see linear algebra in action across all the major applications
today. Examples include sentiment analysis on a LinkedIn or a
Twitter post (embeddings), detecting a type of lung infection from X-
ray images (computer vision), or any speech to text bot (NLP).
 All of these data types are represented by numbers in tensors. We
run vectorized operations to learn patterns from them using a neural
network. It then outputs a processed tensor which in turn is decoded
to produce the final inference of the model.
 Each phase performs mathematical operations on those data arrays.
Dimensionality Reduction — Vector
Space Transformation
Recommendation Engines — Making use
of embeddings
Vector Representation based on the
features
 Contd..

 here this matrix of 4X5, 4 rows, and 5 features, was broken

down into two matrices, one that's 4X2 and the other that's
2X5. We basically have new smaller dimensional vectors for
users and movies.
Plotting in 2D vector space
 Contd..

 The concept of a dot product (matrix multiplication) of

vectors tells us more about the similarity of two vectors. And it
has applications in correlation/covariance calculation, linear
regression, logistic regression, PCA, convolutions, PageRank
and numerous other algorithms.
Industries where Linear Algebra is used
heavily

 Statistics
 Chemical Physics
 Genomics
 Word Embeddings — neural networks/deep learning
 Robotics
 Image Processing
 Quantum Physics
How much Linear Algebra should you
know to get started with ML / DL?

 Now, the important question is how you can learn to program these concepts of
linear algebra. The answer is you don’t have to reinvent the wheel, you just
need to understand the basics of vector algebra computationally and you then
learn to program those concepts using NumPy.

 NumPy is a scientific computation package that gives us access to all the

underlying concepts of linear algebra. It is fast as it runs compiled C code and it
has a large number of mathematical and scientific functions that we can use.
 MODULE II
RELATION TO STATISTICS
Statistics: Descriptive Statistics
Statistics

 Statistics is one that is mainly focused on data collection, data

organization, data analysis, data interpretation, and data visualization.

 Earlier, statistics was practiced by statisticians, economists, business

owners to calculate and represent relevant data in their field.

 Nowadays, statistics has taken a pivotal role in various fields like data
science, machine learning, data analyst role, business intelligence
analyst role, computer science role, and much more.
 Basic Terminologies in Statistics

 Population: A population is the set of resources from where we can collect data
 Sample: A Sample is nothing but a subset of the Population which is used for sampling of
data and in inferential statistics to predict the outcome.
 Variable: A Variable can be a number, a characteristic, or a quantity that can be counted. It
can be also called a data point.
 Probability Distribution: A probability distribution is a mathematical concept that primarily
gives the probabilities of occurrence of different possible outcomes generally for an
experiment conducted by statisticians.
 Statistical Parameter: Statistical or population parameter is basically a quantity that helps
in indexing a family of probability distributions like the mean, median, or mode of a
population.
Descriptive Vs Inferential
Statistics

 DESCRIPTIVE STATISTICS : Descriptive Statistics is a

statistics or a measure that describes the data.

 INFERENTIAL STATISTICS : Using a random sample of data

taken from a population to describe and make inferences
about the population is called Inferential Statistics.
Descriptive Statistics

 Descriptive statistics uses data that provides a description of

the population either through numerical calculation or graph
or table. It provides a graphical summary of data. It is simply
used for summarizing objects, etc.
Categories of Descriptive
Statistics
Measure of central tendency

 Measure of central tendency is also known as summary

statistics that is used to represents the center point or a
particular value of a data set or sample set.

 Outliers influence the central tendency of the data.

 Outliers

 Outliers are extreme behaviours. An outlier is a data point that differs

significantly from other observations. It can cause serious problems in analysis.
 Solution for Outliers problem
Removing the outliers while taking average will give us good results.
i) Mean

 It is measure of average of all value in a sample set.

 Outliers can affect the central tendency of the data
 Example:
 ii) Median

 It is measure of central value of a sample set. In these, data set is

ordered from lowest to highest value and then finds exact middle.

 Example:
 Contd…

 Outliers don’t influence the data.

 iii) Mode

 It is value most frequently arrived in sample set. The value repeated

most of time in central set is actually mode.

 The mode can be calculated for both quantitative and qualitative data.

 Outliers don’t influence the data.

 Example:
Contd..
 Contd..

 If there is only one number that appears maximum number of times, the
data has one mode, and is called Uni-modal.
 If there are two numbers that appear maximum number of times, the
data has two modes, and is called Bi-modal.
 If there are more than two numbers that appear maximum number of
times, the data has more than two modes, and is called Multi-modal.
Measure of Variability

 Measure of Variability is also known as measure of dispersion

and used to describe variability in a sample or population.
i) Range

 It is given measure of how to spread apart values in sample

set or data set.

 Range = Maximum value - Minimum value

 Outliers influence the data

Example
 ii) Interquartile Range

 Interquartile Range are the points in the data set that divides the
data set into four equal parts. Q1, Q2 and Q3 are the first, second
and third quartile of the data set.

 The data should sorted and ordered from the smallest to the largest.

 Interquartile Range = Q3-Q1

Example 1

 Interquartile Range with Even Sample Size

Example 2

 Interquartile Range with Odd Sample Size

Variance

 The variance is a measure of variability. It is the average

squared deviation from the mean.

 The symbol σ2 represents the population variance and the

symbol for s2 represents sample variance.
 Example

 Calculate the population variance and sample variance

from the following 5 observations: 50, 55, 45, 60, 40.
Standard Deviation

 The Standard deviation is the measure of how far the data deviates from
the mean value.

 The standard deviation formula varies for population and sample. Both
formulas are similar, but not the same.

 Symbol used for Sample Standard Deviation – “s” (lowercase)

 Symbol used for Population Standard Deviation – “σ” (sigma, lower case)
 Example

 In a class of 50, 4 students were selected at random and their

total marks in the final assessments are recorded, which are: 812,
836, 982, 769. Find the standard deviation of their marks.
 MODULE II
RELATION TO STATISTICS
Probability and Distributions
Probability
Probability - Terminologies
Probability – Types of Events
Types of Probability
Marginal Probability
Joint Probability
Conditional Probability
Bayes Theorem
Probability - Applications
Probability Distributions
Probability Density Function
(PDF)
Probability – Normal Distribution
Contd..
Central Limit Theorem
 MODULE II
RELATION TO STATISTICS
Statistical Inferences
Population and Samples

 Parameter: It is a measure that could be mean, median, variance, and many

more for population data.
 Statistic: It is a measure that could be mean, median, variance, and many
more for sample data.
Relationship between a parameter and
a statistic considering the measure
“mean”
Sampling and its techniques

 Sampling is the process of selecting a sub-group of data points from the

population based on a certain logic.
 Techniques:
 Simple Random Sampling
 Systematic Sampling
 Stratified Sampling
Random Sampling

 In this method, each member of the population has an equal

chance of being selected in the sample.
Systematic Sampling

 In Systematic sampling, every nth record is chosen from the

population to be a part of the sample.
 Example: For instance, if a local NGO is seeking to form a
systematic sample of 500 volunteers from a population of 5000,
they can select every 10th person in the population to build a
sample systematically.
Stratified Sampling

 In Stratified sampling, a stratum is used to form samples from

a large population.

 A stratum is a subset of the population that shares at least

one common characteristic.

 After this, the random sampling method is used to select a

sufficient number of subjects from each stratum.
Example
 Inferential Statistics

 Inferential Statistics make predictions about a population

based on a sample of data taken from that population.
 The technique of Inferential Statistics involves the following
steps:
 First, take some samples and try to find one that represents
the entire population accurately.
 Next, test the sample and use it to draw generalizations about
the whole population.
Two main objectives of inferential
statistics

 Estimating parameters: We take a statistic from the collected data,

such as the standard deviation, and use it to define a more general
parameter, such as the standard deviation of the complete population.

 Hypothesis testing: Very beneficial when we are looking to gather

data on something that can only be given to a very confined
population, such as a new drug.
 Statistical terminologies

 Statistic: A Single measure of some attribute of a sample. For e.g., the

Mean/Median/Mode of a sample of Data Scientists in Bangalore.
 Population Statistic: The statistic of the entire population in context. For e.g.,
Population mean for the salary of the entire population of Data Scientists across India.
 Sample Statistic: The statistic of a group taken from a population. For e.g., the
Mean salaries of all Data Scientists in New york.
 Standard Deviation: It is the amount of variation in the population data, given by σ.
 Standard Error: It is the amount of variation in the sample data. It is related to
Standard Deviation as σ/√n, where n is the sample size.
Hypothesis Testing

 Hypothesis testing is the way of trying to make sense of

assumptions by looking at the sample data.
Type of Hypothesis

 Null hypothesis. The null hypothesis, denoted by Ho, is

usually the hypothesis that sample observations result
purely from chance.

 Alternative hypothesis. The alternative hypothesis,

denoted by H1 or Ha, is the hypothesis that sample
observations are influenced by some non-random cause.
Steps of Hypothesis Testing

State the hypotheses.

Formulate an analysis
plan.

Analyze sample data

Interpret results.
 Contd…

 State the hypotheses. This involves stating the null and alternative hypotheses, and both should be
mutually exclusive. That is, if one is true, the other must be false.

 Formulate an analysis plan. It describes how to use sample data to evaluate the null hypothesis. This
evaluation often focuses on a single test statistic. Here we choose the significance level (α) among 0.01, 0.05,
or 0.10 and also determine the test method.

 Analyze sample data. Find the value of the test statistic (mean score, proportion, t statistic, z-score, etc.)
and p-value described in the analysis plan.

 Interpret results. Apply the decision rule described in the analysis plan. If the value of the test statistic is
unlikely, based on the null hypothesis, reject the null hypothesis.
 Types of Errors in Hypothesis
Testing

 Type I error (false positive):

` This type of error happens when you say that the null hypothesis is false
when it is actually true. The probability of committing a Type I error is
called the significance level. This probability is also called alpha and is
often denoted by α.
Eg:The test result says you have coronavirus, but you actually don’t.
 Type II error (false negative):
This type of error happens when you say that the null hypothesis is
true when it is actually false The probability of committing a Type II error
is called beta and is often denoted by β
Eg: The test result says you don’t have coronavirus, but you actually do.
 Formulating the null and alternate
hypothesis

There is a common rule to formulate the null and alternate

hypotheses from the claim statement.

 The null hypothesis always has the following signs: = OR ≤ OR ≥

 The alternate hypothesis always has the following signs: ≠ OR >

OR <
 Types

 There are three types of tests which is based on ‘sign’ in the alternate hypothesis:

 ≠ in H₁ → Two-tailed test → Rejection/Critical region on both sides of the

distribution

 < in H₁ → Lower-tailed test → Rejection/Critical region on the left side of the

distribution

 > in H₁ → Upper-tailed test → Rejection region on the right side of the distribution
 Contd..

 If the average commute time is at least 30 minutes, then H₀ ≥ 30 and H₁< 30, that means the
test is a Lower Tailed test since the critical region will be on the left side of the distribution.
 If the average commute time is at most 30 minutes, then H₀≤ 30 and H₁> 30, that means the
test is an Upper Tailed test since the critical region will be on the right side of the distribution.
 If the average commute time is 30 minutes, then H₀= 30 and H₁≠30, that means the test is a
Two-Tailed test since the critical region will be on both sides of the distribution.
 To find the critical values for the critical region, we use the Critical Value Method or p-
Value Method.
Contd..
Rules

 We reject the null hypothesis(H₀) if the sample mean(x̅ ) lies

inside the Critical Region.

 We fail to reject the null hypothesis(H₀) if the sample mean(x̅ )

lies outside the Critical Region.
 Critical Value Method

 The steps to follow to make a decision using the critical value method are
as follows:
 Formulate the hypothesis and determine H₀ and H₁ and confirm the type
of test we have to do.
 Calculate the value of z-Critical Value(Zc) from the given value of
α(Significance Level).
 Calculate the critical values (UCV and LCV) from Zc based on the type of
test.
 The decision is based on the sample mean(x̅ ) for the critical values.
Example

Let’s say weather forecast claims that average rainfall in a

country is 350mm with a standard deviation(σ) of 90. Now,
we took 36 cities in the country as a sample and calculated
the average sample mean(x̅ ) as 370.16. We have to reject
or fail to reject the claim at 5% significance.
Step:1Formulating the
Hypothesis

 Null Hypothesis (H₀): μ = 350

 Alternate Hypothesis (H₁) ≠350

 Since H₁ contains ≠ sign, the test will be of a Two-tailed test

with a critical region on both sides of the normal distribution.
Step2: Find Zc value

 Given α = 0.05,

since it is a two-tailed test, the critical region lies on both sides

of distribution so that the significance level will be 0.025 on both
sides. That means the area of the critical region on the right side
would be 0.025. Which means the area till UCV (Cumulative
Probability till that point) would be 1–0.025 = 0.975.
Contd…
We need to find Z score at the value of 0.975
using Z-Table.

Zc = 1.96
Step:3 Calculate Critical Values

The formula to calculate the critical values is:

 UCV = μ+(Zc * σx)
 LCV = μ-(Zc * σx)
 σx = σ/√n = 90/√36 = 90/6 = 15
 UCV =350+(1.96*15) = 379.4
 LCV =350-(1.96*15) = 320.6
Let’s plot the all the values of μ, x̅ , UCV,
and LCV in the distribution graph and
make a decision.
Step:4 Making Decision

 The Sample Mean(x̅ ) lies outside the Critical Region.

 As the Sample Mean lies outside the Critical Region,

we fail to reject the null hypothesis.
p-Value Method

 p-Value is defined as the probability that the null hypothesis will not be
rejected.
 The following are the steps we need to follow to decide on the null
hypothesis using the p-value method:
 Formulate the hypothesis and determine H₀ and H₁ and confirm the type of
test we have to do.
 Calculate the value of Z for the sample mean.
 Calculate the p-value for the given z-score using the z-table.
 Make a decision based on the p-value for the given value of σ(significance)
Example

To find the p-value using z-score.

Situation 1: If the sample mean is on the right side of the distribution

mean, z-value= +3.02, then from Z-table, we can find the value = 0.9987
 For one-tailed test → p = 1–0.9987 = 0.0013
For two-tailed test → p =2(1–0.9987) = 0.0026

Situation 2: If the sample mean is on the left side of the distribution

mean, z-value= -3.02, then from Z-table, we can find the value = 0.0013
 For one-tailed test → p = 0.0013
For two-tailed test → p =2*0.0013= 0.0026
Example

 Let’s take the same weather forecast example

we’ve used for the critical value method.
We have μ = 350, x̅ =370.16, σ=90, α = 5%

 σx = σ/√n = 90/√36 = 15
1. Formulating the Hypothesis

 Null Hypothesis (H₀): μ = 350

 Alternate Hypothesis (H₁) ≠350

Since H₁ contains ≠ sign, the test will be of a Two-

tailed test with a critical region on both sides of the
normal distribution.
Contd..
2. Calculate the value of Z-score for
the sample mean
 Z =( x̅ -μ)/σx = (370.16–350)/15 = 1.34
 Using the Z-Table, we’ll find the cumulative probability for Z-
Value
P(Z=1.34) = 0.9099
3. Calculate the value of p

 Using the two situations mentioned earlier, since the sample

mean lies to the right side of the distribution mean.

 Since the sample mean is on the right side of the distribution

mean value and the test is of a two-tailed test,

 p =2(1–0.9099) = 0.1802
4. Make a Decision

 Since the p-value (0.1802) is greater than the value of α

(0.05), we fail to reject the null hypothesis.
 MODULE II
RELATION TO STATISTICS
Statistical Inferences
T-test

 T-tests are very much similar to the z-scores, the only

difference being that instead of the Population Standard
Deviation, we now use the Sample Standard Deviation.

 t-values are dependent on the Degree of Freedom of a

sample.
The Degree of Freedom

 Degrees of freedom of an estimate is the number of

independent pieces of information that went into
calculating the estimate.

 Degrees of Freedom = n – 1
Different types of T-test
Contd..
One Sample t-Test
Example
Steps
Steps
Contd…
Contd…
Contd…
Contd…
Contd…
Contd..
Independent Sample T-Test

 Independent sample t-test is a statistical technique that is used to

analyze the mean comparison of two independent groups.

 In independent samples t-test, when we take two samples from the

same population, then the mean of the two samples may be identical.

 But when samples are taken from two different populations, then the
mean of the sample may differ.
Example
Steps
Contd…
Contd..
 Contd…
0
Contd..
Contd..
Contd..
Contd..
 MODULE II
RELATION TO STATISTICS
Statistical Inferences
Paired Sample t-test

 Paired t tests are also known as dependent samples t tests.

The two samples are dependent because they contain the

same subjects.
Steps
Contd..
Contd..
Contd..
Chi-Square Test

 A Chi-square test is a hypothesis testing method. Two

common Chi-square tests involve checking if observed
frequencies in one or more categories match expected
frequencies.
Importance

 Chi-square test for hypothesis tests about whether your

data is as expected.

 The basic idea behind the test is to compare the observed

values in your data to the expected values that you would
see if the null hypothesis is true.
Two Measurements

 If you have a single measurement variable, you use a Chi-

square goodness of fit test.

 If you have two measurement variables, you use a Chi-

square test of independence.
Types of Chi-square tests

 Chi-square goodness of fit test

 Chi-square test of independence
Chi-square goodness of fit test

 The Chi-square goodness of fit test is a statistical hypothesis

test used to determine whether a variable is likely to come
from a specified distribution or not. It is often used to
evaluate whether sample data is representative of the full
population.
 Contd..

 To apply the goodness of fit test to a data set we need:

 Data values that are a simple random sample from the full
population.
 Categorical or nominal data. The Chi-square goodness of fit
test is not appropriate for continuous data.
 A data set that is large enough so that at least five values
are expected in each of the observed data categories.
Example

 Let’s use the bags of candy as an example. We collect a random sample

of ten bags. Each bag has 100 pieces of candy and five flavors. Our
hypothesis is that the proportions of the five flavors in each bag are the
same.

 Let’s start by answering: Is the Chi-square goodness of fit test an

appropriate method to evaluate the distribution of flavors in bags of
candy?
Contd..
Contd..
Contd..
Contd..
Chi-square test of Independence

 The Chi-square test of independence is a statistical

hypothesis test used to determine whether two categorical
or nominal variables are likely to be related or not.
Example

 Let’s take a closer look at the movie snacks example. Suppose we

collect data for 600 people at our theater. For each person, we know
the type of movie they saw and whether or not they bought snacks.

 Let’s start by answering: Is the Chi-square test of independence an

appropriate method to evaluate the relationship between movie type
and snack purchases?
Example
Assignment
 MODULE II
RELATION TO STATISTICS
Statistical Inferences
ANOVA

 ANOVA stands for Analysis of Variance. One-Way Analysis

of Variance tells you if there are any statistical differences
between the means of three or more independent groups.
Example

 An example to understand this can be prescribing medicines.

 Suppose, there is a group of patients who are suffering from fever.

 They are being given three different medicines that have the same
functionality i.e. to cure fever.

 To understand the effectiveness of each medicine and choose the

best among them, the ANOVA test is used.
Contd..
Types of ANOVA

1. One-Way ANOVA
2. Two-Way ANOVA
3. N-Way ANOVA (MANOVA)
One-Way ANOVA

 One-way ANOVA is generally the most used method of

performing the ANOVA test. It is also referred to as one-
factor ANOVA, between-subjects ANOVA, and an
independent factor ANOVA. It is used to compare the
means of two independent groups
Example Problem
Steps
Contd..
Contd..
Contd..
Contd..
Contd..
Contd..
Contd..
Contd..
Contd..
Contd..
Two way ANOVA

 The two-way analysis of variance is an extension to the one-

way analysis of variance. There are two independent
variables (hence the name two-way).
Example
Steps
Contd…
Contd…
Contd..
Contd..
Contd..
Contd..
Contd..
Contd..
Contd..
Contd..
Contd..
Contd..
Contd..
Contd..
Contd..
Contd..
Contd..
Contd..
N-Way ANOVA

 It use more than two independent variables, and this is an

n-way ANOVA (with n being the number of independent
variables you have). For example, potential differences in
IQ scores can be examined by Country, Gender, Age group,
Ethnicity, etc, simultaneously.
 Statistical Modeling

 What is a Statistical Model?

A statistical model is a type of mathematical model that comprises of

the assumptions undertaken to describe the data generation process.

Assumption (Example): Assuming that we have 2 fair dice, and each face has equal
probability to show up i.e. 1/6. Now, we can calculate the probability of two dice
showing up 5 as 1/6*1/6. As we can calculate the probability of every event, it
constitutes a statistical model.
Why do we need Statistical
Modeling?
The statistical model plays a fundamental role in carrying out statistical
inference which helps in making propositions about the unknown properties and
characteristics of the population as below:
1) Estimation:
 It is the central idea behind Machine Learning i.e. finding out the number which
can estimate the parameters of distribution.
 Note that the estimator is a random variable in itself, whereas an estimate is a
single number which gives us an idea of the distribution of the data generation
process. For example, the mean and sigma of Gaussian distribution
 Why do we need Statistical Modeling?

2) Confidence Interval:
 It gives an error bar around the single estimate number i.e. a range of values to
signify the confidence in the estimate arrived on the basis of a number of samples.
For example, estimate A is calculated from 100 samples and has a wider confidence
interval, whereas estimate B is calculated from 10000 samples and thus has a
narrower confidence interval
3) Hypothesis Testing
 It is a statement of finding statistical evidence. Let’s further understand the need to
perform statistical modeling with the help of an example below.
 Types of modeling assumptions

1) Parametric:
 It assumes a finite set of parameters which capture everything
about the data. If we know the parameter θ which very well
embodies the data generation process, then predictions (x) are
independent of the observed data (D)
2) Non-parametric:
 It assumes that no finite set of parameters can define the data
distribution. The complexity of the model is unbounded and grows
with the amount of data
Types of modeling assumptions

3) Semi-parametric:

 It’s a hybrid model whose assumptions lies between

parametric and non-parametric approaches. It consists of
two components – structural (parametric) and random
variation (non-parametric). Cox proportional hazard model is
a popular example of semi-parametric assumptions.
 Techniques in Statistical
Modeling
There are several statistical modeling techniques used during data exploration.
Here are some of the common techniques:
A) Linear Regression
Linear regression uses a linear equation to model the relationship between
two variables, where one variable is dependent and the other is independent.
 If one independent variable is utilized to predict a dependent variable, it is
called simple linear regression.
 If more than one independent variable is used to predict a dependent
variable, it’s called a multiple linear regression.
 Techniques in Statistical
Modeling
B) Classification
Classifications groups the data into different categories to allow for a more accurate
prediction and analysis. This technique can enable effective analysis of very large data
sets. There are two major techniques under classification:
1. Logistic Regression
When the dependent variable is binary, the logistic regression technique is used to model
and predict the relationship between the binary variable and one or more independent
variables.
2. Discriminative Analysis
Here, two or more groups are known as prior and new observations are grouped into
known clusters based on the measured features. The distribution of the predictor variable
X is modeled separately into each of the response classes, Bayes’ theorem is then used to
calculate the probability of each response class, based on the value of X.
Techniques in Statistical
Modeling
C) Resampling

In this technique, repeated samples are drawn from the original set of
data, creating a unique sampling distribution based on actual data. It uses
experimental methods as opposed to analytical methods to create a
unique sampling distribution. Since the samples drawn are unbiased, the
estimates obtained are also unbiased.

 Knowledge of two main concepts are essential to understand the

concept of resampling in its entirety:
 Techniques in Statistical
Modeling

1. Bootstrapping
This takes into account the data samples that weren’t selected in
the initial sample as a replacement. The process is repeated several
times and the average score is calculated for the estimation of the
model performance.
2. Cross-Validation
The training data is divided into k number of parts. Here, k – 1 parts
are considered training sets, and the one remaining set is used as
the test set. This is repeated k number of times and the average of
the k scores are calculated as the performance estimation.
 Techniques in Statistical
Modeling
D) Non-linear Models
Here the data under observation is modeled using a non-linear
combination of model parameters and this is dependent on one or more
independent variables. The data is then fitted using a method of successive
approximations.
E) Tree-Based Methods
In a tree-based method, the predictor space is segmented into different
simple regions. The set of splitting rules can be summarized in a tree,
giving it the name decision-tree method. This can be used for both,
regression and classification problems. Bagging, boosting, and random
forest algorithm are some of the approaches used in this method.
 Techniques in Statistical
Modeling

F) Unsupervised Learning
Unsupervised learning relies on the algorithm to identify a
pattern in the data. Here the categories of data are not known.
For example, in clustering, closely related items are grouped,
making it a method of unsupervised learning.
G) Time Series
This forecasting model can be used to predict future values
based on historical values. It is used to identify the phenomenon
represented by the data and then integrated with other data to
draw predictions for the future.
 Techniques in Statistical
Modeling

H) Neural Networks

 Modeled loosely on the human brain, these are algorithms designed to

identify patterns in the data. Neural networks have non-linear elements
that process information, called neurons.

 These are arranged in layers and normally executed in parallel. Neural

networks are being increasingly used to make predictions and
classifications as they have minimal demands on assumptions and model
structure and can approximate a wide range of models.
 Types of Statistical Models

 The different types of statistical models are essentially the

statistical methods used for computation. These are the
mathematical equations and visual representations that make
statistical modeling possible. Some of them are:
Linear regression Experimental design
Logistic regression Bayesian theory – Naïve Bayes
Cluster analysis classifier
Pearson’s r
Factor analysis
Sampling
Analysis of variation (ANOVA) Association rules
Chi-squared test Matrix operations
Correlation K-nearest neighbor algorithm (k-
Decision trees NN)
Time series
Fitting a model

 Fundamental concepts: Finding “optimal” model

parameters based on data; Choosing the goal for data
mining; Objective functions; Loss functions.

 Exemplary techniques: Linear regression; Logistic

regression; Support-vector machines.
Fitting a model -Linear
Regression

 Linear Regression is a machine learning algorithm based

on supervised learning

 It is mostly used for finding out the relationship between variables and
forecasting.
 Contd..

 Linear regression performs the task to

predict a dependent variable value (y)
based on a given independent variable
(x). So, this regression technique finds out
a linear relationship between x (input)
and y(output). Hence, the name is Linear
Regression. In the figure above, X (input)
is the work experience and Y (output) is
the salary of a person. The regression line
is the best fit line for our model.
Formula –Linear Regression

 y = b0 + b1 x
Where,

X – the value of the independent variable,

Y – the value of the dependent variable.
a – is a constant (shows the value of Y when the value of X=0)
b – the regression coefficient (shows how much Y changes for
each unit change in X)
Contd…
END OF MODULE II