BCSE401L: Internet of

Things
Module-3: Things Data
Analytics
Dr. Raju Patel
Associate Professor
School of Electronics Engineering (SENSE)
[Link]@[Link]
Introduction
• Smart things produce a large volume of data, which needs to be processed to
get some insights.
• There is a need to analyze the data and extract some meaningful patterns or
statistics for decision-making.
• According to American computer scientist, Tom M. Mitchell, learning is defined
as a system that learns from the experience and improves the performance for
a given task.
 Understanding the buzz words
• Artificial Intelligence (AI) helps in taking automated intellectual decisions like a
human. We can use artificial intelligence algorithms in IoT networks for making
decisions from a huge volume of data.
• Machine learning (ML) algorithms are used to learn from past experiences and
data.
• Deep learning (DL) system involves learning using multi-layer perceptrons and
takes more intelligent decisions.
• In traditional machine learning, features are extracted manually.
• Deep learning requires high-level abstract features which are extracted
automatically using the data.
What is Machine Learning?

A canonical definition by Tom Mitchell in 1997: “An agent is said to learn from experience (E) with
respect to some class of tasks (T), and the performance measure (P), if the learner's performance at
T, as measured by P, improves with E". One has to be very careful about defining the set of tasks T,
and the performance measure P. With experience E, the performance P has to improve.”
4
What is Machine Learning?

Machine learning can be defined as a subset of Artificial Intelligence (AI) that

allows a system/computer to learn from some available data. The data can either be
labelled (with a number, tag or type) or unlabelled.
F
5
Advantages and Disadvantages of Machine Learning
Advantages Disadvantages
1. Easy to identify the 1. Chances of error
patterns 2. Data acquiring and
2. No Human Intervention preprocessing
3. Wide range of applications 3. Time and resource
4. Scope for Continuous dependent
improvement 4. Human Expertise for
5. Handling Multi-variety result interpretation
F
Data
F

6
Applications of Machine Learning

7
Real world Applications of Machine Learning

8
Understanding the buzz words
 Types of machine learning algorithms

• There are mainly four types of machine learning algorithms.

1. Supervised Learning: Training data includes labels or desired outputs.
Example: classification and regression problems.
2. Unsupervised Learning: Training data does not include labels or desired outputs.
Example: clustering problem.
3. Semi-supervised Learning: Training data includes a few labels or desired
outputs. It is the combination of supervised and unsupervised learning.
4. Reinforcement Learning: Rewards from sequence of actions.
Supervised Learning:
Supervised Learning:
Supervised Learning: Supervised learning is a category
of machine learning that uses labeled datasets to train
algorithms to predict outcomes and recognize patterns.

Figure: Supervised Learning

 Training set/Validation Set/Test Set
• The Training Set
• It is the set of data that is used to train and make the model learn the hidden
features/patterns in the data.
• The training set should have a diversified set of inputs so that the model is trained in all
scenarios and can predict any unseen data sample that may appear in the future.
• The Validation Set
• The validation set is a set of data, separate from the training set, that is used to validate
our model performance during training.
• This validation process gives information that helps us tune the model’s
hyperparameters and configurations accordingly.
• The Test Set
• The test set is a separate set of data used to test the model after completing the training.
• It provides an unbiased final model performance metric in terms of accuracy, precision,
etc.
How to split your Machine Learning data?
 Classification and Regression
• We use supervised learning for regression and classification tasks.
• We categorize the test data among one of class in classification problem
• Predicting the dependent variable for the given test data is called a regression
problem.
Regression example: RSSI vs Distance
 Classification and Regression
• In supervised learning, the data (also called observations, measurements or
samples) are labeled with classes.
• Given x samples for an output function F (x), we use (x; F (x)) in supervised
learning.
• We predict function F (x) for new examples x, where discrete F (x) is for
classification problem and continuous F (x) is for regression problem. The
supervised learning has target labels.
• There are two types of supervised learning.
• First, the classification categorizes the samples into different classes.
• Second, the regression predicts an expected dependent variable using
interpolation and extrapolation.
 Linear Regression
• In regression, we predict the value of a continuous dependent variable given a
new independent variable. In linear regression, the fitted or modeled curve is
linear. Examples of regression include RSSI prediction at a given distance, heart-
rate prediction, and productivity prediction amongst others.
• Regression problem: Apply linear regression, y = alx + bl for the following data of
Table 6.3. Estimate the value of y when x = 8 using least square method.
• Here, x and y represent the day index and productivity of the corresponding day,
respectively.
Limitations of Linear Regression
• A linear regression gives outcome any value that may be less than
zero, greater than one, and between zero and one.
• On the other hand, the logistic regression gives output zero or one. If
the output is greater than some threshold, say, 0.5, the logistic
regression gives unity, otherwise zero.
Logistic Regression
free for five patients is given in Table 6.4.
Use logistic regression as the classifier.
Assume the hypothesis for linear regression
is :
2
• 1.× glucose
Determine level −of56.
the probability having a disease who has glucose level
30.
• 2. At least how much glucose level ensures disease case with a
probability of more than 98%?
 K Nearest Neighbor Classification
 The k-nearest neighbours algorithm, also known as KNN or k-NN,
is a non-parametric, supervised learning classifier, which uses
proximity to make classifications or predictions about the
grouping of an individual data point.

k-NN classification rule is to assign to a test sample the majority

category label of its k nearest training samples
In practice, k is usually chosen to be odd, so as to avoid ties
The k = 1 rule is generally called the nearest-neighbor
classification rule.
Definition of Nearest Neighbor

X X X

(a) 1-nearest neighbor (b) 2-nearest neighbor (c) 3-nearest neighbor

K-nearest neighbors of a record x are data points that

have the k smallest distance to x
Definition of Nearest Neighbor

X X X

(a) 1-nearest neighbor (b) 2-nearest neighbor (c) 3-nearest neighbor

K-nearest neighbors of a record x are data points

that have the k smallest distance to x
Value of K
• Choosing the value of k:
• If k is too small, sensitive to noise points
• If k is too large, neighborhood may include points from other
classes

Rule of thumb:
K = sqrt(N) X

N: number of training points

 Nearest Neighbor Classifiers
Basic idea:
If it walks like a duck, quacks like a duck, then it’s probably a
duck

Compute
Distance Test Record

Training Choose k of the

Records “nearest” records
Distance Metrics
K-Nearest Neighbour
• A supervised learning technique in which kn ϵ [1 n] is the number of
nearest samples with respect to a test sample.
• Note that n is the total number of samples.
• Based on majority voting among kn nearest neighbors, the sample is
classified into the majority class.
• Example: Perform kNN classification on the following raw dataset of a
smart home as shown in Table 6.1. Determine the class for
Temperature = 4 and Humidity = 8 with kn = 3.
Solution
Solution……continued
Naïve Bayes Classification
• The Naïve Bayes algorithm is comprised of two words Naïve and Bayes,
Which can be described as:

• Naïve: It is called Naïve because it assumes that the occurrence of a

certain feature is independent of the occurrence of other features. Such
as if the fruit is identified on the bases of color, shape, and taste, then
red, spherical, and sweet fruit is recognized as an apple. Hence each
feature individually contributes to identify that it is an apple without
depending on each other.
• Bayes: It is called Bayes because it depends on the principle of Bayes'
Theorem.
Bayes Theorem
Bayes Theorem
• Bayes' theorem is also known as Bayes' Rule or Bayes' law, which is
used to determine the probability of a hypothesis with prior
knowledge. It depends on the conditional probability.
• The formula for Bayes' theorem is given as:

P(A) is Prior Probability: Probability of hypothesis before observing the

evidence.
P(B) is Marginal Probability: Probability of Evidence.
Example: Consider the given raw dataset as shown
in Table 6.2. Perform
Naive Bayes classification algorithm and determine
the posterior probability if the weather is windy.
Example: Consider the given raw dataset as shown
in Table 6.2. Perform
Naive Bayes classification algorithm and determine
the posterior probability if the weather is windy.
(Outlook= Sunny, Temperature= Cool, Humidity=High, Wind- Strong)
(Outlook= Sunny, Temperature= Cool, Humidity=High, Wind- Strong)
 Unsupervised Learning
• Given x1 , x2 , ..., x n (without labels)
• Output hidden structure behind the x’s
– E.g., clustering

31
 What is Unsupervised Machine Learning?
• Discovering
patterns in
the data.
• It deals with
unlabelled
data.

• The process of finding cohesive groups in the input data is called

`clustering'.
• The process of finding the frequent co-occurance of items in the
data is called `association rule mining'.
56
How does Unsupervised Machine Learning Works?

57
Difference between Supervised and Unsupervised

58
Supervised and Unsupervised Algorithms

59
Unsupervised Learning
• We use unsupervised learning when labels of data are unknown. We know that the
clustering is an example of unsupervised learning. Given data samples, we group
the similar data or features. The flow-chart of unsupervised learning is shown in
Figure 6.5.
 Principal Component Analysis
• Principal Component Analysis (PCA) is an unsupervised learning approach for
dimensionality reduction technique of a large dataset. The original large dataset
is transformed for dimensionality reduction and reduced dimensionality
eliminates redundancy. This maximizes the variance along a few axes or
dimensions. Before applying PCA, we make the data set having zero mean using
the centering technique.
• Problem: Find the first principal component of the following data of Table 6.5.
Problem on PCA:

Step 1: Calculate Mean

Step 2: Calculation of the covariance matrix
Step 3: Eigenvalues of the covariance matrix
Step 4: Computation of the eigenvectors
Step 5: Computation of first principal components
 Principal Component Analysis (PCA)#Solved Example
Given the data in table, reduce the dimension from 2 to 1 using the Principal Component Analysis (PCA) algorithm.
Feature Example 1 Example 2 Example 3 Example 4
X1 4 8 13 7
X2 11 4 5 14

69
 Principal Component Analysis (PCA)#Solved Example
Given the data in table, reduce the dimension from 2 to 1 using the Principal Component Analysis (PCA) algorithm.
Feature Example 1 Example 2 Example 3 Example 4
X1 4 8 13 7
X2 11 4 5 14

Step-1: Calculate Mean

=8
=
=8.5
Step-2: Calculation of Covariance Matrix

𝐶𝑂𝑉 ( 𝑋 2 , 𝑋 2 )=23
𝑆=
[ 14
− 11
− 11
23 ]
Step-3: Eigen Values of the Covariance Matrix
The characteristic equation of the Covariance matrix is given by
det(S-λI)=0

=14 | 14 − λ
− 11
− 11
23 − λ | 70
=0
 Principal Component Analysis (PCA)#Solved Example
Step-3: Eigen Values of the Covariance Matrix

(14-λ)(23-λ)-(-11)×(-11)=0
37 λ − 20 1=0

𝑈 1=
Step-4: Computation of the Eigen Vectors (U) [ 14 − λ
11
]
Step-5: Computation of first principal
components
To find a unit eigen vector, we compute
the length of U1 which is given by

71
 Principal Component Analysis (PCA)#Solved Example
Step-5: Computation of first principal components

72
 Principal Component Analysis (PCA)#Solved Example
Step-6: Geometrical meaning of first principal components

73
 Principal Component Analysis (PCA)#Solved Example
Given the data in table, reduce the dimension from 2 to 1 using the Principal Component Analysis (PCA) algorithm.

74
 Bias and Variance in Machine Learning

• Choosing a classification algorithm or any algorithm for that matter in Supervised

Machine learning domain has to do with Bias Variance tradeoff and it’s a central issue
to it.

• Bias is defined as error is an error from erroneous assumptions in the learning

algorithm . High bias can cause an algorithm to miss the relevant relations between
features and target outputs (underfitting).
• Where as,
• Variance is defined as is an error from sensitivity to small fluctuations in the training
set. High variance may result from an algorithm modeling the random noise in the
training data (overfitting).
Underfitted and Overfitted Models
 Underfitting and Overfitting

• In under-fitting, the data is modeled by an order lower than the actual order of
the model. This is referred to as the high bias condition. The error is high for the
training and testing phases.
• On the other hand, in over-fitting, the data is modeled by an order higher than
the actual order of the model. This is also called high variance condition.
• In this scenario, the model memorizes the training data. If we apply unseen test
data to this over-fitted model, the error is high. This is because the learning of
the model is not proper for unseen data.
Bias Variance Tradeoff
Bias Variance Tradeoff- Choice of classifier
based on dataset size
• So expected behavior of some common classification algorithms provided similar conditions be
like:

• So in essence if the choice is based on data set size then go with models with High Bias and
Low Variance in case of lesser data and with high number of data points we can experiment
with other classification algo’s since that’ll give better result