Machine Learning Module-4

Prof. Salma Itagi

Asst. Professor
Dept. of CSE,SVIT

Prof. Salma Itagi,Dept. of CSE, SVIT 1

 SYLLABUS
• Bayesian Learning: Introduction to Probability-based Learning, Fundamentals of Bayes
Theorem, Classification Using Bayes Model, Naïve Bayes Algorithm for Continuous
Attributes.
• Artificial Neural Networks: Introduction, Biological Neurons, Artificial Neurons,
Perceptron and Learning Theory, Types of Artificial Neural Networks, Popular Applications
of Artificial Neural Networks, Advantages and Disadvantages of ANN, Challenges of
ANN.

• Chapter-8 (8.1-8.4), Chapter-10 (10.1-10.5, 10.9-10.11)

• Text book: S Sridhar, M Vijayalakshmi, “Machine
Learning”, OXFORD University Press 2021, First
Edition.

Prof. Salma Itagi,Dept. of CSE, SVIT 2

Bayesian Learning-Introduction to Probabilistic learning

• Probability-based learning is one of the most important practical learning

methods which combing prior knowledge or prior probabilities with
observed data.
• Probabilistic learning uses the concept of probability theory that describes
how to model randomness, uncertainty, and noise to pred future events.
• It is a tool for modelling large datasets and uses Bayes rule to infer
unknown quantities, predict and learn from data.
• In a probabilistic model, randomness plays a major role which gives
probability distribution a solution, while in a deterministic model there is no
randomness hence it exhibits the same initial conditions every time the
model is run and is likely to get a single possible outcome as the solution.
Prof. Salma Itagi,Dept. of CSE, SVIT 3
Introduction to Probabilistic learning(cont..,)
• Bayesian learning differs from probabilistic learning as it uses
subjective probabilities (i.e., probability that is based on an
individual's belief or interpretation about the outcome of an event and
it can change over time) to infer parameters of a model.
• Two practical learning algorithms called Naïve Bayes learning and
Bayesian Belief Network (BBN) form the major part of Bayesian
learning.
• These algorithms use prior probabilities and apply Bayes rule to infer
useful information.

Prof. Salma Itagi,Dept. of CSE, SVIT 4

Fundamentals of Bayes Theorem

Prof. Salma Itagi,Dept. of CSE, SVIT 5

Classification using Bayes Model

Prof. Salma Itagi,Dept. of CSE, SVIT 6

Prof. Salma Itagi,Dept. of CSE, SVIT 7
Prof. Salma Itagi,Dept. of CSE, SVIT 8
Prof. Salma Itagi,Dept. of CSE, SVIT 9
Prof. Salma Itagi,Dept. of CSE, SVIT 10
Naïve Bayes Algorithm

Prof. Salma Itagi,Dept. of CSE, SVIT 11

Prof. Salma Itagi,Dept. of CSE, SVIT 12
ꞵ1

Prof. Salma Itagi,Dept. of CSE, SVIT 13

Prof. Salma Itagi,Dept. of CSE, SVIT 14
Prof. Salma Itagi,Dept. of CSE, SVIT 15
Prof. Salma Itagi,Dept. of CSE, SVIT 16
Prof. Salma Itagi,Dept. of CSE, SVIT 17
Prof. Salma Itagi,Dept. of CSE, SVIT 18
Bayes Optimal Classifier

Prof. Salma Itagi,Dept. of CSE, SVIT 19

Naïve bayes for continuous
attributes

1. Discrete continuous features to discrete features.

2. Apply Normal or Gaussian for continuous feature.

Prof. Salma Itagi,Dept. of CSE, SVIT 20

Prof. Salma Itagi,Dept. of CSE, SVIT 21
Prof. Salma Itagi,Dept. of CSE, SVIT 22
Prof. Salma Itagi,Dept. of CSE, SVIT 23
Prof. Salma Itagi,Dept. of CSE, SVIT 24
Prof. Salma Itagi,Dept. of CSE, SVIT 25
Prof. Salma Itagi,Dept. of CSE, SVIT 26
CHAPTER 2
Artificial Neural Networks: Introduction, Biological Neurons, Artificial Neurons, Perceptron and Learning
Theory, Types of Artificial Neural Networks, Popular Applications of Artificial Neural Networks, Advantages
and Disadvantages of ANN, Challenges of ANN.

Chapter-10 (10.1-10.5, 10.9-10.11)

Prof. Salma Itagi,Dept. of CSE, SVIT 27

Artificial Neural Networks-Biological Neurons
& Artificial Neurons

Prof. Salma Itagi,Dept. of CSE, SVIT 28

Simple Model of an Artificial Neuron

Prof. Salma Itagi,Dept. of CSE, SVIT 29

Prof. Salma Itagi,Dept. of CSE, SVIT 30
Artificial Neural Network Architecture

Prof. Salma Itagi,Dept. of CSE, SVIT 31

Activation Functions

Prof. Salma Itagi,Dept. of CSE, SVIT 32

Prof. Salma Itagi,Dept. of CSE, SVIT 33
Prof. Salma Itagi,Dept. of CSE, SVIT 34
Perceptron and Learning theory

Prof. Salma Itagi,Dept. of CSE, SVIT 35

Prof. Salma Itagi,Dept. of CSE, SVIT 36
Prof. Salma Itagi,Dept. of CSE, SVIT 37
Prof. Salma Itagi,Dept. of CSE, SVIT 38
Prof. Salma Itagi,Dept. of CSE, SVIT 39
Prof. Salma Itagi,Dept. of CSE, SVIT 40
Prof. Salma Itagi,Dept. of CSE, SVIT 41
Prof. Salma Itagi,Dept. of CSE, SVIT 42
Prof. Salma Itagi,Dept. of CSE, SVIT 43
XOR Problem

Prof. Salma Itagi,Dept. of CSE, SVIT 44

Delta Training Rule and Gradient
Descent

Prof. Salma Itagi,Dept. of CSE, SVIT 45

 Types of Artificial Neural
[Link]
Feed forward Neural Network

Prof. Salma Itagi,Dept. of CSE, SVIT 46

2. Fully Connected Neural
Network

Prof. Salma Itagi,Dept. of CSE, SVIT 47

3. Multilayer Perceptron

Prof. Salma Itagi,Dept. of CSE, SVIT 48

4. Feedback Neural Network

Prof. Salma Itagi,Dept. of CSE, SVIT 49

Applications of Neural Networks
ANN learning mechanisms are used in many complex applications that involve modelling of non-linear
processes. ANN is a useful model that can handle even noisy and incomplete data. They are used to model
complex patterns, recognize patterns and solve prediction problems like humans in many areas such as:
1. Real-time applications: Face recognition, emotion detection, self-driving cars, navigation systems, routing
systems, target tracking, vehicle scheduling, etc.
2. Business applications: Stock trading, sales forecasting, customer behaviour modelling, Market research and
analysis, etc.
3. Banking and Finance: Credit and loan forecasting, fraud and risk evaluation, currency price prediction, real-
estate appraisal, etc.
4. Education: Adaptive learning software, student performance modelling, etc.
5. Healthcare: Medical diagnosis or mapping symptoms to a medical case, image interpre- tation and pattern
recognition, drug discovery, etc.
6. Other Engineering Applications: Robotics, aerospace, electronics, manufacturing, commu- nications,
chemical analysis, food research, etc.

Prof. Salma Itagi,Dept. of CSE, SVIT 50

 Advantages of ANN
1. ANN can solve complex problems involving non-linear processes.
2. ANNs can learn and recognize complex patterns and solve problems as
humans solve a problem. 3. ANNs have a parallel processing capability and can
predict in less time.
4. They have an ability to work with inadequate knowledge. It can even handle
incomplete and noisy data.
5. They can scale well to larger data sets and outperforms other learning
mechanisms.

Prof. Salma Itagi,Dept. of CSE, SVIT 51

Limitations of ANN
1. An ANN requires processors with parallel processing capability to train the
network running for many epochs. The function of each node requires a CPU
capability which is difficult for very large networks with a large amount of
data.
2. They work like a 'black box' and it is exceedingly difficult to understand
their working in inner layers. Moreover, it is hard to understand the
relationship between the representations learned at each layer.
3. The modelling with ANN is also extremely complicated and the
development takes a much longer time.
4. Generally, neural networks require more data than traditional machine
learning algorithms, and they do not perform well on small datasets.
5. They are also more computationally expensive than traditional learning
techniques.
Prof. Salma Itagi,Dept. of CSE, SVIT 52
 Structure of a Decision Tree
• A decision tree has a structure that consists of a root node, internal nodes/decision
nodes, branches, and terminal nodes/leaf nodes.
• The topmost node in the tree is the root node.
• Internal nodes are the test nodes and are also called as decision nodes.
• These nodes represent a choice or test of an input attribute and the outcome or
outputs of the test condition are the branches emanating from this decision node.
• The branches are labelled as per the outcomes or output values of the test condition.
• Each branch represents a sub-tree or subsection of the entire tree.
•

Prof. Salma Itagi,Dept. of CSE, SVIT 53

Challenges of ANN

The major challenges while modelling a real-time application with ANNs are:
1. Training a neural network is the most challenging part of using this technique.
• Overfitting or underfitting issues may arise if datasets used for training are not
correct.
• It is also hard to generalize to the real-world data when trained with some
simulated data.
• Moreover, neural network models normally need a lot of training data to be robust
and are usable for a real-time application.
2. Finding the weight and bias parameters for neural networks is also hard and it is
difficult to calculate an optimal model.
Prof. Salma Itagi,Dept. of CSE, SVIT 54