Scribd
Upload
2 views20 pages

Introduction to Artificial Neural Networks

The document outlines a session aimed at familiarizing students with Artificial Neural Networks (ANN) and their application to real-world problems. It includes instructional objectives, learning outcomes, and key concepts such as overfitting, underfitting, and optimization techniques. Additionally, it presents practical questions related to backpropagation, loss functions, and gradient descent for hands-on learning.

Uploaded by

hope98754
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
2 views20 pages

Introduction to Artificial Neural Networks

The document outlines a session aimed at familiarizing students with Artificial Neural Networks (ANN) and their application to real-world problems. It includes instructional objectives, learning outcomes, and key concepts such as overfitting, underfitting, and optimization techniques. Additionally, it presents practical questions related to backpropagation, loss functions, and gradient descent for hands-on learning.

Uploaded by

hope98754
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd

ANN

Course: Nature inspired soft computing

Module - 1 CO - 2
AIM OF THE SESSION

To familiarize students with the concepts of ANN ,


To make students apply ANN on a real world problem

INSTRUCTIONAL OBJECTIVES

This unit is designed to:


1. Demonstrate the ANN and its concepts
2. Describe the nature and features of the ANN
3. List out the techniques of evolution used in the ANN
4. Demonstrate the process of optimization in GA

LEARNING OUTCOMES

At the end of this unit, you should be able to:


1. Define the functions of the ANN
2. Summarize the techniques used for building the ANN
3. Describe ways to build mathematical models for optimization.

2
INTRODUCTION
WHAT IS ANN
WHY ANN
COMPONENTS OF AN ANN
TYPES OF NEURAL NETWORKS
ACTIVATION FUNCTION
UPDATE RULE
FORWARD PROP
BACKPROPAGATION
BACKPROPAGATION
LOSS FUNCTION
GRADIENT DESCENT
OVERFITTING

Definition: Overfitting happens when the model memorizes training data instead of generalizing to new data.
Symptoms: High training accuracy but low test accuracy.
Causes: Too many model parameters or insufficient training data.
Solution Techniques:
Use regularization (e.g., L1/L2 penalties, Dropout).
Increase training data or use data augmentation.
Visualization: Training loss decreases while validation loss increases after some epochs.
UNDERFITTING

Definition: Underfitting occurs when a model is too simple to capture the


underlying patterns in the data, leading to poor performance on both training and
test data.
Symptoms: Low accuracy on training data and test data.
Insufficient model complexity (too few layers or neurons).
Poor choice of hyperparameters (e.g., high learning rate).
Inadequate training time or data.
Solution Techniques:
Increase model complexity (e.g., add more neurons/layers).
Train for more epochs.
Use more features or improve feature engineering.
Visualization: Training and validation losses remain high throughout training.
XOR PROBLEM
XOR PROBLEM
Terminal Questions


Backpropagation Gradient Calculation:

Given a neural network with inputs x1=1.5x1​=1.5, x2=2.0x2​=2.0, weights w1=0.5w1​=0.5, w2=0.4w2​
=0.4, bias b=0.2b=0.2, and sigmoid activation function, compute the gradient of the loss function with
respect to w1w1​if the target output y=1y=1.

Loss Function Application:

For a binary classification task, calculate the cross-entropy loss for the following:

Target output y=0y=0, predicted probability y^=0.7y^​=0.7.

Gradient Descent Weight Update:

If a model weight w=0.3w=0.3, learning rate η=0.05η=0.05, and gradient of the loss function
∂L∂w=−0.4∂w∂L​=−0.4, compute the updated weight after one iteration of gradient descent.
20

You might also like