Scribd
Upload
37 views9 pages

Error Correction Learning in Neural Networks

The document outlines various learning mechanisms in neural networks, including Error-Correction Learning, Hebbian Learning, Competitive Learning, and Reinforcement Learning, each with distinct ideas and applications. It describes the weight update rules and common algorithms associated with each learning type, emphasizing their biological inspirations and use cases. Additionally, it highlights the overlap between these learning laws and methods like Gradient Descent and Policy Gradients.

Uploaded by

tayyab.15.9381
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
37 views9 pages

Error Correction Learning in Neural Networks

The document outlines various learning mechanisms in neural networks, including Error-Correction Learning, Hebbian Learning, Competitive Learning, and Reinforcement Learning, each with distinct ideas and applications. It describes the weight update rules and common algorithms associated with each learning type, emphasizing their biological inspirations and use cases. Additionally, it highlights the overlap between these learning laws and methods like Gradient Descent and Policy Gradients.

Uploaded by

tayyab.15.9381
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Error-Correction Learning

Idea: Learning occurs by minimizing the difference (error) between actual and desired outputs.

Mechanism: Update weights based on error (often using Gradient Descent).

Also called: Delta Rule, Widrow-Hoff Rule

Used in: Perceptron, Multi-Layer Perceptron (MLP), Backpropagation

Weight Update Rule:

—w = · × (target 2 output) × input

(· = learning rate)

' Common Algorithms:

Perceptron Learning
Backpropagation (for deep networks)
Logistic Regression
Linear Regression
2. Hebbian Learning
Idea: "Neurons that fire together, wire together."

Mechanism: Strengthen the connection between neurons if both are active simultaneously.

Biological Basis: Inspired by brain functioning.

Weight Update Rule:

—w = · × output × input

' Used in:

Associative memory models (e.g., Hopfield Network)


Unsupervised Hebbian-based networks
3. Competitive Learning
Idea: Neurons compete to become active. Only one (or a few) neurons "win" and update their weights.

Mechanism: Winner-takes-all strategy

Used in: Clustering, feature mapping

Only the "winning" unit's weights are adjusted.

Kohonen Self-Organizing Maps (SOM) Winner-Take-All Networks


4. Boltzmann Learning /
Stochastic Learning
Idea: Probabilistic learning with stochastic (random) processes.

Mechanism: Uses energy-based models and gradient approximations.

Used in: Boltzmann Machines, Restricted Boltzmann Machines (RBM)


5. Reinforcement Learning Law

Idea
Learn actions that maximize cumulative reward over time.

Mechanism
Uses trial-and-error interactions with an environment.

Learning Rule
Based on reward feedback, not direct target values.

Temporal Difference (TD) Learning and Q-Learning are common.


6. Correlation Learning
Idea: Strengthen weights when input and output are correlated.

Mechanism: Similar to Hebbian learning, but emphasizes statistical


correlation.
7. Outstar and Instar Learning
Outstar Learning Instar Learning
Used in supervised output Used in input pattern learning
associations

These are part of Grossberg9s Adaptive Resonance Theory (ART) and


biologically inspired models.
í Summary Table
Error-Correction Reduce output error (— = Target - Supervised learning (e.g., MLP)
Output)

Hebbian Learning Co-activation strengthens weights Unsupervised learning, memories

Competitive Learning Neurons compete; winner updates Clustering, feature extraction


weights

Reinforcement Learning Maximize reward through feedback RL agents, games, robotics

Correlation Learning Strengthen if input & output are Pattern association


correlated

Instar / Outstar Learn input/output association patterns Biological neural nets

Stochastic / Boltzmann Use randomness in energy models Deep generative models


t Note:
These learning laws are the foundation of how models update themselves, and they overlap with learning methods like:

Gradient Descent Policy Gradients Hebbian-based


(used in error correction) (used in RL) unsupervised updates

You might also like