DL
Deep Learning: Overview and Key Concepts
Deep Learning is a subset of machine learning
that uses neural networks with many layers (hence
"deep") to model and solve complex problems. It is
inspired by the structure and functioning of the
human brain and excels in tasks such as image
recognition, natural language processing, and
speech recognition.
1. Core Components of Deep Learning
(a) Neural Networks (d) Loss Functions
A neural network consists of layers of Measure how well the model's predictions
interconnected nodes (neurons): match the true labels:
•Input Layer: Accepts the input data (features). •Mean Squared Error (MSE): For
•Hidden Layers: Process the input data through regression.
weights, biases, and activation functions. •Cross-Entropy Loss: For classification.
•Output Layer: Produces predictions or •Hinge Loss: For SVM-like tasks.
classifications. (e) Optimizers
(b) Weights and Biases Algorithms to update weights and biases to
•Weights: Determine the importance of each input minimize the loss:
feature. •Gradient Descent: Basic optimization
•Biases: Shift the activation function to improve algorithm.
flexibility. •Variants: SGD, Adam, RMSprop, Adagrad.
2. Architecture of Neural Networks
(d) Transformer Networks
(a) Feedforward Neural Networks (FNNs) •Used in NLP and sequence-to-sequence tasks.
•Information flows in one direction (input → hidden
•Key feature: Attention mechanism to focus on
→ output).
relevant parts of the input.
•Suitable for structured data.
•Example: BERT, GPT, T5.
(b) Convolutional Neural Networks (CNNs)
(e) Generative Adversarial Networks (GANs)
•Designed for image data.
•Two networks (generator and discriminator)
•Key components:
compete to create realistic outputs.
• Convolutional Layers: Detect spatial
•Applications: Image generation, style transfer.
patterns.
• Pooling Layers: Downsample feature
maps.
• Fully Connected Layers: Final decision-
making.
(c) Recurrent Neural Networks (RNNs)
•Designed for sequential data (e.g., time series,
text).
•Uses feedback loops to process sequences.
•Variants: LSTM, GRU (to address vanishing
gradient issues)