Scribd
Upload
52 views4 pages

Deep Learning Comprehensive Notes

Uploaded by

prachiksk07
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd
52 views4 pages

Deep Learning Comprehensive Notes

Uploaded by

prachiksk07
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd

Here’s a **complete and in-depth note on Deep Learning (DL)** — written in clear,

structured points for easy understanding and exam or interview use 👇

---

# 🧠 DEEP LEARNING – COMPLETE NOTES

---

1## 1 What is Deep Learning?


1️⃣

**Definition:**
Deep Learning (DL) is a **subset of Machine Learning (ML)** that uses **Artificial
Neural Networks (ANNs)** with **multiple hidden layers** to automatically learn
complex patterns from large amounts of data.

**Key idea:**
It tries to **mimic how the human brain works** — by processing data through layers
of interconnected “neurons.”

---

2️⃣
## Why “Deep”?

The term **“deep”** refers to the **number of layers** in a neural network.

* Traditional ML: Few layers (shallow learning).


* Deep Learning: Many hidden layers that allow automatic feature extraction and
abstraction.

Example:

```
Input → Hidden Layer 1 → Hidden Layer 2 → Hidden Layer 3 → Output
```

---

3️⃣
## Difference between Machine Learning and Deep Learning

| Feature | Machine Learning | Deep Learning |


| ------------------ | --------------------- | ------------------------------ |
| Data requirement | Works with small data | Requires huge data |
| Feature extraction | Manual (by experts) | Automatic (learned by network) |
| Execution time | Faster to train | Slower (high computation) |
| Hardware | Runs on CPU | Needs GPU/TPU |
| Example | Decision Tree, SVM | CNN, RNN, Transformer |

---

4️⃣
## Biological Inspiration

Deep Learning is inspired by the **human brain’s neurons**:

* Each **neuron** receives inputs, applies weights, adds bias, and passes through
an **activation function**.
* The result is transmitted to the next layer, simulating how the brain processes
signals.
---

5️⃣
## Structure of a Neural Network

### Basic Elements:

1. **Input Layer** – Takes the features (e.g., pixel values of an image).


2. **Hidden Layers** – Perform transformations and extract patterns.
3. **Output Layer** – Produces the final prediction/class.

### Example:

For image recognition of a cat:

```
Input: Image pixels
Hidden layers: Feature extraction (edges, shapes, textures)
Output: “Cat”
```

---

6️⃣
## Key Concepts

### 1. **Neuron / Perceptron**

* Mathematical model of a biological neuron.


* Formula:
[
y = f(w_1x_1 + w_2x_2 + ... + w_nx_n + b)
]
where
(w) = weights, (x) = inputs, (b) = bias, (f) = activation function.

---

### 2. **Activation Function**

Adds **non-linearity** to the model so it can learn complex patterns.

| Function | Formula | Characteristics


|
| ---------- | ------------------------- |
------------------------------------------ |
| Sigmoid | 1 / (1 + e^-x) | Output between 0–1, good for
probabilities |
| Tanh | (e^x - e^-x)/(e^x + e^-x) | Output between -1 to 1
|
| ReLU | max(0, x) | Fast and popular for hidden layers
|
| Leaky ReLU | x if x>0 else 0.01x | Fixes ReLU “dead neuron” issue
|
| Softmax | e^xᵢ / Σ e^xⱼ | Used in output layer for classification
|

---

### 3. **Loss Function**


Measures **how far predictions are from actual output**.

| Task | Example Loss Function |


| -------------- | ------------------------ |
| Regression | Mean Squared Error (MSE) |
| Classification | Cross Entropy / Log Loss |

---

### 4. **Optimization**

Used to minimize loss function.

* **Gradient Descent**: Main technique for optimization.


Updates weights in opposite direction of the gradient.

[
w_{new} = w_{old} - \eta \times \frac{dL}{dw}
]

where ( \eta ) = learning rate.

* Variants:

* Stochastic Gradient Descent (SGD)


* Mini-batch Gradient Descent
* Adam Optimizer (adaptive)

---

### 5. **Backpropagation**

* Algorithm to **update weights** based on error propagation.


* Steps:

1. Forward pass → Compute output and loss.


2. Backward pass → Compute gradients (errors).
3. Update weights → Using optimizer.

---

7️⃣
## Types of Deep Learning Networks

### 1. **Feedforward Neural Network (FNN)**

* Simplest type, where data moves in one direction: Input → Output.


* No feedback loops.

### 2. **Convolutional Neural Network (CNN)**

* Used for **image/video** data.


* Automatically extracts features like edges, textures, shapes.
* Components:

* Convolution Layer
* Pooling Layer
* Fully Connected Layer
* Examples: VGGNet, ResNet, AlexNet, LeNet.
### 3. **Recurrent Neural Network (RNN)**

* Used for **sequential data** (text, time series).


* Maintains memory of previous inputs using feedback loops.
* Types:

* Vanilla RNN
* LSTM (Long Short-Term Memory)
* GRU (Gated Recurrent Unit)
* Example: Sentiment analysis, speech recognition.

### 4. **Autoencoders**

* Used for **data compression or noise reduction**.


* Structure:

* Encoder: Compress data


* Decoder: Reconstruct original
* Example: Denoising images.

### 5. **Generative Adversarial Networks (GANs)**

* Two networks:

* **Generator** – creates fake data.


* **Discriminator** – detects if data is real or fake.
* Example: Deepfake videos, art generation.

### 6. **Transformers**

* Replaces RNNs for NLP.


* Uses **attention mechanism** to focus on relevant parts of input.
* Examples: BERT, GPT (ChatGPT uses Transformer architecture).

---

8️⃣
## Training Process in Deep Learning

1. **Collect Data**
2. **Preprocess Data** (normalization, resizing, tokenization)
3. **Define Model Architecture**
4. **Choose Loss Function & Optimizer**
5. **Train Model** (Forward + Backward propagation)
6. **Validate & Tune Hyperparameters**
7. **Test & Deploy**

---

9️⃣
## Hyperparameters in Deep Learning

| Hyperparameter | Meaning |
| -------------- | -------------------------------- |
| Learning Rate | Controls how fast weights update |
| Batch Size | Number of samples per update |
| Epochs | Number of passes through dataset |
| Number | |

Common questions

Powered by AI

CNNs are optimized for image and video data through their specialized layers such as convolutional layers for automatic feature extraction, pooling layers for dimensionality reduction, and fully connected layers for final classification. These specialized architectures allow CNNs to efficiently capture spatial hierarchies in visual data .

Feedforward neural networks (FNNs) process data in a single direction and are typically used for static data. In contrast, recurrent neural networks (RNNs) are designed for sequential data, like text or time series, as they utilize feedback loops to maintain memory of previous inputs, which is critical for tasks like speech recognition and sentiment analysis .

The attention mechanism in transformer models is crucial because it allows the model to focus on relevant parts of the input data when generating predictions. This mechanism enables transformers to handle long-range dependencies in data, which enhances their performance in natural language processing tasks such as translation and text summarization, by effectively modeling contextual relations between words .

The choice of neural network architecture depends on the characteristics of the data and the nature of the task. CNNs are preferred for tasks involving spatial data like images, due to their structural advantages in feature extraction. In contrast, RNNs are suited for sequential data like texts or time series, as they maintain context across data points through their feedback loops, which is essential for understanding temporal dependencies .

Deep learning involves the use of multiple hidden layers within artificial neural networks, which allows it to perform automatic feature extraction and abstraction from large datasets. In contrast, traditional machine learning typically uses fewer layers, necessitating manual feature extraction by experts .

Gradient descent facilitates training by iteratively adjusting the model's parameters to minimize the loss function. It calculates the gradient of the loss function with respect to the model parameters and updates the parameters in the direction that reduces the loss, thereby improving model predictions over time .

Deep learning models, due to their large number of layers and deep architecture, require significant computational resources to process vast amounts of data and perform complex operations involved in learning automatic features. As a result, the training process is slower and requires powerful hardware such as GPUs or TPUs to accelerate computations, unlike traditional machine learning that can run efficiently on simpler hardware like CPUs .

Activation functions introduce non-linearity into neural networks, enabling them to learn and model complex patterns beyond linear transformations. Functions like ReLU are popular for hidden layers due to their computational efficiency and ability to handle large datasets, while Softmax is used for classification tasks .

GANs consist of a generator that creates fake data and a discriminator that distinguishes real from fake data. Adversarial training involves these two networks competing against each other: the generator aims to improve its data quality so the discriminator fails more often, while the discriminator continuously enhances its ability to differentiate. This process results in the generator producing increasingly realistic data .

Backpropagation is an algorithm that iteratively adjusts the weights of neurons based on error gradients to minimize the loss function. During backpropagation, the loss is propagated backwards through the network to update weights using an optimizer, enabling the model to learn the parameters that contribute to accurate predictions .

You might also like