Advanced Machine Learning & Neural

Architectures
A Comprehensive Technical Reference Guide | Volume I: Deep Learning Principles

1. Foundations of Deep Neural Networks

Modern deep learning systems rely on the composition of parametric non-linear transformations. A standard
feedforward network maps an input vector x ∈ ℝd through multiple hidden layers to produce an output ŷ ∈ ℝk.
Each layer applies an affine transformation followed by a non-linear activation function σ(·).

Mathematical Formulation:
For layer l, the forward propagation is governed by:

h(l) = σ(W(l) h(l-1) + b(l))

where W(l) represents the weight matrix and b(l) is the bias vector.

1.1 Optimization and Backpropagation

Training neural networks involves minimizing an empirical risk functional L(θ) over a dataset of size N.
Stochastic Gradient Descent (SGD) and its adaptive variants (e.g., Adam, AdamW) update model parameters θ
iteratively using backpropagation, which leverages the chain rule of calculus to compute exact gradients.

Optimizer Update Formula Key Advantage

SGD with
vt = βvt-1 + gt Accelerates along flat directions, reduces oscillations.
Momentum

RMSprop st = βst-1 + (1-β)gt2 Adapts learning rate per parameter based on recent gradients.

θt = θt-1 - η m̂t / (√v̂t + Decouples weight decay from gradient updates for superior
AdamW
ε) generalization.

Regularization strategies such as Dropout, Batch Normalization, and Weight Decay play a vital role in
preventing overfitting and stabilizing gradient flow during deep network optimization.

Advanced Machine Learning & Neural Architectures Page 1 of 5

2. Convolutional & Spatial Architectures

Convolutional Neural Networks (CNNs) introduce inductive biases tailored for grid-structured data such as
images. By exploiting spatial locality and translation invariance through weight sharing, CNNs dramatically
reduce parameter counts compared to dense layers.

2.1 Core Operations in Vision Models

• Discrete 2D Convolution: Slide a kernel K ∈ ℝk×k across feature maps to extract local spatial features.

• Strided Convolutions & Pooling: Downsample spatial resolution to expand the receptive field of deeper
layers.

• Residual Connections: Introduced by ResNet, skip connections allow gradients to flow directly through
identity shortcuts: y = F(x, {Wi}) + x.

Receptive Field Analysis:

The effective receptive field Rl at layer l increases additively with kernel sizes:

Rl = Rl-1 + (kl - 1) × ∏i=1l-1 si

where kl is the kernel size and si is the stride of layer i.

2.2 Evolution of Vision Architectures

Architecture Primary Innovation Top-1 Accuracy (ImageNet)

AlexNet (2012) GPU acceleration, ReLU activations, Dropout 63.3%

VGG-16 (2014) Homogeneous 3x3 convolutions throughout 71.3%

ResNet-50 (2015) Residual learning and bottleneck blocks 76.0%

EfficientNet-B7 (2019) Compound scaling of depth, width, and resolution 84.3%

Advanced Machine Learning & Neural Architectures Page 2 of 5

3. Transformer Architectures & Self-Attention

The Transformer architecture revolutionized sequence modeling by replacing recursive mechanisms with self-
attention mechanism, enabling complete parallelization during training and modeling long-range dependencies
effectively.

3.1 Scaled Dot-Product Attention

Given matrices of Queries Q, Keys K, and Values V, the attention map calculates dynamic weights based on
pairwise similarity between sequence elements.

Attention Equation:

Attention(Q, K, V) = softmax( (Q KT) / √(dk) ) V

Scaling by 1/√(dk) prevents vanishing gradients in the softmax function for high dimensions.

3.2 Multi-Head Attention Mechanism

Rather than performing a single attention function, Multi-Head Attention projects queries, keys, and values h
times with distinct learned linear projections, allowing the model to jointly attend to information from different
representation subspaces.

Component Dimensionality Purpose

Query / Key / Value

dmodel × dk Project inputs into head-specific subspaces.
Projections

Inject sequence order information into non-recurrent

Positional Encodings N × dmodel
tokens.

dmodel → 4dmodel → Apply non-linear transformations per position

Feed-Forward Network (FFN)
dmodel independently.

Advanced Machine Learning & Neural Architectures Page 3 of 5

4. Large Language Models & Pre-training Paradigm

Modern Large Language Models (LLMs) are trained on massive text corpora using self-supervised objectives.
They demonstrate remarkable emergent capabilities when scaled up in parameter size and training data
volume.

4.1 Pre-training Objectives

1. Autoregressive Causal Language Modeling (Decoder-only): Predicts token x conditioned on x .

t <t
Examples: GPT-4, LLaMA, PaLM.

2. Masked Language Modeling (Encoder-only): Predicts masked tokens within a bidirectional context.
Examples: BERT, RoBERTa.

3. Sequence-to-Sequence (Encoder-Decoder): Maps input sequences to output sequences. Examples: T5,

BART.

4.2 Scaling Laws in AI

Research by Kaplan et al. and Chinchilla (Hoffmann et al.) established that model performance scales
predictably as a power-law with compute C, dataset size D, and parameter count N.

Chinchilla Optimal Scaling Rule:

For optimal compute efficiency, model size and dataset size should be scaled in equal proportions. For
every doubling of model parameters, the number of training tokens should also double.

Model Parameters Tokens Trained Context Window

LLaMA-1 65 Billion 1.4 Trillion 2,048 tokens

LLaMA-3 70 Billion 15.0 Trillion 8,192 tokens

Gemini 1.5 Pro MoE Architecture Multi-Trillion 2,000,000 tokens

Advanced Machine Learning & Neural Architectures Page 4 of 5

5. Alignment, Post-Training & Frontiers

Raw base models trained on web data often output unhelpful, biased, or harmful responses. Post-training
alignment aligns model outputs with human intent, safety, and operational preferences.

5.1 Reinforcement Learning from Human Feedback (RLHF)

RLHF refines models using human preference datasets through three main phases:

1. Supervised Fine-Tuning (SFT): Train the model on high-quality demonstrative instruction-response pairs.

2. Reward Model (RM) Training: Train a discriminator to score candidate responses based on human
rankings.

3. PPO Alignment: Optimize the SFT policy against the Reward Model while imposing a KL-divergence
penalty:

R(x, y) = RRM(x, y) - β DKL( πθ(y|x) || πSFT(y|x) )

5.2 Direct Preference Optimization (DPO)

DPO simplifies alignment by parameterizing the reward function directly through the language model policy,
eliminating the need to train a separate reward model or use complex reinforcement learning algorithms.

Summary of Key Takeaways:

• Deep architectures rely on gradient flow optimization and non-linear representations.

• Transformers remain the foundational backbone across NLP, vision, and multimodal AI.

• Alignment techniques like DPO and RLHF bridge the gap between raw capacity and practical usability.

Advanced Machine Learning & Neural Architectures Page 5 of 5