SCHOOL OF ARTIFICIAL INTELLIGENCE
Deep Learning
Syllabus
[Link]
DEEP LEARNING
Overview
Build foundational skills in deep learning by designing and training neural networks to solve complex real-world
problems. You’ll begin with the essentials of neural networks, advancing to specialized architectures like Convolutional
and Recurrent Neural Networks, along with Transformers, Generative Adversarial Networks and Diffusion Models.
Through projects, create models for applications such as image classification, Q&A, and CAPTCHA image generation,
gaining hands-on experience with PyTorch and advanced training techniques. Ideal for those aiming to harness the
potential of deep learning, this experience prepares you to tackle AI challenges across various domains.
Nanodegree Program Intermediate 50 hours 4.7 (979 Reviews)
Prerequisites
Prior to enrolling, you should have the following knowledge:
Pandas Intermediate Python Vectors PyTorch Basic calculus NumPy Overfitting prevention
Deep learning fluency Basic probability Feedforward neural networks Gradient descent Derivatives
Neural network basics Python proficiency Matrix operations Matrix multiplication Linear algebra
Fully convolutional neural networks Differential calculus matplotlib Variational autoencoders
Jupyter notebooks Machine learning frameworks in Python
You will also need to be able to communicate fluently and professionally in written and spoken English.
Skills You'll Learn
Constructing and Training Neural Networks
Neural network basics | Backpropagation | Implementing Feedforward Neural Networks | Deep learning model
optimization | Model evaluation | Deep learning fluency | Implementing Neural Network Forward Propagation |
Deep learning models | Data pre-processing for ML | Hyperparameter tuning | Neural network activation | Neural
network mechanics | Model performance metrics | Deep Learning Model Diagnostics | Perceptron | Neural Network
Loss Function Concepts | Implementing Neural Network Activation Functions | Implementing Deep Learning Data
Pipelines | Feedforward neural networks | Training neural networks | Implementing Deep Learning Model Diagnostics
| Implementing Neural Network Loss Functions
Building Convolutional Neural Networks for Computer Vision
Implementing Data Augmentation Pipelines | Implementing Neural Style Transfer | Computer vision fluency | Data
augmentation | Semantic image segmentation | Implementing Object Detection | Convolutional kernels | Convolutional
Layer Configuration | Deep learning techniques | Convolutional neural networks | Implementing Convolutional Layers
[Link]
DEEP LEARNING
| Implementing Convolutional Autoencoders | Autoencoders | Implementing CNN Pooling Layers | CNN Architecture
Evolution | CNN Pooling Concepts | Transfer learning | Implementing Advanced CNN Training | Implementing Image
Segmentation | Feature visualization | Implementing Convolutional Neural Networks | Object detection | Implementing
CNN Transfer Learning
Creating Sequence Models and Transformers
Recurrent neural networks | Implementing Word Embeddings | Transformer neural networks | Word embeddings |
Implementing Attention Mechanisms | Generative AI Evaluation | Implementing Recurrent Neural Networks | Hugging
Face | Implementing Sequence-to-Sequence Models | Generative AI Evaluation Implementation | Sequential text analysis
| Sequence-to-Sequence Models | NLP models | Implementing Tokenization | Attention mechanisms | Tokenization |
Implementing Transformer Fine-Tuning
Building Generative Models
GAN Training Optimization | Generative adversarial networks | U-net | Generative AI Fluency | Implementing
GAN Training Optimization | Implementing Diffusion Model Sampling | Implementing Generator Networks | Deep
Convolutional GAN Architecture | Latent Space Concepts | Implementing Diffusion Models | Implementing Discriminator
Networks | Conditional Image Generation | Adversarial Training Concepts | GenAI Architectures | Implementing Deep
Convolutional GANs | Diffusion Models | Implementing GAN Training | Implementing Conditional GANs
[Link]
DEEP LEARNING
Courses
01. Constructing and Training Neural Networks 15 hours
This course covers foundational deep learning theory and practice. We begin with how to think about deep learning
and when it is the right tool to use. The course covers the fundamental algorithms of deep learning, deep learning
architecture and goals, and interweaves the theory with implementation in PyTorch.
L1 Course Overview Explore course objectives, prerequisites, and your instructor as you prepare
to build, train, and evaluate neural networks from scratch in deep learning.
L2 Introduction to Deep Learning Discover deep learning: its place in AI, how neural networks learn patterns
from data, why it's so powerful, and when to use it for complex, large-scale,
unstructured data tasks.
L3 Building Blocks of Neural Networks Explore how simple artificial neurons combine into layers to form neural
networks, enabling machines to learn complex patterns for tasks like image
recognition and decision-making.
L4 Coding Your First Single-Layer Build and understand a single-neuron network using PyTorch; explore how
Network weights and bias create decision boundaries; solve real and logic problems
with perceptrons.
L5 How Do Networks Learn Non-Linearity Discover how activation functions add non-linearity to neural networks,
enabling them to learn complex patterns beyond linear relationships for
real-world tasks.
L6 Selecting Activation Functions in Explore how activation functions empower neural networks to learn
Practice non-linear patterns, compare Sigmoid, Tanh, and ReLU, and understand their
limitations in solving complex problems.
L7 From Single-layer to Deep Neural Explore how neural networks use layers, depth, and width to learn complex
Networks patterns. Learn to design and balance network architectures for effective
deep learning solutions.
L8 Architecting Multi-layer Neural Learn to build, analyze, and compare multi-layer neural networks (MLPs) in
Networks PyTorch, exploring model architecture, hidden layers, and parameter counts
with real and classic datasets.
L9 How Do Networks Move Data Forward Discover how data moves through neural networks using computational
graphs and tensor shapes, enabling you to trace, debug, and refine model
predictions with confidence.
[Link]
DEEP LEARNING
L10 Following Tensors Through Forward Learn to control data flow through PyTorch neural networks, implement
Propagation custom forward passes, and debug architectural errors using shape tracking
for robust model building.
L11 How Do Networks Measure Error Learn how neural networks use loss functions to measure errors, guide
learning, and choose the right function for regression, binary, and multiclass
tasks.
L12 Investigating Loss Functions In Learn how and why to select and implement appropriate loss functions for
Different Tasks regression (MSE) and binary classification (BCE) tasks to ensure effective
neural network training.
L13 How Do Networks Learn During Explore how neural networks learn: gradient descent and backpropagation
Training optimize weights to reduce error, with optimizers like Adam accelerating and
stabilizing the training process.
L14 Building Your First Training Loop Learn to build a five-step PyTorch training loop, prepare data, train neural
networks, and compare optimizers (SGD vs. Adam) for effective model
learning.
L15 How Do You Prepare Data for Training Learn essential steps to prepare data for machine learning: split datasets,
preprocess for quality and optimization, and efficiently load data for reliable
model training.
L16 Building Efficient Data Loading Learn to build efficient data pipelines: clean, encode, scale, batch, and load
Pipelines tabular data for PyTorch models, optimizing preprocessing and DataLoader
settings for robust training.
L17 How Do You Diagnose Model Learn to diagnose machine learning model performance by identifying
Performance underfitting, overfitting, and training instability using the bias-variance
tradeoff and loss curves.
L18 Plotting and Interpreting Loss Curves Learn to plot and interpret loss curves to diagnose underfitting, overfitting,
good fit, and unstable training by comparing training and validation losses
during model training.
L19 How Do You Evaluate a Model Learn to evaluate machine learning models beyond loss and accuracy
by choosing metrics like precision, recall, MAE, and RMSE that align with
real-world needs and error costs.
L20 Measuring Model Performance Learn to assess model performance using metrics beyond accuracy, using
Beyond Accuracy precision, recall, F1, PR and ROC curves to evaluate and optimize models,
especially with imbalanced data.
L21 How Do You Improve Model Learn a systematic framework to improve deep learning models by
Performance diagnosing issues and applying techniques in Data, Model, Optimization, and
Inference to boost performance and stability.
[Link]
DEEP LEARNING
L22 Tuning Model Performance Across Learn to diagnose and address overfitting, underfitting, and instability by
Scenarios applying techniques like dropout, learning rate decay, and systematic model
improvement strategies.
L23 Moving Beyond Fully-Connected Discover why fully-connected networks struggle with structured data and
Networks how specialized architectures like CNNs, RNNs, and Transformers leverage
inductive bias for better AI solutions.
L24 Project: In this project, you will design a multi-layer perceptron (MLP) to predict
Diabetes Risk Prediction with PyTorch diabetes risk using CDC health data, developing a complete workflow from
raw data to a tuned and tested deep learning model.
02. Building Convolutional Neural Networks for Computer Vision 13 hours
This course introduces Convolutional Neural Networks, the most widely used type of neural networks specialized in
image processing. You will learn the main characteristics of CNNs that make them so useful for image processing,
their inner workings, and how to build them from scratch to complete image classification tasks. You will learn
what are the most successful CNN architectures, and what are their main characteristics. You will apply these
architectures to custom datasets using transfer learning. You will also learn about autoencoders, a very important
architecture at the basis of many modern CNNs, and how to use them for anomaly detection as well as image
denoising. Finally, you will learn how to use CNNs for object detection and semantic segmentation.
L1 Course Overview Explore the course goals, meet your instructor, and discover how computer
vision and CNNs enable computers to interpret images and solve real-world
visual tasks.
L2 Introduction to Computer Vision and Explore how Convolutional Neural Networks (CNNs) revolutionize computer
CNNs vision by preserving spatial structure and enabling translation-invariant
image recognition.
L3 Convolutions and Feature Extraction Discover how convolutional neural networks extract image features by
applying learnable filters, enabling efficient visual recognition through spatial
awareness and translation invariance.
L4 Implementing Convolutions with Learn to implement 2D convolutional layers in PyTorch, use custom filters
PyTorch to extract features from images, and visualize feature maps for foundational
computer vision skills.
L5 Kernels, Strides, and Padding Learn how padding and stride control feature map size and detail in CNNs,
preserve spatial information, and enable efficient, deeper models for robust
computer vision tasks.
L6 Pooling in PyTorch Learn how pooling layers in PyTorch downsample feature maps, boost
efficiency, and build robust CNNs using max and average pooling techniques
for image recognition tasks.
[Link]
DEEP LEARNING
L7 CNN Architectures Learn how CNN architectures stack convolution, activation, and pooling layers
to extract features and classify images, powering modern computer vision
applications.
L8 Building a CNN from Scratch Learn to build, train, and visualize a CNN for image classification, interpret
feature maps and filters, and refine models using data, architecture, and
regularization techniques.
L9 Data Augmentation Learn how data augmentation expands training data using transformations,
helping models generalize better and prevent overfitting by teaching
invariance to real-world variations.
L10 Data Augmentation Pipelines with Learn to build and apply data augmentation pipelines in PyTorch, creating
PyTorch robust vision models by chaining transforms for training and evaluation using
torchvision and DataLoader.
L11 Advanced CNN Training Master advanced CNN training by tackling imbalanced data, optimizing
learning rates, and using regularization techniques for robust, reliable, and
generalizable deep models.
L12 Advanced CNN Training in PyTorch Master advanced CNN training with PyTorch: tackle overfitting using data
augmentation, dropout, batch normalization, learning rate scheduling, and
early stopping for robust models.
L13 Famous Vision Architectures Explore key vision architectures like LeNet, AlexNet, VGG, ResNet, ConvNeXt,
and Vision Transformers, learning their innovations and how they shaped
modern computer vision.
L14 Transfer Learning and Fine-Tuning Learn to leverage pretrained models with transfer learning through feature
extraction and fine-tuning to quickly build accurate computer vision models
with limited data.
L15 Transfer Learning in PyTorch Learn transfer learning in PyTorch: fine-tune pretrained models, adapt
classifiers, handle real-world data challenges, and build custom image
classifiers efficiently.
L16 Explainable and Interpretable CNNs Learn how visualization and attribution methods like feature maximization
and Grad-CAM make CNNs explainable, interpretable, and more trustworthy
in critical applications.
L17 Application: Neural Style Transfer Learn to blend the content of one image with the style of another using neural
style transfer and a pre-trained VGG19 network, creating unique, artistic
images with deep learning.
L18 Encoder-Decoder Networks Explore encoder-decoder networks (autoencoders) that compress and
reconstruct data, enabling image denoising, anomaly detection, and
unsupervised feature learning.
[Link]
DEEP LEARNING
L19 Building and Using Autoencoders with Learn to build and train autoencoders with PyTorch for image reconstruction
PyTorch and denoising, using fully-connected and convolutional architectures on the
MNIST dataset.
L20 Object Detection Discover object detection by learning how models find and classify objects in
images using bounding boxes, multi-head networks, and metrics like IoU and
mean Average Precision (mAP).
L21 Object Detection with PyTorch Learn to use and fine-tune PyTorch pretrained models for object detection.
Build YOLO from scratch, apply Faster R-CNN, process outputs, and visualize
results on real-world images.
L22 Image Segmentation Learn how semantic segmentation assigns a class to every pixel using
encoder-decoder networks like U-Net, combining context and spatial detail
via skip connections for precise image mapping.
L23 Image Segmentation with PyTorch Learn semantic segmentation with PyTorch by building and training U-Net
models to classify pixels, using theory, demos, and hands-on exercises for
real-world image analysis tasks.
L24 Project: In this project, you will apply the skills you have acquired in the Convolutional
Landmark Classification & Tagging for Neural Network (CNN) course to build a landmark classifier.
Social Media
03. Creating Sequence Models and Transformers 11 hours
This course covers the fundamentals and applications of sequence modeling. The course begins with an overview
of sequence models and their significance, followed by hands-on lessons to tokenize text and develop embeddings
using PyTorch. Participants will explore recurrent neural networks (RNNs) and their variants, including LSTMs and
GRUs, progressing to Seq2Seq models and the implementation of attention mechanisms. The course culminates
in a comprehensive understanding of transformers, self-attention, and industry evaluation practices. By the end,
students will build a transformer-based Q&A system, solidifying their grasp of modern NLP frameworks.
L1 Course Overview Get introduced to Creating Sequence Models and Transformers, meet your
instructor, and discover the course objectives.
L2 Why Sequence Models Matter Discover why the order of data matters, real-world applications of sequence
models, key challenges, and foundational techniques for processing and
modeling sequential information.
L3 Breaking Text into Tokens Learn how text is broken into tokens, assigned numerical IDs, and prepared
for AI models. Explore tokenization strategies and their impact on model
performance and costs.
[Link]
DEEP LEARNING
L4 Creating a Tokenizer with PyTorch Learn how to build a tokenizer in PyTorch, transforming text into numerical
data for NLP models, and compare simple word tokenization with advanced
subword techniques used in modern AI.
L5 Finding Meaning with Word Learn how word embeddings transform words into vectors, letting machines
Embeddings grasp meaning, relationships, and analogies for smarter NLP applications like
search, recommendations, and translation.
L6 Pretrained Embeddings with PyTorch Learn to use pre-trained word embeddings in PyTorch, explore GloVe
vectors, visualize relationships, solve analogies, and understand static versus
contextual embeddings in NLP.
L7 Sequence Modeling with RNNs, LSTMs, Learn how RNNs, LSTMs, and GRUs model sequences, overcome memory
and GRUs limitations, and enable applications like text prediction, translation, and time
series forecasting.
L8 Predicting Sequences with RNN Discover how RNN, LSTM, and GRU networks in PyTorch excel at
Variants in PyTorch predicting sequences, comparing their memory, accuracy, and efficiency in
character-level text tasks.
L9 Translating Ideas with Seq2Seq Explore how Seq2Seq models use encoder-decoder architectures to
Models transform sequences, enabling tasks like translation, summarization, and
chatbot responses.
L10 Building Your First Seq2Seq Network Build a Seq2Seq model in PyTorch for tasks like translation and Q&A,
with PyTorch learning about encoders, decoders, teacher forcing, and the context vector
bottleneck.
L11 Focusing Model Attention on What Learn how attention mechanisms let AI models focus on relevant input parts,
Matters overcoming Seq2Seq bottlenecks and improving tasks like translation and
summarization.
L12 Enhancing Seq2Seq with Attention in Learn to enhance Seq2Seq models in PyTorch using attention mechanisms,
PyTorch robust QA evaluation with EM and F1, and data-centric strategies to address
complex language tasks.
L13 Transformers and the Power of Explore Transformer architecture, self-attention, multi-head attention, and
Self-Attention positional encoding to understand how modern AI models process language
efficiently and contextually.
L14 Exploring Transformers with Discover how Transformers process language by exploring tokenization,
HuggingFace contextual embeddings, and attention visualization using HuggingFace and
BERT for model interpretability.
L15 Evaluating Transformer Models: Learn to rigorously evaluate Transformer models using metrics like EM,
Concepts and Industry Practices F1, BLEU, and ROUGE, and qualitative error analysis for reliable, real-world
language tasks.
[Link]
DEEP LEARNING
L16 Implementing Evaluation Metrics with Learn to implement and interpret evaluation metrics like EM, F1, Precision@k,
Hugging Face and MRR for QA models using Hugging Face, and perform error analysis to
guide model improvement.
L17 Fine-Tuning Transformers for Q&A Learn to fine-tune pretrained transformers for extractive Q&A: prepare data,
train with Hugging Face, evaluate using SQuAD, EM, and F1, and apply
techniques in real-world scenarios.
L18 Comparing RNNs and Transformers Compare RNNs (sequential processing) and Transformers (parallel
self-attention) for sequence tasks in AI, focusing on strengths, limits, and
real-world applications.
L19 Project: You will build a semantic retriever for an internal AI assistant. You’ll use
Project: Build a Transformer-Based transformer embeddings to encode queries and documents and return top-k
Q&A System results for LLM answers.
04. Building Generative Models 11 hours
This course covers the construction and training of Generative Adversarial Networks (GANs), providing a
comprehensive understanding of generative models. Starting with foundational concepts of latent spaces and data
distributions, learners will progress to implementing generator and discriminator networks using PyTorch. The
curriculum emphasizes step-by-step training processes, improvements in GAN architecture, and the exploration of
Deep Convolutional GANs. Additionally, the course presents conditional image generation and introduces diffusion
models, highlighting comparisons with GANs. Practical applications culminate in a hands-on project focused on
creating synthetic handwriting for CAPTCHA systems, reinforcing learned concepts.
L1 Course Overview Get an introduction to building Generative Adversarial Networks, meet your
instructor, and preview key topics covered in the course.
L2 Introduction to Generative Models Explore the shift from data recognition to creation in AI through generative
models, their applications, differences from classifiers, and crucial ethical
considerations.
L3 Latent Spaces and Data Distributions Learn how generative AI uses latent spaces and data distributions to
transform random noise into structured, creative outputs by mapping simple
vectors to complex data like images.
L4 Building a Simple Generator Network Learn to build a simple generator network in PyTorch that transforms random
noise into images, which is a core foundation for Generative Adversarial
Networks (GANs) and creative AI models.
L5 Discriminators and Adversarial Discover how Generative Adversarial Networks use competition between
Training a Generator and Discriminator to create realistic synthetic data through
adversarial training.
[Link]
DEEP LEARNING
L6 Implementing a Discriminator with Learn how to build, train, and evaluate a discriminator in PyTorch as part of a
PyTorch GAN, mastering adversarial training, data preparation, and model evaluation
techniques.
L7 Generative Adversarial Networks Explore GANs: two neural networks, a generator and a discriminator, compete
to create realistic data, driving advances in creative AI like image, art, and text
generation.
L8 Training GANs Step by Step with Learn step-by-step GAN training with PyTorch by building, training, and
PyTorch analyzing generator and discriminator models to create realistic images and
understand adversarial processes.
L9 Improving GAN Training Learn to overcome GAN training challenges like mode collapse and
instability using techniques such as label smoothing, batch normalization, and
LeakyReLU for stable, diverse outputs.
L10 GAN Training Improvements with Learn how label smoothing and feature matching stabilize GAN training in
PyTorch PyTorch, alleviating common issues like instability and mode collapse for
more reliable, diverse outputs.
L11 Deep Convolutional GANs Learn to build Deep Convolutional GANs (DCGANs) using convolutional
layers, transposed/strided convolutions, and stable training to generate
high-quality, realistic images.
L12 Building and Training a DCGAN with Build and train a DCGAN in PyTorch to create realistic images from
PyTorch and TorchVision random noise, mastering adversarial training, architecture, and techniques
for creative image generation.
L13 Conditional Image Generation Learn how Conditional GANs generate specific images by combining labels
with noise, enabling control for tasks like data augmentation, text-to-image,
and image-to-image translation.
L14 Implementing a Conditional GAN Learn to build, train, and use a conditional GAN to generate images of specific
classes, gaining control over generative models by conditioning on class
labels.
L15 Introduction to Diffusion Models Learn how diffusion models generate images by adding and removing noise
in steps, using neural networks, and why they're stable and effective for
generative AI.
L16 U-Net Architecture for Diffusion Discover how the U-Net architecture enables diffusion models to create
images with both structure and fine detail, using encoder-decoder paths, skip
connections, and time embedding conditioning.
L17 Implementing a Simple Diffusion Learn to build, train, and implement a basic diffusion model from scratch,
Model mastering forward noising, U-Net architecture, and image generation
through denoising steps.
[Link]
DEEP LEARNING
L18 Sampling and Image Generation with Learn how diffusion models generate images by reverse diffusion, denoising
Diffusion random noise step-by-step, and explore the trade-off between generation
speed and image quality.
L19 Comparing GANs and Diffusion Explore the key differences between GANs and diffusion models, speed
Models versus image quality, and learn to choose the right AI tool for any creative
image generation task.
L20 Project: Build and compare a Conditional GAN and a Conditional Diffusion
Project: Synthetic Handwriting Model to generate diverse synthetic handwriting data, strengthening a
Generator for CAPTCHA Systems next-generation CAPTCHA system against sophisticated bot attacks.
[Link]
DEEP LEARNING
Meet Your Instructors
Samantha Guerriero Antje Muntzinger
AI Consultant Professor of Computer Vision
Sam is a freelance AI consultant, passionate about building tech that’s Antje Muntzinger is a professor of computer vision at Stuttgart
useful, responsible, and accessible. After earning her master’s degree University of Applied Sciences, where she teaches AI and applied
in AI & Robotics cum laude, she quickly grew from Machine Learning mathematics. Previously, she was an algorithm developer and tech
Engineer to Acting Head of AI Automation at a data & AI consultancy, lead for sensor fusion in the field of automated driving at Mercedes.
driven by her sense of ownership, collaboration, and innovation. She holds a PhD in engineering and a diploma in mathematics.
These days, she partners with startups and contributes to thought
leadership initiatives that bridge theory and practice for real-world AI
applications.
Sohbet Dovranov Temi Afeye
Senior Data Scientist at Microsoft Technical Lead/Senior AI Scientist
Sohbet is a Senior Data Scientist at Microsoft focused on building Temi Afeye has 10+ years of experience in data science and AI across
solutions that integrate AI to solve complex challenges. With CS and finance, consulting, and the public sector. He is currently a Technical
Data Science degrees from Columbia University and UT Austin, he Lead in AI and ML at Lloyds Banking Group, delivering impact in
previously worked at Bank of America, Harvard University, the Federal generative AI, pricing, and causal inference. Previously at Deloitte, and
Reserve Bank of Boston, and Hanover Insurance. He is passionate M-KOPA, he focuses on responsible, production-grade ML.
about building and teaching AI to create real-world impact.
[Link]
DEEP LEARNING
Why Udacity
Demonstrate proficiency with practical projects
Projects are based on real-world scenarios and challenges, allowing you to apply the skills you
learn to practical situations, while giving you real hands-on experience
Gain proven experience Retain knowledge longer Apply new skills immediately
24/7 access to real human support
Reviewers provide timely and constructive feedback on your project submissions, highlighting
areas of improvement and offering practical tips to enhance your work
Get help from subject matter experts Gain valuable insights and improve your skills
Learn industry best practices
[Link]