Deep Learning

View PDF

Instructor(s):

Bálint Gyires-Tóth

Short description of the course:

Artificial Intelligence (AI) has become a focused research area in the past decade. Among its subfields, deep
learning is one of the most important due to the state-of-the-art results achieved in several application
scenarios, e.g. image recognition, speech recognition and synthesis, natural language processing and
reinforcement learning. Deep learning models are capable of representation learning and modeling jointly,
thus, other machine learning methods (which require feature engineering) can be outperformed in case a
large amount of data is available.

This course teaches not only the fundamentals of deep learning but also helps to gain practical knowledge
through hands-on classes, assignments and project work. The course covers the most important building
blocks of deep neural networks, the optimization methods that are used for training, advanced architectures
for computer vision, speech technologies, natural language processing and self-supervised learning, the deep
learning hardware and software stack, and we discuss the possible applications throughout the complete
course.

Aim of the course:

The objective of the course is to teach a fundamental knowledge of the most important methods in deep
learning and to develop a practical knowledge, which helps the students to create complex deep learning
solutions (e.g. as a researcher, a programmer, or even a startup founder).

Prerequisites:
This is a beginner level deep learning class, basic math and computer science knowledge are required:

Basic programming skills (e.g. able to write a small game with a simple interface, like invaders)
Basic matrix algebra knowledge (addition, subtraction, multiplication, transpose, L1, L2
norm, inverse, linear independence)
Basic probability theory knowledge (mean and variance calculation, probability distributions)

Grading:
40% Assessments: there are deep learning related assignments. A successful proficiency assignment is
required for this class. The assignments have to be done individually.
40% Project work: students in groups of two have to develop a complete deep learning solution with source
code and documentation.
20% Presentation: the project work has to be presented at the end of the semester.

Syllabus:

Assignments and
Week Lecture Practice
deadlines
 Fundamentals of machine and deep
learning. Supervised, unsupervised and Deep learning software stack.
1
reinforcement learning. A complete Python basics.
machine learning pipeline.

2 Fundamentals of Deep Learning part 1/2

3 Fundamentals of Deep Learning part 2/2 Assessment 1.

High-level deep learning

Training deep neural networks
frameworks with GPU Deadline for project
4 backpropagation algorithm. Advanced
acceleration. Regression and work milestone 1.
optimization algorithms.
classification toy examples.

Advanced activation functions. Solving a regression problem

5 Regression and classifiaction. Cost with a fully-connected neural
functions. network.

Regularization methods. Pattern

recognition with convolutional neural Image classification with simple
6 Assessment 2.
networks (CNN). 2D CNN for computer 2D CNNs.
vision.

Advanced deep learning models for

Inference with pretrained CNN
computer vision. Training neural networks
7 for image classification and
with few data. Data augmentation.
transfer learning.
Transfer learning.

1D CNN for pattern recognition in Constructing 1D CNN for

8 sequential data. Recurrent neural networks natural language processing
indroduction. (NLP) task.

Recurrent neural networks (RNN), Deadline for project

9 Applying RNN for NLP task.
backpropagation through time. work milestone 2.

Advanced architectures. Residual,

Constructing and displaying
10 highway, dense networks. Skip Assessment 3.
advanced deep neural networks.
connections.

11 Building Conversational AI Applications part 1/2

12 Building Conversational AI Applications part 2/2 Assessment 4.

Example application(s) of
Deadline for project
autoencoders (e.g. clustering,
13 Auto encoders work final
anomaly detection, image
submission.
generation).

14 Presentations of the project works, discussing next steps.

The theoretical parts are introduced in the lectures, while the corresponding hands-on practices take place
afterwards, before the next lecture.

Example of assessment:
Create a deep learning model that predicts the temperature in Budapest
First, the students have to find a proper data source. There are many public weather homepages, that can be
used. Next, the data is downloaded, cleansed and preprocessed. When the data preprocessing is ready, a deep
learning model is built, trained and evaluated. The source codes with detailed comments and the output of
the scripts are submitted until the deadline.

Example of project work:

Design and implement a deep learning solution for webcam based real-time sex and age prediction
First, the data collection is performed: public, easy-to-access datasets with sex and age labels are used for
training. The choosen dataset and the source code of preprocessing are submitted until the milestone 1
deadline. Next, to be able to detect faces on images and to make sex and age predictions a pretrained deep
neural network is optimized. For milestone 2 the initial approach of the solution is submitted with source
code. Last, the video feed of a webcam is streamed to the deep neural network to make real-time sex and age
prediction. Furthermore, enhancements are made to increase the speed and accuracy of the deep learning
models, and evaluation is carried out. At the end of the semester, the source codes and a 2-4 pages long
documentation are submitted, furthermore, an 8 minutes long presentation is held in the exams period.

The possible topics for project work is announced in the first week.

Textbooks:
Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016., online:
[Link]

Franchois Chollet, Deep Learning with Python, Manning Publications, 384 pages, 2017,
[Link]

Instructors' bio:

Bálint Gyires-Tóth conducts research on fundamental and applied machine learning since 2007. With his
leadership, the first Hungarian hidden Markov-model based Text-To-Speech (TTS) system was introduced
in 2008. He obtained his PhD degree from the Budapest University of Technology and Economics with
summa cum laude in January 2014. Since then, his primary research field is deep learning. His main research
interests are sequential data modeling with deep learning and deep reinforcement learning. He also
participates in applied deep learning projects, like time series classification and forecast, image and audio
classification and natural language processing. He was involved in various successful research and industrial
projects. In 2017 he was certified as NVidia Deep Learning Institute (DLI) Instructor and University
Ambassador.