University of Gujrat
Department of Information Technology
Title Deep Learning
Code CS-----
Credit Hours 3.0
Prerequisite
Course Intro Deep Learning is one of the most highly sought after skills in AI. In this course, you
will learn the foundations of Deep Learning, understand how to build neural
networks, and learn how to lead successful machine learning projects. You will learn
about Convolutional networks, RNNs, LSTM, Adam, Dropout, BatchNorm,
Xavier/He initialization, and more.
Instructor Dr. Zafar Mahmood Khattak
Understand the main fundamentals that drive Deep Learning
Aims and Objectives Be able to build, train and apply fully connected deep neural networks
Know how to implement efficient CNN or RNN.
Understand the key features in a neural network’s architecture
Course Description This course aims to present the core fundamentals behind the much talked about field
of Deep Learning. We will delve into selected topics of Deep Learning, from
discussing basics of neural networks, to understanding how CNN and RNN works
with common examples and publicly available datasets.
Machine learning and deep learning. Single layer neural networks, architecture and
training. Multi-layer neural networks architecture. Activating via forward
propagation, training via back propagation. Parallelizing neural network training,
training neural networks via TensorFlow. Other DNN frameworks and hardware
(Caffe, Theano,
Nvidia cuDNN). Machine learning algorithms with neural networks as a bridge to
deep learning. Design and development strategies for real-world applications
Convolutional Neural Network (CNN) architecture. Training deep CNNs. Image
classification with deep CNNs. Recurrent Neural Network (RNN) architecture and
training. RNN Extensions: Deep RNNs, Bidirectional RNNs, Long Short Term
Memory (LSTM) networks. RNN Applications in machine translation, language
modelling, joint language and translation modelling. Autoencoders. Deep Residual
Networks. Generative Adversarial Networks (GANs). Real-world use case
Recommended Text Books • [Link]
• Book: [Link]
• Deep Learning with Python, by Francois Challet, Manning Publications
• Introduction to Machine Learning by Ethem Alpadyn (latest edition)
• Machine Learning: A Probabilistic Perspective by Kevin P. Murph
Reference Books
Assessment Criteria Sessional 25% Mid 25% Final 50%
Quizzes 12%
Assignments 4%
Project/Presentation 9%
Sixteen-week lecture plan
Week Lecture Topic
Evolution of ML and DL, Fundamental of Deep learning, Deep Learning Applications
1 1,2 Importance of Artificial Neural Networks (ANNs), Weights and Activation functions
Perceptron, Single layer neural networks
Architecture and Training
2 3,4 Multi-layer neural networks ( Architecture, activating via forward propagation)
Training via back propagation, linear and non-linear function
Back propagation, Shallow ANNs, Single layer, Multi-layer, Perceptron Rule, Gradient descent,
3 5,6 Backpropagation, Loss Functions, Hyperparameter tuning [Hands-On]
MNIST example for Neural Networks
Back propagation, Shallow ANNs, Single layer, Multi-layer, Perceptron Rule, Gradient descent,
4 7,8 Backpropagation, Loss Functions, Hyperparameter tuning [Hands-On]
MNIST example for Neural Networks
5 9,10 Recurrent Neural Network (RNN) Architecture, RNN Training, RNN Extensions, RNN
Example, RNN Applications
6 11,12 Recurrent Neural Network (RNN) Architecture, RNN Training, RNN Extensions, RNN
Example, RNN Applications
7 13,14 Recurrent Neural Network (RNN) Architecture, RNN Training, RNN Extensions, RNN
Example, RNN Applications
8 15,16 Mid Term
Convolutional Neural Network (CNN), What are convolutional neural network and tensorflow,
Convolutional layer, Pooling layer, feature map, How to create the layers in Python,
Convolutional Neural Networks, ConvNets, Edge Detection, Padding, Convolution Operator,
9 17,18 CNN architecture, Parameter Sharing, Object Localization and Detection, Le-Net, AlexNet,
VGG, Residual Networks, Inception Net [Hands-On], Transfer learning
RCNN (Fast RCNN, Faster RCNN, Mask RCNN)
Convolutional Neural Network (CNN), What are convolutional neural network and tensorflow,
Convolutional layer, Pooling layer, feature map, How to create the layers in Python,
Convolutional Neural Networks, ConvNets, Edge Detection, Padding, Convolution Operator,
10 19,20 CNN architecture, Parameter Sharing, Object Localization and Detection, Le-Net, AlexNet,
VGG, Residual Networks, Inception Net [Hands-On], Transfer learning
RCNN (Fast RCNN, Faster RCNN, Mask RCNN)
Convolutional Neural Network (CNN), What are convolutional neural network and tensorflow,
Convolutional layer, Pooling layer, feature map, How to create the layers in Python,
Convolutional Neural Networks, ConvNets, Edge Detection, Padding, Convolution Operator,
11 21,22 CNN architecture, Parameter Sharing, Object Localization and Detection, Le-Net, AlexNet,
VGG, Residual Networks, Inception Net [Hands-On], Transfer learning
RCNN (Fast RCNN, Faster RCNN, Mask RCNN)
Convolutional Neural Network (CNN), What are convolutional neural network and tensorflow,
Convolutional layer, Pooling layer, feature map, How to create the layers in Python,
Convolutional Neural Networks, ConvNets, Edge Detection, Padding, Convolution Operator,
12 23,24 CNN architecture, Parameter Sharing, Object Localization and Detection, Le-Net, AlexNet,
VGG, Residual Networks, Inception Net [Hands-On], Transfer learning
RCNN (Fast RCNN, Faster RCNN, Mask RCNN)
Representation Learning and Generative Learning, Auto encoder, Generative adversarial
13 25,26 Networks, Simple example with MNIST dataset, Limitation of GAN and Deep Convolutional
GANs
Representation Learning and Generative Learning, Auto encoder, Generative adversarial
14 27,28 Networks, Simple example with MNIST dataset, Limitation of GAN and Deep Convolutional
GANs
Representation Learning and Generative Learning, Auto encoder, Generative adversarial
15 29,30 Networks, Simple example with MNIST dataset, Limitation of GAN and Deep Convolutional
GANs
Representation Learning and Generative Learning, Auto encoder, Generative adversarial
16 31,32 Networks, Simple example with MNIST dataset, Limitation of GAN and Deep Convolutional
GANs