INF8953DE(Fall 2021):
Reinforcement Learning
Lecture 01
Sarath Chandar
Instructor and TAs
Priyesh Vijayan (Lead) Dongyan Lin
Sarath Chandar
Xutong Zhao Michel Ma
Course Details
Website: [Link]
Schedule: [Link]
Schedule will be updated regularly!
Hybrid-mode Lectures
● You can join the lectures either in-class or via zoom.
● All the tutorials are in zoom only.
Lecture Recordings
● The lectures are recorded for offline viewing.
● The lectures will be released to public on YouTube.
● By registering for the course, you agree for the recording and release of lecture videos.
What is this course about?
This is an introductory course in Reinforcement Learning.
We will not cover supervised learning in this course. Check INF8245E.
Prerequisites
● Basic knowledge of Probability Theory/statistics (MTH2303 or equivalent), calculus, and
linear algebra (MTH1070 or equivalent).
● Python for programming.
● Some AI/ML background is recommended but not required.
● First week’s reading covers some prerequisite materials.
● The first three optional tutorials cover linear algebra, probability, and python if you want
to brush up on these topics.
Course structure
● 1 lecture per week (3 hours on Wednesdays).
● 3 hours of lab for every 2 weeks (on Tuesdays or Fridays).
○ These lab slots are meant for you to work on your assignments and the final competition.
○ There will be several 1-hour tutorials conducted in the lab slots.
● Lecture notes will be released a few days before the lecture.
● Every week has some reading materials.
● I will be using a digital whiteboard for the rest of the course.
Grading Info
● 3 assignments (individual) - 45%
○ Includes both theory and programming.
● Course Project (team of 3*) - 30%
● End-term Exam - 25%
Late submission policy
If you submit your assignments and competition reports after the deadline, we will follow the
following penalty scheme:
● You will be penalized 5% if your submission is within 24 hours (1 day) from the
deadline.
● You will be penalized 10% if your submission is after 24 hours from the deadline and
within 48 hours (2 days) from the deadline.
● You will be penalized 20% if your submission is after 48 hours from the deadline and
within 72 hours (3 days) from the deadline.
● You cannot submit your assignments/reports after 72 hours from the deadline.
GradeScope
● We will use GradeScope for all the evaluations.
● Details on how to join GradeScope is available in moodle.
End-term Exam
● End-term exam will be a take-home exam with limited amount of time.
● You will be submitting your answers in GradeScope.
Code of Conduct
● Zero tolerance for plagiarism and cheating!
● Each assignment/competition comes with rules and you MUST follow them.
Discussions in Piazza
● We will use Piazza for course discussions.
● All your questions about the lectures, assignments should be posted in Piazza only. You
cannot directly email them to the instructor or the TA.
● Use the following link to signup for Piazza:
[Link]
Office Hours
● We will have 1 office hour (60 minutes) per day throughout the week (Mon-Fri).
● Please check the moodle for more details.
Course Feedback
● You can submit your anonymous feedback about the course at any point of time.
● Check moodle for a link to the Google form.
● Feel free to give both positive and negative feedback about every lecture!
Email policy
● This is a class of 100+ students.
● Please do not email the instructor or TAs for your doubts.
● Use Piazza and office hours.
● Any emails regarding an assignment should only be sent to the in-charge TA for the
assignment (through Piazza).
● You can email me directly only when it is extremely necessary. Your email subject should
contain “Course:INF8953DE” for faster reply. Use only [Link]@[Link].
Do not email to my other email ids. Always cc Priyesh (vijayanp@[Link]) when you
email me.
Tips for making this course useful to you
● Mode 1: Read the lecture notes for the lecture before the lecture.
● Mode 2: Read the lecture notes for the lecture after the lecture.
In both the modes, you MUST have mastered all the previous lectures.
Tips for making this course useful to you
● Join the lectures with a cup of coffee! It helps me :)
● Take notes with pen and paper.
● Do NOT look at your phone while sitting in the lecture. Resist the temptation!
● Go and rewatch the lectures if needed. Use the 2X option in YouTube for faster
revisions.
If you want to join my lab..
● If you want to work with me in the future, getting an A or A+ in this course will increase
the chances.
● If you want to TA for this course in the future, you should aim to get an A+.
ML/RL research for Undergraduate Students
● If you are an undergraduate student interested in pursuing ML/RL research, you can
apply to work with me in the following semester through INF8900 or INF8901.
● INF8900 is a 3-credit course for guided reading and literature search in a topic.
● INF8901 is a 3-credit course to do a research project in a topic.
● You must get an A or A+ in this course in order to be considered for doing
INF8900/INF8901 with me.