40719 Deep Learning

Course Description

The course helps to understand the fundamentals of Deep Learning. The course starts off gradually with multi-layer preceptrons and it progresses into the more complicated concepts such as attention and sequence-to-sequence models. This course also covers other models of deep learning such as convolutional neural networks, recurrent neural networks, deep generative models such as autoregressive, GAN, VAE, NFM, representation learning, and deep reinforcement learning methods. We use frameworks such as PyTorch and Tensorflow, which are very important for implementing deep Learning models.

Course Information

Required Texts

  1. [GYC] Ian Goodfellow and Yoshua Bengio and Aaron Courville, Deep Learning, MIT Press, 2016.

  2. [CH] Charniak, Eugene, Introduction to Deep Learning, The MIT Press, 2019.

  3. [SB] Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction, Second edition. The MIT Press, 2018.

Grading Policy

  1. 25%: Mid-term exam (1402/08/20).

  2. 25%: Final exam (1402/10/25).

  3. 30%: Homeworks.

  4. 15%: Quiz.

  5. 7.5%: Paper & Explore a theoretical or empirical question and present it. Deadline for choosing paper: 1402/08/20.

Lecture Schedule


Lecture Lecture Date Topics Related Readings and Links Homeworks & Assignments Quizes
1 1402-07-03Introduction: what is deep learning? Chapter 1 of GYC
Chapter 1 of CH
2
3
1402-07-08
1402-07-10
Machine learning overview Chapter 5 of GYC
4
5
1402-07-15
1402-07-17
Feedforward deep networks &
backpropagation
Chapter 6 of GYC Quiz 1
6
7
1402-07-22
1402-07-24
Optimization and regularization Chapters 7 & 8 of GYC
Papers given in the slides
8
9
1402-07-29
1402-08-01
Convolutional networks Chapter 9 of GYC
Papers given in the slides
HW1 release
Quiz 2
10
11
12
1402-08-06
1402-08-08
1402-08-13
Recurrent neural networksChapter 10 of GYC
Papers given in the slides


HW1 deadline & HW2 release

Quiz 3
13 1402-08-15Representation learning Chapter 14 of GYC &
Papers given in the slides
14 1402-08-20 Mid-term exam Presentation topic selection deadline
15
16
17
1402-08-22
1402-08-27
1402-08-29
Representation learning Chapter 14 of GYC &
Papers given in the slides


HW 2 deadline & HW 3 release
18
19
1402-09-04
1402-09-06
Attention models Papers given in the slides
20
21
22
23
24
1402-09-11
1402-09-13
1402-09-18
1402-09-20
1402-09-25
Deep generative models Chapter 20 of GYC
Papers given in the slides
HW 3 deadline & HW 4 release



HW 4 deadline & HW 5 release


Quiz 4

25
26
1402-09-27
1402-10-02
Deep reinforcement learning Chapters 1 to 6 and 13 of SB
Papers given in the slides

Quiz 5
27
28
29
1402-10-04
1402-10-09
1402-10-11
Graph Neural Networks Papers given in the slides
HW 6 release & HW 5 theory deadline


Quiz 6
1402-10-25 Final exam At 15:00 HW 6 theory deadline
1402-11-06 HW 5 coding deadline
HW 6 coding deadline