40719 Deep Learning

Hamid Beigy, Sharif University of Technology, Spring Semester 2020-21.

Course Description

The course helps to understand the fundamentals of Deep Learning. The course starts off gradually with multi-layer preceptrons and it progresses into the more complicated concepts such as attention and sequence-to-sequence models. This course also covers other models of deep learning such as convolutional neural networks, recurrent neural networks, deep generative models such as autoregressive, GAN, VAE, NFM, representation learning, and deep reinforcement learning methods. We use frameworks such as PyTorch and Tensorflow, which are very important for implementing deep Learning models.

Course Information

Required Texts

  1. [GYC] Ian Goodfellow and Yoshua Bengio and Aaron Courville, Deep Learning, MIT Press, 2016.

  2. [DY] Li Deng and Dong Yu, Foundations and Trends in Signal Processing, Vol. 7, No. 3–4, pp 197-387, 2014.

  3. [SB] Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction, Second edition. The MIT Press, 2018.

Grading Policy

  1. 15%: Mid-term exam 1 (1400/01/19).

  2. 15%: Mid-term exam 2 (1400/02/22).

  3. 20%: Final exam (1400/04/07).

  4. 30%: Homeworks.

  5. 15%: Quiz.

  6. 5%: Paper & Explore a theoretical or empirical question and present it. Deadline for choosing paper: 1400/02/25.

Lecture Schedule


Lecture Date Topics Related Readings and Links Homework & Assignments
1 1399-11-25Introduction: what is deep learning? Chapter 1 of GYC
Chapter 1 of DY
2
3
1399-11-27
1399-12-02
Machine learning overview Chapter 4 of GYC
4
5
1399-12-04
1399-12-09
Feedforward deep networks &
backpropagation
Chapter 6 of GYC
6
7
1398-12-11
1398-12-16
Optimization and regularization Chapters 7 & 8 of GYC
Papers given in the slides
8
9
10
1399-12-18
1399-12-23
1399-12-25
Convolutional networks Chapter 9 of GYC
Papers given in the slides
Homework 1 (Due date: 1400-01-06)

Quiz 1
1400-01-06 Homework 2 (Due date: 1400-01-27)
11
12
1400-01-14
1400-01-16
Recurrent neural networksChapter 10 of GYC
Papers given in the slides
1400-01-19 The first exam At 13-15
13
14
1400-01-21
1400-01-23
Recurrent neural networks Chapter 10 of GYC
Papers given in the slides

Quiz 2
15
16
17
18
1400-01-28
1400-01-30
1400-02-04
1400-02-06
Representation learning Chapter 14 of GYC &
Papers given in the slides


Homework 3 (Due date: 1400-02-18)
Quiz 3
19
20
1400-02-11
1400-02-13
Attention models Papers given in the slides
21 1400-02-18 Dual Learning
Sum-product networks
Papers given in the slides
1400-02-22 The second exam At 13:30-15:30
22
23
24
25
26
1400-02-20
1400-02-25
1400-02-27
1400-03-01
1400-03-03
Deep generative models Chapter 20 of GYC
Papers given in the slides


Quiz 4

1400-03-06 Homework 4 (Due date: 1400-03-18)
27 1400-03-08 Deep generative models
Deep reinforcement learning
Papers given in the slides
Chapters 1 to 6 and 13 of SB
28
29
1400-03-10
1400-03-15
Deep reinforcement learning Chapters 1 to 6 and 13 of SB
Papers given in the slides

Quiz 5
30 1400-03-17Grap neural networks Papers given in the slides
1400-03-19 Homework 5 (Due date: 1400-04-25)
1400-04-07 The final exam At 9-12

Problem sets

  1. Problem set 1 Due 1400-01-06

  2. Problem set 2 Due 1400-01-27

  3. Problem set 3 Due 1400-02-18

  4. Problem set 4 Due 1400-03-18

  5. Problem set 5 Due 1400-04-25