Course Description
The course helps to understand the fundamentals of Deep Learning. The course starts off gradually with multi-layer preceptrons and it progresses into the more complicated concepts such as attention and sequence-to-sequence models. This course also covers other models of deep learning such as convolutional neural networks, recurrent neural networks, deep generative models such as autoregressive, GAN, VAE, NFM, representation learning, and deep reinforcement learning methods. We use frameworks such as PyTorch and Tensorflow, which are very important for implementing deep Learning models.
Course Information
Instructor: Hamid Beigy
Teaching Assistants: Amir Moeini,
Mohammad Moein Shirzady,
Parsa Haghighi,
Mehdi Dousti,
Vida Ramezanian,
Masoud Ghafouri,
Ali Javani,
Shima Rezaei,
Zeinab Taghavi,
Zahra Rahimi,
Mehran Sarmadi,
Reihaneh Zohrabi,
Hamidreza Amirzadeh,
Hamed Hematian,
Amir Hossein Hadian,
Ehsan Shobeiri,
Hamed Saadati,
Mojtaba Nafez,
Zahra Khoramnejad,
Elham Abolhasani,
Ali Abdollahi,
Mohammad Ebrahimzadeh,
Alireza Heidari,
Abolfazl Malekahmadi.
Time and Location: Saturday/Monday: 10:30 am - 12:00 am, CE 201, virtual class
Contact: Please contact the teaching staffs on Piazza site.
Required Texts
[GYC] Ian Goodfellow and Yoshua Bengio and Aaron Courville, Deep Learning, MIT Press, 2016.
[CH] Charniak, Eugene, Introduction to Deep Learning, The MIT Press, 2019.
[SB] Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction, Second edition. The MIT Press, 2018.
Grading Policy
25%: Mid-term exam (1402/08/20).
25%: Final exam (1402/10/25).
30%: Homeworks.
15%: Quiz.
7.5%: Paper & Explore a theoretical or empirical question and present it. Deadline for choosing paper: 1402/08/20.
Lecture Schedule
Lecture | Lecture Date | Topics | Related Readings and Links | Homeworks & Assignments | Quizes |
1 | 1402-07-03 | Introduction: what is deep learning? | Chapter 1 of GYC Chapter 1 of CH | | |
2 3 | 1402-07-08 1402-07-10 | Machine learning overview | Chapter 5 of GYC | | |
4 5 | 1402-07-15 1402-07-17 | Feedforward deep networks & backpropagation | Chapter 6 of GYC | | Quiz 1 |
6 7 | 1402-07-22 1402-07-24 | Optimization and regularization | Chapters 7 & 8 of GYC Papers given in the slides | | |
8 9 | 1402-07-29 1402-08-01 | Convolutional networks | Chapter 9 of GYC Papers given in the slides | HW1 release | Quiz 2 – |
10 11 12 | 1402-08-06 1402-08-08 1402-08-13 | Recurrent neural networks | Chapter 10 of GYC Papers given in the slides | HW1 deadline & HW2 release | – Quiz 3 – |
13 | 1402-08-15 | Representation learning | Chapter 14 of GYC & Papers given in the slides | | |
14 | 1402-08-20 | Mid-term exam | | Presentation topic selection deadline | |
15 16 17 | 1402-08-22 1402-08-27 1402-08-29 | Representation learning | Chapter 14 of GYC & Papers given in the slides | – – HW 2 deadline & HW 3 release | |
18 19 | 1402-09-04 1402-09-06 | Attention models | Papers given in the slides | | |
20 21 22 23 24 | 1402-09-11 1402-09-13 1402-09-18 1402-09-20 1402-09-25 | Deep generative models | Chapter 20 of GYC Papers given in the slides | HW 3 deadline & HW 4 release – – – HW 4 deadline & HW 5 release | – – Quiz 4 – – |
25 26 | 1402-09-27 1402-10-02 | Deep reinforcement learning | Chapters 1 to 6 and 13 of SB Papers given in the slides | | – Quiz 5 |
27 28 29 | 1402-10-04 1402-10-09 1402-10-11 | Graph Neural Networks | Papers given in the slides | – HW 6 release & HW 5 theory deadline – | – – Quiz 6 |
| 1402-10-25 | Final exam | At 15:00 | HW 6 theory deadline | |
| 1402-11-06 | | | HW 5 coding deadline HW 6 coding deadline |
|
|