Schedule

As mentioned in the Logistics section, the course will be taught virtually on Zoom for the entire duration of the quarter. Unless otherwise specified:

Updated lecture slides will be posted here shortly before each lecture. For ease of reading, we've color-coded the lectures in white, discussion sections in blue, project-related deadlines in yellow and the midterm in red.

DateDescriptionCourse MaterialsEventsDeadlines
03/30 Lecture 1: Introduction
Computer vision overview
Historical context
Course logistics
[slides 1] [slides 2]
04/01 Lecture 2: Image Classification
The data-driven approach
K-nearest neighbor
Linear classification I
[slides] 		Image Classification Problem
Linear Classification
04/02 Python / Numpy Review Session
[Colab] [Tutorial] 		11:30 - 12:30 PM Assignment 1 out
[handout] [colab]
04/06 Lecture 3: Loss Functions and Optimization
Linear classification II
Higher-level representations, image features
Optimization, stochastic gradient descent
[slides] 		Linear Classification
Optimization
04/08 Lecture 4: Neural Networks and Backpropagation
Backpropagation
Multi-layer Perceptron
The neural viewpoint
[slides] 		Backprop
Linear backprop example
Suggested Readings:
  1. Why Momentum Really Works
  2. Derivatives notes
  3. Efficient backprop
  4. More backprop references: [1], [2], [3]
04/09 Backprop Review Session
[slides] 		11:30 - 12:30 PM
04/13 Lecture 5: Convolutional Neural Networks
History
Convolution and pooling
ConvNets outside vision
[slides] 		Convolutional Networks
04/15 Lecture 6: Deep Learning Hardware and Software
CPUs, GPUs, TPUs
PyTorch, TensorFlow
Dynamic vs Static computation graphs
[slides]
04/16 Project Overview and Guidelines
[slides] 		11:30 - 12:30 PM Assignment 2 out
[handout] [colab] 		Assignment 1 due
04/19 Project proposal due
04/20 Lecture 7: Training Neural Networks, part I
Activation functions, data processing
Batch Normalization, Transfer learning
[slides] 		Neural Networks, Parts 1, 2, 3
Suggested Readings:
  1. Stochastic Gradient Descent Tricks
  2. Efficient Backprop
  3. Practical Recommendations for Gradient-based Training
  4. Deep Learning, Nature 2015
04/22 Lecture 8: Training Neural Networks, part II
Backpropagation
Update rules, hyperparameter tuning, learning rate scheduling, data augmentation
[slides] 		Neural Networks, Part 3
Suggested Readings:
  1. An Overview of Gradient Descent Algorithms
  2. Why Momentum Really Works
  3. A Disciplined Approach to Neural Network Hyper-Parameters
04/23 Pytorch / Tensorflow Review Session
[pytorch] [tensorflow] 		11:30 - 12:30 PM
04/27 Lecture 9: CNN Architectures
AlexNet, VGG, GoogLeNet, ResNet
[slides] 		AlexNet, VGGNet, GoogLeNet, ResNet
04/29 Lecture 10: Recurrent Neural Networks
RNNs, LSTMs
Language modeling, Image captioning, Vision + Language
[slides] 		Suggested Readings:
  1. DL book RNN chapter
04/30 Midterm Review Session
11:30 - 12:30 PM Assignment 2 due
05/04 Midterm (No Lecture)
24 hours Assignment 3 out
[handout] [colab]
05/06 Lecture 11: Attention and Transformers
[slides] 		Suggested Readings:
  1. Attention is All You Need [Original Transformers Paper]
  2. Attention? Attention [Blog by Lilian Weng]
  3. The Illustrated Transformer [Blog by Jay Alammar]
  4. ViT: Transformers for Image Recognition [Paper] [Blog] [Video]
  5. DETR: End-to-End Object Detection with Transformers [Paper] [Blog] [Video]
05/07 Review Session: Learning on Videos
[slides] 		11:30 - 12:30 PM
05/10 Project milestone due
05/11 Lecture 12: Generative Modeling
[slides] 		Suggested Readings:
  1. Image GPT: Generative Pretraining From Pixels [Paper] [Blog]
05/13 Lecture 13: Self-supervised Learning
[slides] 		Suggested Readings:
  1. Lilian Weng Blog Post
  2. DINO: Emerging Properties in Self-Supervised Vision Transformers [Paper] [Blog] [Video]
05/14 Review Session: Detection Software
[slides] 		11:30 - 12:30 PM
05/18 Visualizing and Understanding
Feature visualization and inversion
Adversarial examples
DeepDream and style transfer
[slides]
05/20 Lecture 15: Detection and Segmentation
Semantic segmentation
Object detection
Instance segmentation
[slides]
05/25 Lecture 16: Neural Radiance Fields (guest lecture by Jon Barron)
[slides] 		Assignment 3 due
05/27 Lecture 17: Scene Graphs
[slides]
06/01 Lecture 18: Multimodal Learning (guest lecture by Ruohan Gao)
06/03 Lecture 19: Robot Learning (guest lecture by Yuke Zhu)
Project final report due
06/04 Project final presentation due