Source: Willow
Course webpage for the NYU Spring 2023 Course Special Topics in Data Science, DS-GA 3001-009 (Introduction to Computer Vision). This course aims to cover broad topics in computer vision, and is not primarily a deep learning course. We will cover topics in traditional computer vision such as camera geometry, image formation, segmentation, object recognition, classification, and detection (see Syllabus).
Logistics
-
DS-GA 3001-009 (Lecture)
Mondays 4:55pm-6:35pm
Location: 208, Silver Center for Arts and Science, 32 Waverly Pl, New York, NY 10011 (SILV 208, location link)
The class will be only in-person. Slides will be available after the class on this webpage and the Brightspace. (see Schedule). -
DS-GA 3001.010 (Lab)
Thursdays 5:55pm-6:45pm
Location: 60 Fifth Avenue, New York, NY 10011 (60FA 150, location link)
The labs will be in-person. Labs will be used to cover additional materials or to work through practical exercises with the TA. -
Office Hours:
Jean Ponce (jean.ponce@inria.fr)
Wednesdays, 2:00pm-3:30pm
Room No. 500, 60 5th Ave, New York, NY 10011
- Campuswire Link: Link
Please try first to post any questions about course logistics and material, HWs and final project on Campuswire. We also highly encourage you to help each other out (but please do not reveal answers). For additional questions, please email course staff.
Instructors
Jean Ponce (jean.ponce@inria.fr)
TAs and Graders
Ayush Jain (aj3152@nyu.edu)
Zuhaib Akhtar (za2023@nyu.edu)
Grading
Four programming assignments (50% of the grade) + final project (40% of the grade) + class participation and attendance (5%) + lab participation and attendance (5%). Homeworks should be submitted using the Brightspace site.
- Homework 1 on camera calibration. Link Due on Feb. 17.
- Homework 2 on Canny edge detector. Link Due on Mar. 10.
- Homework 3 on Epipolar Line Estimation. Link Due on Apr. 14.
- Homework 4 on Mean Shift Algorithm. Link Due on May 4.
- Final project: Fill in the course project survey form and submit the project abstract by March 13 on Brightspace. Draft report submissions by April 14. Final presentations will be held during the last lecture (Room 208, Silver Center for Arts and Science, 4:55 PM - 6:35 PM, May 8) and in final exam slot (Room 269, Global Center for Academic and Spiritual Life, 4:00 PM - 5:50 PM, May 12). The project report will be due May 7, 11.59 PM EST.
Participation and Attendance
You are expected to attend and participate in classes and labs in person. Class attendance will count for 5% of your grade and lab attendance will count for 5% of your grade.
Syllabus
- Introduction
- Camera geometry and calibration
- Filtering and feature detection
- Radiometry and color
- Texture and image segmentation
- Stereopsis
- Structure from motion and 3D models from images
- Object recognition - historical perspective
- CNNs for object classification and detection
- 3D CNNs, Applications in Medical Imaging
- Weakly-supervised and unsupervised approaches to image and video interpretation
References:
We do not require purchase of any textbooks and the course will be self-contained. You may wish to consult the resources below for additional material formalization.
- D.A. Forsyth and J. Ponce, “Computer Vision: A Modern Approach”, second edition, Pearson, 2011. (Link)
- R. Szeliski, “Computer Vision: Algorithms and Applications”. (PDF)
- R. Hartley and A. Zisserman, “Multiple View Geometry in Computer Vision”, Cambridge University Press, 2004. (Link)
- M.F. Land and D.E. Nilsson, “Animal Eyes”, Oxford University Press, 2012.
- O. Faugeras, Q.T. Luong, and T. Papadopoulo, “Geometry of Multiple Images,” MIT Press, 2001.
Schedule:
Note: lecture slides will be posted after each lecture.
| Date | Lecture | Topic | Link |
|---|---|---|---|
| 01/23 | Lecture 1 | Introduction to Computer Vision - 1 | Slides (PPTX) |
| 01/26 | Lab 1 | Introduction to Computer Vision - 1.5 | Slides (PPTX) |
| 01/30 | Lecture 2 | Introduction to Computer Vision - 2 | Slides (PPTX) |
| 02/02 | Lab 2 | Class Logistics + HW1 review | Slides (PPTX), HW1 (Link) |
| 02/06 | Lecture 3 | Camera geometry and calibration | Slides (PPTX) |
| 02/09 | Lab 3 | Introduction to Computer Vision - 3.5 | Slides (PPTX) |
| 02/13 | Lecture 4 | Image Filtering | Slides (PPTX) |
| 02/16 | Lab 4 | Canny Edge Detectors and HW2 Review | Slides (PPTX) |
| 02/17 | Homework 1 DUE | HW on Camera Calibration | HW1 (Link) |
| 02/20 | No class | (Presidents’ Day) | |
| 02/23 | Lab 5 | Radiometry | Slides (PPTX) |
| 02/27 | Lecture 5 | Color and Radiometry | Slides (PPTX) |
| 03/02 | Lab 6 | Color and Radiometry | No slides |
| 03/06 | Lecture 6 | Color | Slides (PPTX) |
| 03/02 | Lab 7 | Radiometry and Reflectance | Slides (PPTX) |
| 03/10 | Homework 2 DUE | HW on Canny Edge Detector | HW2 (Link) |
| 03/13 | No class | (spring break) | |
| 03/13 | Course Project DUE | Project abstract DUE | Submission form (link) |
| 03/20 | Lecture 7 | Guest Lecture - Visual Learning in the Open World | Slides (PPTX) |
| 03/23 | Lab 8 | NMS and RANSAC Implementation | Notebook (PPTX) |
| 03/27 | Lecture 8 | Stereopsis | Slides (PPTX) |
| 03/23 | Lab 9 | HW3 Review: Epipolar Line Computation | Slides (PPTX) |
| 04/03 | Lecture 9 | Structure from Motion | Slides (PPTX) |
| 04/10 | Lecture 10 | Self-Calibration | Slides (PPTX) |
| 04/13 | Lab 10 | Introduction to computer vision - 11.5 | Slides (PPTX) |
| 04/14 | Course Project DUE | Draft Report DUE | Brightspace |
| 04/17 | Lecture 11 | Introduction to computer vision - 12 | Slides (PPTX) |
| 04/20 | Lab 11 | HW4 Review: Mean Shift | Slides (PPTX) |
| 04/24 | Lecture 12 | Guest Lecture by Albert Bietti on Generative Modeling | Slides (PPTX) |
| 04/27 | Lab 12 | Introduction to computer vision - 13 (Deep Learning) | Slides (PPTX) |
| 05/01 | Lecture 13 | Introduction to computer vision - 14 (Deep Learning) | Slides (PPTX) |
| 05/04 | Lab 13 | Introduction to computer vision - 14.5 | Slides (PPTX) |
| 05/07 | Course Project | Final Project Report DUE | Brightspace |
| 05/08 | Course Project | Project Presentations - I | 4:55 PM - 6:35 PM, Room 208, Silver Center for Arts and Science |
| 05/12 | Course Project | Project Presentations - II | 4:00 PM - 5:50 PM, Room 269, Global Center for Academic and Spiritual Life |
Acknowledgements
Much of the material for this course relies on the Computer Vision course given at ENS Paris by Mathieu Aubry, Karteek Alahari, Ivan Laptev, and Josef Sivic. Many of the slides are taken from James Hays, Svetlana Lazebnik, and Derek Hoeim. Website was originally designed by Matthew Trager.