Welcome to STATS 315 / DATA SCI 315
This is an introductory deep learning course using the Python programming language and the TensorFlow deep learning library.
- Textbook: We will not follow any one textbook too closely. Here are a few references we will use:
- Dive into Deep Learning by Zhang, Lipton, Li and Smola. An advanced text from research scientists at Amazon. It weaves together math, figures, and code in an interactive online resource available here. Code examples are provided in three frameworks: MXNet, PyTorch and TensorFlow.
- Deep Learning with Python (2nd edition) by Chollet. A solid hands-on guide oriented towards programmers from the creator of the Keras deep learning library. Ebook and print versions are available from Manning Publications
- Deep Learning by Goodfellow, Bengio and Courville. Written by three top deep learning researchers, this comprehensive book is required reading if you want to pursue your study of deep learning at a more advanced level. Print version is available from MIT Press and an online version is here.
- Undergraduate Courses on Deep Learning: Many universities now offer an introductory deep learning course, e.g., Berkeley, CMU, MIT, Stanford
- Canvas: You should access the Canvas class page for this course frequently. It will let you access important announcements and track course deliverables. (requires UM login)
- Slack: The course slack workspace is at um-wn23-stats315.slack.com (requires UM login)
- Days and Times: Tuesdays and Thursdays, 1-2:30
- Location: 170 WEISER (links for virtual lectures, if any, will be saved in the syllabus page on canvas)
Instructor Information
Name: Ambuj Tewari
Office Hours: 45 minutes immediately before and after the lecture in 270 West Hall
Email: tewaria@umich.edu
GSI Information
Name: Unique Subedi (Lab 002 Thu 08:30-10:00 in WH335)
Email: subedi@umich.edu
Name: Jacob (Jake) Trauger (Lab 003 Thu 2:30-4:00 in USB2234)
Email: jtrauger@umich.edu
Name: Sahana Rayan (Lab 004 Thu 4:00-5:30 in EH1084)
Email: srayan@umich.edu
Lab webpage (also has GSI office hours info): link
Grading
- Canvas quizzes (20%): Will drop two lowest scores
- Homeworks (30%): Assigned roughly every other week. Will drop one lowest score
- Midterm Exam (20%): In class, timed, multiple choice, open book
- Final Exam (30%): In class, timed, multiple choice, open book
Academic Integrity
The University of Michigan community functions best when its members treat one another with honesty, fairness, respect, and trust. The college promotes the assumption of personal responsibility and integrity, and prohibits all forms of academic dishonesty and misconduct. All cases of academic misconduct will be referred to the LSA Office of the Assistant Dean for Undergraduate Education. Being found responsible for academic misconduct will usually result in a grade sanction, in addition to any sanction from the college. For more information, including examples of behaviors that are considered academic misconduct and potential sanctions, please see https://lsa.umich.edu/lsa/academics/academic-integrity.html
Accommodation for Students with Disabilities
If you think you need accommodation for a disability, please let me know at your earliest convenience. Some aspects of this course, the assignments, the in-class activities, and the way the course is usually taught may be modified to facilitate your participation and progress. As soon as you make me aware of your needs, we can work with the Office of Services for Students with Disabilities (SSD) to help us determine appropriate academic accommodations. SSD (734-763-3000; http://ssd.umich.edu/) typically recommends accommodations through a Verified Individualized Services and Accommodations (VISA) form. Any information you provide is private and confidential and will be treated as such.
Mental Health and Well-Being
Students may experience stressors that can impact both their academic experience and their personal well-being. These may include academic pressures and challenges associated with relationships, mental health, alcohol or other drugs, identities, finances, etc. If you are experiencing concerns, seeking help is a courageous thing to do for yourself and those who care about you. If the source of your stressors is academic, please contact me so that we can find solutions together. For personal concerns, U-M offers a variety of resources, many which are listed on the Resources for Student Well-being webpage. You can also search for additional well-being resources here.
Schedule
DLPy = Deep Learning with Python (2nd edition) by Chollet
DL = Deep Learning by Goodfellow, Bengio and Courville
D2L = Dive into Deep Learning by Zhang, Lipton, Li and Smola
Note: A “V” in the date column denotes a virtual lecture.
| Date | Topic | Reading Assignment |
|---|---|---|
| Jan 05 | Course logistics Introduction slides |
DLPy What is deep learning?, Chap. 1 DL Introduction, Chap. 1 D2L Introduction, Chap. 1 |
| Linear Algebra Boot Camp | ||
| Jan 10 | Linear Algebra notebook |
D2L Geometry and Linear Algebraic Operations, Sec. 22.1.1-2 |
| Jan 12 | Linear Algebra (continued) notebook |
D2L Geometry and Linear Algebraic Operations, Sec. 22.1.3-5 |
| Jan 17 | Linear Algebra (continued) notebook |
D2L Geometry and Linear Algebraic Operations, Sec. 22.1.6-7 D2L Geometry and Linear Algebraic Operations, Sec. 22.1.9 |
| Basics | ||
| Jan 19 | Basic Elements of Linear Regression slides |
D2L Linear Regresion, Sec. 3.1.1 |
| Jan 20 | HW 1 out | |
| Jan 24 | Regression Loss functions and gradient descent slides |
D2L Linear Regresion, Sec. 3.1.1 |
| Jan 26 V |
Regression wrap-up slides |
D2L Linear Regression, Sec. 3.1.3-4 |
| Jan 31 | Review of last two lectures | |
| Feb 02 | Classification Softmax Operation Cross Entropy Loss Function slides |
D2L Softmax Regression, Sec. 4.1.1 D2L Loss Function, Sec. 4.1.2 |
| Feb 03 | HW 1 due | |
| Feb 04 | HW 2 out | |
| Feb 07 | Softmax Derivatives Information Theory Basics slides |
D2L Information Theory Basics, Sec. 4.1.3 |
| TensorFlow/Keras | ||
| Feb 09 | TensorFlow, Keras, Google Colab notebook |
DLPy, Sec. 3.1-4 DLPy, Sec. 3.5.1-2 |
| Feb 14 V |
First steps with TensorFlow notebook |
DLPy, Sec. 2.4.4 |
| Feb 16 V |
First steps with TensorFlow (continued) notebook |
DLPy, Sec. 3.5.3-4 |
| Feb 17 | HW 2 due | |
| Feb 18 | HW 3 out | |
| Feb 21 | STUDY DAY | |
| Feb 23 | MIDTERM EXAM | |
| Feb 28 | SPRING BREAK | |
| Mar 02 | SPRING BREAK | |
| Mar 07 | Getting started with NNs: Classification MNIST notebook |
DLPy, Sec. 2.1 |
| Mar 09 | Getting started with NNs: Classification IMDB notebook |
DLPy, Sec. 4.1 |
| Mar 12 | HW 3 due HW 4 out |
|
| Mar 14 | Getting started with NNs: Regression Boston Housing Price notebook |
DLPy, Sec. 4.3 |
| Mar 16 | Generalization Evaluating ML models notebook |
DLPy, Sec. 5.1-2 |
| Mar 21 | Improving model fit Regularizing your model notebook |
DLPy, Sec. 5.3 DLPy, Sec. 5.4.4 |
| Convolutional Neural Networks | ||
| Mar 23 | From Fully-Connected Layers to Convolutions notebook |
D2L, Sec. 7.1 |
| Mar 24 | HW 4 due | |
| Mar 25 | HW 5 out | |
| Mar 28 | Convolutions for Images notebook Padding and Stride notebook Multiple Input and Multiple Output Channels notebook |
D2L, Sec. 7.2 D2L, Sec. 7.3-4 |
| Mar 30 V |
Pooling notebook LeNet Different ways to build Keras models notebook |
D2L, Sec. 7.5-6 DLPy, Sec. 7.2 |
| Deep Learning for Time Series | ||
| Apr 04 | A temperature-forecasting example notebook |
DLPy, Sec. 10.2 |
| Apr 06 V |
A temperature-forecasting example (continued) notebook |
DLPy, Sec. 10.2 |
| Apr 11 | Understanding recurrent neural networks Advanced use of recurrent neural networks notebook |
DLPy, Sec. 10.3 DLPy, Sec. 10.4 |
| Apr 13 | Recurrent Neural Networks notebook |
D2L, Sec. 9.4 |
| Apr 14 | HW 5 due | |
| Apr 18 | Course Conclusion Ask Me Anything! slides |
no assigned reading not part of syllabus not recorded |
| Apr 26 | FINAL EXAM FROM 4 to 6 pm |