MATH 6373 Deep Learning and Neural Networks

M,W 1-2:30, Room: SEC 206

	When and Where Meeting time: MW 1-2:30 Meeting place: SEC 206 Office Hours: MW 2:30-3:30 or by appointment PyTorch workshop: Tue 11:30-12:30 on Teams Instructor Name: Demetrio Labate E-mail address: dlabate@math.uh.edu URL: http://www.math.uh.edu/~dlabate Teaching Assistant Name: Ummey Tanin E-mail address: utanin@cougarnet.uh.edu Office Hours: Thu 12-2 in person at F11; Tue 11:30-2:30 on Teams

Course Objectives

This course will introduce students to Artificial Neural Networks, with particular emphasis on the class of Deeep Neural Networks emerged during the last decade, and their applications. The lectures will cover basic algorithmic principles underlying automatic learning, implementation and properties of deep learning architectures and selected applications using real data.

Topics of the course include: multi-layer perceptrons, autoencoders, convolutional neural networks and a number of applications to illustrate aspectes of their implementation and performance on real data. There will be a weekly workshop focusing on Python coding, specifically the use of the PyTorch library, to implement different neural network architectures.

Prerequisite

Useful background resources on statistics:

Lecture notes

Homework and student assignments

Student evaluation is based on homework and final project.

(i) The homework assignments will consist of problems involving calculations, implementations and applications of neural networks discussed in class or during the PyTorch tutorials. For some assignments, rather than collecting the homework, there might be a quiz in class about the material of the homework. The quiz wil be announced in advance and you might be required to bring your laptop in class to solve the quiz.

(ii) The Final project requires to apply a combination of tools developed during the course to an applied problem. It can be carried out as a group project of 2-3 students. I will select the project in coordination with the students and will provide assistance during the senester. I will set up several deadlines during the semester to verify the completion of a number of intermediate objectives finalized to the preparation of a 15-to-20 min in-class presentation.

Homework Problems (list to be updated during the semester):

Final Projects

Final Project Overview

Reviewers questionnaire

Grading

90-100: A; 80-89: B; 70-79: C; 60-69: D; 0-59: F

LECTURE NOTES (to be updated):

Lecture Set 0: Class overview
Lecture Set 1: Feedforward Neural Networks - part 1
Lecture Set 2: Feedforward Neural Networks - part 2
Lecture Set 3: Convolutional Neural Networks - part 1
Lecture Set 4: Convolutional Neural Networks - part 2
Lecture Set 5: Beyond Convolutional Neural Networks
Lecture Set 6: Object Recognition Networks
Lecture Set 7: Autoencoders
Lecture Set 8: Recurrent Neural Networks
Lecture Set 9: Transformers
Lecture Set 10: Generative Adversarial Networks
Lecture Set 11: Implicit Neural Reperesentations
Lecture Set 12: Large Language Models and Worlds Models

PyTorch Lectures: PyTorch tutorial material by Michela Marini

Bibliographical References

Deep Learning, by Ian Goodfellow, Yoshua Bengio and Aaron Courville, available (with additional videos and links) at https://www.deeplearningbook.org/lecture_slides.html.
Foundations of Data Science, by Blum, Hopcroft and Kannan’s available free online at https://www.cs.cornell.edu/jeh/book.pdf. It includes material on the Curse of Dimensionality and various topics in machine learning.
The Elements of Statistical Learning by Hastie, Tibshirani and Friedman, Springer 2017. The authors have made this book freely available on the website https://web.stanford.edu/~hastie/ElemStatLearn/printings/ESLII_print12_toc.pdf

Deep Learning with PyTorch by Stevens, Antiga and Viehmann. The authors have made this book freely available here. Note that a second edition of this book was recently published.
Additional resources to PyTorch are found at the pytorch.org website which includes a manual for pytorch_with_examples.

Additional Course Information

Excused Absence Policy Regular class attendance, participation, and engagement in coursework are important contributors to student success. Absences may be excused as provided in the University of Houston Undergraduate Excused Absence Policy for reasons including: medical illness of student or close relative, death of a close family member, legal or government proceeding that a student is obligated to attend, recognized professional and educational activities where the student is presenting, and University-sponsored activity or athletic competition. Under these policies, students with excused absences will be provided with an opportunity to make up any quiz, exam or other work that contributes to the course grade or a satisfactory alternative. Please, when it is possible, inform the instructor in advance so that appropriate arrangements can be made for the make up evaluation. Please read the Consult the Excused Absence Policy for details regarding reasons for excused absences, the approval process, and extended absences. Additional policies address absences related to military service, religious holy days, pregnancy and related conditions, and disability.

Reasonable Academic Adjustments/Auxiliary Aids: The University of Houston complies with Section 504 of the Rehabilitation Act of 1973 and the Americans with Disabilities Act of 1990, pertaining to the provision of reasonable academic adjustments/auxiliary aids for disabled students. In accordance with Section 504 and ADA guidelines, UH strives to provide reasonable academic adjustments/auxiliary aids to students who request and require them. If you believe that you have a disability requiring an academic adjustments/auxiliary aid, please contact the Justin Dart Jr. Student Accessibility Center (formerly the Justin Dart, Jr. Center for Students with DisABILITIES).

Recording of Class: Students may not record all or part of class, livestream all or part of class, or make/distribute screen captures, without advanced written consent of the instructor. If you have or think you may have a disability such that you need to record class-related activities, please contact the Justin Dart, Jr. Student Accessibility Center. If you have an accommodation to record class-related activities, those recordings may not be shared with any other student, whether in this course or not, or with any other person or on any other platform. Classes may be recorded by the instructor. Students may use instructor’s recordings for their own studying and notetaking. Instructor’s recordings are not authorized to be shared with anyone without the prior written approval of the instructor. Failure to comply with requirements regarding recordings will result in a disciplinary referral to the Dean of Students Office and may result in disciplinary action.

Resources for Online Learning: The University of Houston is committed to student success, and provides information to optimize the online learning experience through our Power-On website. Please visit this website for a comprehensive set of resources, tools, and tips including: obtaining access to the internet, AccessUH, and Blackboard; requesting a laptop through the Laptop Loaner Program; using your smartphone as a webcam; and downloading Microsoft Office 365 at no cost. For questions or assistance contact UHOnline@uh.edu.

UH Email: Please check and use your Cougarnet email for communications related to this course. To access this email, login to your Microsoft 365 account with your Cougarnet credentials.

Academic Honesty Policy: High ethical standards are critical to the integrity of any institution, and bear directly on the ultimate value of conferred degrees. All UH community members are expected to contribute to an atmosphere of the highest possible ethical standards. Maintaining such an atmosphere requires that any instances of academic dishonesty be recognized and addressed. The UH Academic Honesty Policy is designed to handle those instances with fairness to all parties involved: the students, the instructors, and the University itself. All students and faculty of the University of Houston are responsible for being familiar with this policy.

Title IX/Sexual Misconduct: Per the UHS Sexual Misconduct Policy, your instructor is a “responsible employee” for reporting purposes under Title IX regulations and state law and must report incidents of sexual misconduct (sexual harassment, non-consensual sexual contact, sexual assault, sexual exploitation, sexual intimidation, intimate partner violence, or stalking) about which they become aware to the Title IX office. Please know there are places on campus where you can make a report in confidence. You can find more information about resources on the Title IX website at https://uh.edu/equal-opportunity/title-ix-sexual-misconduct/resources/.

Security Escorts and Cougar Ride: UHPD continually works with the University community to make the campus a safe place to learn, work, and live. Our Security escort service is designed for the community members who have safety concerns and would like to have a Security Officer walk with them, for their safety, as they make their way across campus. Based on availability either a UHPD Security Officer or Police Officer will escort students, faculty, and staff to locations beginning and ending on campus. If you feel that you need a Security Officer to walk with you for your safety please call 713-743-3333. Arrangements may be made for special needs. Parking and Transportation Services also offers a late-night, on-demand shuttle service called Cougar Ride that provides rides to and from all on-campus shuttle stops, as well as the MD Anderson Library, Cougar Village/Moody Towers and the UH Technology Bridge. Rides can be requested through the UH Go app. Days and hours of operation can be found at https://uh.edu/af-university-services/parking/cougar-ride/.

Syllabus Changes: Please note that the instructor may need to make modifications to the course syllabus. Notice of such changes will be announced as quickly as possible through your official cougarnet email.

MATH 6373 Deep Learning and Neural Networks

M,W 1-2:30, Room: SEC 206

When and Where

Instructor

Teaching Assistant

Course Objectives

Prerequisite

Useful background resources on statistics:

Homework and student assignments

Homework Problems (list to be updated during the semester):

Final Projects

Grading

LECTURE NOTES (to be updated):

Bibliographical References