HKU Moodle

Course Instructor:  Professor Yi Ma

Teaching Assistants: Tianzhe Chu and Feng Chen

Lecture Time: Mondays 2:00pm - 4:50pm 

Course Description: 

This course aims to provide a rigorous and systematic introduction to the mathematical and computational principles of deep learning. We achieve this by centering the course around a common and fundamental problem behind almost all modern practices of artificial intelligence and machine learning such as image recognition and generation. The problem is how to effectively and efficiently learn a low-dimensional distribution of data in a high-dimensional space and then transform the distribution to a compact and structure representation. Such a representation can be generally referred to as a  memory learned from the sensed data. 

We will start with the most basic and classical cases of PCA, ICA, and Dictionary Learning that assume the distribution has linear and independent structures. To generalize these classical models and solutions to general data distributions, we introduce a universal computational principle for learning low-dimensional distributions: compression. As we will see, data compression provides a unifying view on popular approaches for distribution or representation learning such as Score Matching (for denoising) and coding Rate Reduction. Within this framework, modern deep neural networks, such as ResNet and Transformers, can all be mathematically fully interpreted as (unrolled) optimization algorithms to achieve better compression and representation. To ensure the learned representation to be correct and consistent, we will study the effective Auto-Encoding architecture that consists of both encoding and decoding (say for denoising and diffusion). In order for a learning system to be fully automatic and continuous, we will also study a powerful framework of Closed-Loop Transcription that enables the encoding and decoding networks to self-correct hence self-improve via the ubiquitous mechanism of closed-loop feedback. 

Prerequisites:

This course will explore various topics in Embodied Artificial Intelligence (AI), which is concerned with the perception and representation of the physical world by autonomous agents and their consequent physical interactions.