课程一​

• 课程名称：Modern Optimization

• 授课老师：Yurii Nesterov教授,美国科公司院士，比利时鲁汶大学

• 授课时间：2024/7/8-2024/7/19 14:00-16:00pm

•  

• 教学内容：

• In this course, we present all basic elements which are necessary for understanding the main results and the most important research directions of the Modern Optimization Theory. After demonstration of spectacular achievements of Artificial Intelligence, it became clear that Optimization is going to play a central role in the modern Computational Mathematics. However, in order to apply successfully this technique, it is necessary to know what we can and what we cannot do by Optimization Methods. We need to distinguish solvable and unsolvable problems and estimate properly the computational resources required for their solution. All this information is provided by Complexity Theory, which establishes the limits of our abilities and presents the best possible optimization schemes. This is the main subject of our course. All results are presented in a rigorous mathematical form, which makes our conclusions independent on the current level of the hardware development. For participants, only a basic knowledge of Nonlinear Analysis is needed.

•  

• 教学大纲：

• Part I: Black-Box Optimization

• Chapter 1. General Optimization 

• Lecture 1. The world of Nonlinear Optimization (A.1.1).

• Lecture 2. Local methods in Unconstrained Optimization (A.1.2).

• Chapter 2. Convex Differentiable Optimization

• Lecture 3. Minimization of Smooth Convex Functions (A.2.1).

• Lecture 4. First-order Optimal Methods (A.2.2).

• Chapter 3. Convex Non-Differentiable Optimization

• Lecture 5. General convex functions (A.3.1.1 – A.3.1.6).

• Lecture 6. Methods of Nonsmooth Optimization (A.3.2).

• Part II: Structural Optimization

• Lecture 7. Smoothing technique (A.6.1).

• Lecture 8. Self-concordant functions (A.5.1.1 – A.5.1.4, A.5.2.1, A.5.2.2).

• Lecture 9. Self-concordant barriers (A.5.3).

• Lecture 10. Applications to problems with explicit structure (A.5.4 – A.5.4.5, A.5.4.7 – A.5.4.9).

•  

• Main source

• A. Yu. Nesterov. Lectures on Convex Optimization. Springer (2018). 

• Lectures on Convex Optimization（点击下载）

•  

课程二

• 课程名称：Computational methods for sampling and analyzing rare and extreme events

• 授课老师：Jonathan Weare教授，美国纽约大学柯朗数学科学研究所

• 授课时间：2024/7/8-2024/7/19  9:00-11:00am

•  

• 教学内容：

• This course focuses on computational methods for sampling and analyzing rare and extreme events. Though they appear infrequently or with low probability, rare events have an outsized impact on our lives and society and are often the most important characteristic of a system to understand. Examples include extreme weather or climate events, failure of a reliable engineering product, and the conformational rearrangements that determine critical functions of biomolecules in our bodies. Unfortunately, by definition, rare events are difficult to observe and study. For this, across many fields, significant effort has been devoted to developing computational tools for the study of rare events. This class will be divided roughly into two parts. In the first part, after introducing some basic ideas and notations from Monte Carlo simulation, we will study tools for sampling from the tails of a given density, focusing mostly on methods based on importance sampling and stratification. This problem arises, for example, in parameter estimation problems in statistics, and free energy calculations in chemistry. In the second part we will focus on sampling rare paths of a given Markov process, a problem that arises in a wide range of fields, including, for example, climate science, chemistry, and logistics. Our focus will be on ``non-intrusive'' methods that do not require direct access to the rules governing the Markov process, making them appropriate for applications in settings in which the process is accessible only through experimental observations or simulation using legacy software.

•  

课程三

• 课程名称：Random Matrix Theory

• 授课老师：Joel Tropp 教授，加州理工公司

• 授课时间：

• 2024/7/22-2024/7/26，9:00-11:00 am，

  2024/7/29-2024/7/30，9:00-11:00 am，14:00-16:00 pm

•  

• 教学内容：

• This short course offers an accessible introduction to basic methods and ideas from random matrix theory. Topics will include matrix concentration inequalities, Gaussian ensembles, universality results, and some applications. The class requires strong foundations in linear algebra and probability theory, but the presentation will be self-contained.

•  

•  

课程四

• 课程名称：The Mathematics of Artificial Intelligence

• 授课老师：Zhi-Qin John Xu （许志钦）教授，上海交通大学

• 授课时间：2024/07/22-2024/07/26 14:00-16:00pm

•  

• 教学内容：

•  

  Artificial intelligence (AI), which is mainly powered by deep neural networks, is currently leading to one breakthrough after the other, both in public life with, for instance, autonomous driving and speech recognition, and in the sciences in areas such as medical diagnostics or molecular dynamics. However, one current major drawback is the lack of a mathematical foundation of such methodologies as well as an analysis of how to apply AI-based approaches reliably to mathematical problem settings.

   

  In this lecture we will provide an introduction and overview of this exciting area of research. First, we will discuss several phenomena observed during the training of neural networks. And, secondly, we will utilize these phenomena to uncover the underlying mathematical theories of neural networks and understand their performance, such as transformer-based large language models.

•  

  教学大纲：

• We will cover the following topics:

  (1) Training behavior of neural networks in the frequency domain (frequency principle)

  (2) Training behavior of neural networks in the parameter space (neuron condensation)

  (3) Loss landscape structure (embedding principle)

  (4) Implicit bias of stochastic optimization (SGD, Dropout)

  (5) Memorization and inference of Transformer networks