New Faculty Spotlight: Simon Shaolei Du

What is your Research Focus?

In the last decade, breakthroughs in machine learning have relied on large models, especially deep neural networks. Most state-of-the-art models are over-parameterized, i.e., their number of parameters significantly exceeds the minimal number needed. For example, state-of-the- art deep neural networks have more than one billion parameters, far surpassing the size of the data. These models have become the backbones of recent advances in computer vision and natural language processing, where human-level performance was achieved in many tasks.

My research aims to gain a deep understanding of over-parameterization, including both its benefits and drawbacks. My work was among the first to show why over-parameterized neural networks can be optimized by simple algorithms, such as gradient descent. The grand goal is to have a mathematical theory on over-parameterization that rigorously characterizes the optimization and generalization properties of neural networks.

What opportunities at the UW excite you?

Machine learning is connected to many areas. I am excited about the interdisciplinary collaborations at UW.