Implicit regularization with strongly convex bias: Stability and acceleration
Abstract
Implicit regularization refers to the property of optimization algorithms to be biased towards a certain class of solutions. This property is relevant to understand the behavior of modern machine learning algorithms as well as to design efficient computational methods. While the case where the bias is given by a Euclidean norm is well understood, implicit regularization schemes for more general classes of biases are much less studied. In this work, we consider the case where the bias is given by a strongly convex functional, in the context of linear models, and data possibly corrupted by noise. In particular, we propose and analyze accelerated optimization methods and highlight a trade-off between convergence speed and stability. Theoretical findings are complemented by an empirical analysis on high-dimensional inverse problems in machine learning and signal processing, showing excellent results compared to the state of the art.
Document Details
- Document Type
- Pub Defense Publication
- Publication Date
- Dec 28, 2022
- Source ID
- 10.1142/s0219530522400139
Entities
People
- Bằng Công Vũ
- Lorenzo Rosasco
- Silvia Villa
- Simon Matet
Organizations
- Air Force Office of Scientific Research
- European Research Council
- Huawei
- University of Genoa