Integral: A Foundational Approach to Label Complexity via Information Theory and Graph Signal Processing
Abstract
The project's primary accomplishments can be summarized as follows: Firstly, the project made significant contributions to the field of transfer learning by establishing statistical minimax bounds. These bounds offer a precise understanding of the limits of knowledge transfer between related domains in classification and regression tasks. The study also extends to scenarios involving multiple source domains. Secondly, the research introduced a novel approach to understanding and optimizing complex deep learning systems through non-negative kernel regression (NNK) graphs, facilitating improved generalization estimation, clustering, and geometric metrics for network invariance assessment. Thirdly, the project rigorously assessed the performance of popular heuristics for data reduction, feature learning, and transfer learning. Lastly, the team proposed Federated Alternate Training (FAT) as a framework for global semi-supervised federated learning, providing a solution for collaboration in machine learning when labeled data is limited. Additionally, the project made significant contributions to statistical query lower bounds, showcasing their relevance in the presence of noisy data and cryptographic hardness, and also proposing gradient-descent type algorithms matching some of the lower bounds in specific cases.
Document Details
- Document Type
- Technical Report
- Publication Date
- Nov 01, 2023
- Accession Number
- AD1214479
Entities
People
- A. Salman Avestimehr
- Antonio Ortega
- Ilias Diakonikolas
- Mahdi Soltanolkotabi
Organizations
- University of Southern California