Active Similarity Learning and Manifold Graphs for Learning With Few Labels
Abstract
While significant strides had been made in deep learning in the 5 years subsequent to its resurgence (2012), a large part of this success was through large-scale training of deep neural networks on labeled data. Transfer learning was commonly used, but in reality understanding how to effectively fine-tune pre-trained models (such as those pre-trained on ImageNet) was not well understood. For example, the methods were susceptible to a shift in the data distribution from the source dataset (domain shift) and it was often not clear where to transfer from given a new problem. While early methods involving unsupervised, semi-supervised, domain adaptation, and few-shot learning had been developed (including by this team), they tackled small subsets of real-world problems and in practice often could not beat strong baselines with transfer learning alone. Since then, the landscape has drastically changed, and the methods developed under this program as well as by the community at large have enabled the usage of large-scale unlabeled pre-training combined with semi-supervised learning to significantly increase performance under low-labeled conditions. Their team contributed a significant number of advancements to this body of work, ranging from scientific formulations and understanding of the problems to the development of practical algorithms that performed well under both established academic datasets to the evaluations. Their work viewed these low-labeled problems by explicitly representing the structure of the data manifold using graph representations. Through this lens, they proposed a number of innovations that answer the questions of what to transfer, how to transfer, and where to transfer from.
Document Details
- Document Type
- Technical Report
- Publication Date
- Nov 01, 2023
- Accession Number
- AD1214504
Entities
People
- Chris Rozel
- Mark A. Davenport
- Phillip Odom
- Zsolt Kira
Organizations
- Georgia Tech