Research on Advanced Continual Learning Algorithms Beyond Simple Supervised Classification
Abstract
Self-supervised learning has recently emerged as a cost-efficient approach for learning representations from data, eliminating the need for laborious data labelling. Specifically, the representations learned by cross correlation-based contrastive learning algorithms, are shown to have excellent quality, comparable to those learned from supervised learning. Despite such success, huge memory and computational complexities are the apparent bottlenecks for easily maintaining and updating the self-supervised learned models, since they typically require a large-scale unsupervised data, large mini-batch sizes, and numerous gradient update steps for effective training. To that end, continual self-supervised learning (CSSL), in which the aim is to learn progressively improved representations from a sequence of unsupervised data, can be an efficient alternative to the high-cost, jointly trained self-supervised learning.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Feb 16, 2024
- Source ID
- FA23862314079
Entities
People
- Taesup Moon
Organizations
- Air Force Office of Scientific Research
- Seoul National University
- United States Air Force