AI Prospects and Challenges Workshop
Abstract
We are currently in a Cambrian phase of AI, with thousands of researchers globally exploring variants of a common paradigm: Big datasets, deep neural networks, empirical validation on benchmarks. Some of this work has had outsize impact in the academic community, industrial domain, and the popular narrative. There are high expectations among the general population, unlikely to be met with the current paradigm. There are many efforts to explore beyond the common paradigm, but still relatively few. These include means of learning with few data: It has been remarked that the relevant limit for learning is not when the volume of training data goes to infinity, but when it goes to zero. Yet current systems are trained with massive datasets. Something appear amiss when the literature describes evidence that humans store about 1.5MBin the process of language acquisition, whereas we train models with billions if not trillions of parameters. Since natural language is a human construct, its complexity is no larger than the complexity of the union of human brains, whose capacity we appear to be approaching, possible having surpassed, with the latest wave of massive models. Accordingly, many have focused on semi-supervised, un-supervised, self-supervised learning, meta-learning, few-shot learning, zeroshot learning. Gaining the ability to learn with few data requires the ability to establish relations among concepts, entities and representation previously learned ontogenically or phylogenically from large volumes of data.
Document Details
- Document Type
- Technical Report
- Publication Date
- Jan 03, 2023
- Accession Number
- AD1212741
Entities
People
- Stefano Soatto
Organizations
- University of California, Los Angeles