Surrogate and Invariance Boosted Contrastive Learning for Data Scarce Applications in Science
Abstract
Deep learning techniques have been increasingly applied to the natural sciences, e.g., for property prediction and optimization or material discovery. A fundamental ingredient of such approaches is the vast quantity of labeled data needed to train the model. This poses severe challenges in data-scarce settings where obtaining labels requires substantial computational or labor resources. Noting that problems in natural sciences often benefit from easily obtainable auxiliary information sources, we introduce surrogate and invariance-boosted contrastive learning (SIB-CL), a deep learning framework which incorporates three inexpensive and easily obtainable auxiliary information sources to overcome data scarcity. Specifically, these are: abundant unlabeled data, prior knowledge of symmetries or invariances, and surrogate data obtained at near-zero cost. We demonstrate SIB-CL's effectiveness and generality on various scientific problems, e.g., predicting the density-of-states of 2D photonic crystals and solving the 3D time independent Scherdinger equation. SIB-CL consistently results in orders of magnitude reduction in the number of labels needed to achieve the same network accuracies.
Document Details
- Document Type
- Technical Report
- Publication Date
- Jul 21, 2022
- Accession Number
- AD1204920
Entities
People
- Christensen Thomas
- Dangovski Rumen
- Kim Samuel
- Loh Charlotte
- Soljai Marin
Organizations
- Massachusetts Institute of Technology