Validating Simulator-based Learning via Interpretation
Abstract
Simulated data has recently been shown useful for training machine learning algorithms for a wide variety of applications where real data is expensive to collect. However, the validity of machine learning algorithms trained with such simulated data has not been rigorously studied yet, raising signicant concerns if these algorithms can be employed in safety-critical applications such as emergency response- planning, medical decision-making and autonomous driving. Recently, researchers have employed interpretation as a tool to better understand validity and trustability of deep learning models. Inspired by this, in this work, we analyze the validity of simulator-based training methodologies by first obtaining interpretations and then measuring the quality of them. It is standard to measure the quality of the interpretation results based on human evaluations, which are however costly and subject to individual biases. To address this challenge, we leverage the flexibility of data simulator to propose a novel method for measuring the quality of interpretation without relying on human evaluations.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Oct 20, 2022
- Source ID
- FA23861914050
Entities
People
- Changho Suh
Organizations
- Air Force Office of Scientific Research
- KAIST
- United States Air Force