Validating Simulator-based Learning via Interpretation
Abstract
Simulated data has recently been proven useful for training machine learning algorithms for a wide variety of applications where real data is expensive to collect. However, the validity of machine learning algorithms trained with such simulated data has not been rigorously studied yet, raising significant concerns if these algorithms can be employed in safety-critical applications such as emergency response-planning, medical decision-making and autonomous driving. Recently, researchers have employed interpretation as a tool to better understand validity and trustability of deep learning models. Inspired by this, in this work, we analyze the validity of simulator-based training methodologies by first obtaining interpretable explanations and then measuring the accuracy of them. It is standard to measure the accuracy of explanations based on human evaluations, which are however costly and subject to individual biases. To address this challenge, we leverage the flexibility of data simulator to develop a novel method for measuring the accuracy of explanations without relying on human evaluations. More specifically, we extract ground-truth explanations from a simulator byaccessing the internal states of the simulator. With the collected ground truth explanations, which we call explanation labels, one can measure the accuracy of explanation without human evaluation. In order to quantify the explanation accuracy of a given machine learning model, one can apply an off-the-shelf explanation algorithm such as LIME and compare their results with the explanation labels. Our framework can be applied to any machine learning model trained with simulated data as long as the underlying data simulator is flexible and transparent enough to generate explanation labels. As an effort to exemplify the effectiveness of our methodology, we also demonstrate how one can apply it to collision prediction algorithms trained with a driving simulator.
Document Details
- Document Type
- Technical Report
- Publication Date
- Oct 11, 2022
- Accession Number
- AD1189109
Entities
People
- Changho Suh
Organizations
- KAIST