Quantitative Analysis of Evaluation Criteria for Generative Models
Abstract
The goal of this research is to provide a framework that can be used to inform and improve the process of generating synthetic semi-structured sequential data. A series of experiments evaluating a chosen set of metrics on discriminative ability and efficiency is performed. This research shows that the choice of feature space in which distances are calculated in is critical. The ability to discriminate between real and generated data hinges on the space that the distances are calculated in. Additionally, the choice of metric significantly affects the sample distance distributions in a suitable feature space. There are three main contributions from this work. First, this work provides the first known framework for evaluating metrics for semi-structured sequential synthetic data generation. Second, this work provides a "black box" evaluation framework which is generator agnostic. Third, this research provides the first known evaluation of metrics for semi-structured sequential data.
Document Details
- Document Type
- Technical Report
- Publication Date
- Mar 26, 2020
- Accession Number
- AD1104221
Entities
People
- Marvin W Newlin
Organizations
- Air Force Institute of Technology