Analyzing a Method to Determine the Utility of Adding a Classification System to a Sequence for Improved Accuracy
Abstract
Frequently, ensembles of classification systems are combined into a sequence in order to better enhance the accuracy in classifying objects of interest. However, there is a point in which adding an additional system to a sequence no longer enhances the sequence. Either the increase in operational costs exceeds the improvement in accuracy or the addition of the system does not increase accuracy at all. This research will examine a utility measure that determines the valid or invalid nature of adding a classification system to a sequence based on the ratio of the change in accuracy to the change in operational costs. Three sequential strategies defined on a two-class population outcome will be examined: Believe the Positive, Believe the Negative, and Believe the Extreme. This work expands upon known accuracy and cost equations for each strategy in order to generalize them for any fixed sequence length. Through simulation, this research identified which characteristics have the greatest impact on the utility measure and provides guidance on the threshold value of the utility measure that differentiates between when the addition of a system to the sequence may be useful (valid) and when it is not (invalid).
Document Details
- Document Type
- Technical Report
- Publication Date
- Mar 21, 2019
- Accession Number
- AD1075550
Entities
People
- Kevin S. Pamilagas
Organizations
- Air Force Institute of Technology