Information Theoretic Measures for Performance Evaluation and Comparison

Abstract

This paper discusses the performance comparison of different algorithms for classification, estimation and filtering problems. Two information theoretic measures, namely, the empirical mutual information and the asymptotic information rate are proposed for simulation based performance evaluation and algorithm comparison. They can be used as a guideline for designing a practical procedure to measure the performance of different algorithms with limited computational resources. Other useful performance measures are reviewed and their relation to the two new measures discussed. Several practical examples are used to provide some insights on the inherent difficulty of algorithm ranking and the advantage of using the information theoretic measures for algorithm comparison.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Jul 01, 2009
Accession Number
ADA533345

Entities

People

  • Erik P. Blasch
  • Genshe Chen
  • Huimin Chen
  • Khanh Pham
  • Philip Douville

Organizations

  • University of New Orleans

Tags

Communities of Interest

  • Human Systems
  • Materials and Manufacturing Processes

DTIC Thesaurus Topics

  • Accuracy
  • Algorithms
  • Classification
  • Computer Simulations
  • Data Analysis
  • Detectors
  • Electrical Engineering
  • Estimators
  • Filtration
  • Multiple Hypothesis Tracking
  • Noise
  • Probability
  • Random Variables
  • Simulations
  • Statistical Inference
  • Target Tracking
  • Test And Evaluation

Readers

  • Operations Research
  • Regression Analysis.
  • Team-Based Human-Centered Cognitive Task Decision Making and Information Performance.