Axiomatic Characterization of A Family of Information Measures That Contains the Directed Divergences.

Abstract

Let p and q be probability densities. The directed divergences of p and q are given by the integral p(x) log (p(x)/q(x))dx and by the same expression with p and q interchanged; the divergence is the sum of the directed divergences. These quantities have applications in information theory and to the problem of assigning prior probabilities subject to constraints. In this report, it is shown that the directed divergences and their positive linear combinations, including the divergence, are characterized by axioms of positivity, additivity, and finiteness; in the course of the proof, the latter two are shown to imply yet another axiom; imvariance. These axioms are fundamental in work on prior probabilities. It has been claimed that they characterize only constant multiples of the single directed divergence; that claim is here refuted.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Nov 01, 1977
Accession Number
ADA050096

Entities

People

  • Rodney W. Johnson

Organizations

  • United States Naval Research Laboratory

Tags

Communities of Interest

  • Autonomy
  • C4I

DTIC Thesaurus Topics

  • Discrete Distribution
  • Equations
  • Hypotheses
  • Information Systems
  • Information Theory
  • Integrals
  • Intervals
  • Invariance
  • Military Research
  • Numbers
  • Probability
  • Real Numbers
  • Real Variables
  • Theorems
  • Uncertainty

Readers

  • Mathematical Modeling and Probability Theory.
  • Statistical inference.
  • Systems Analysis and Design