Functional Bregman Divergence and Bayesian Estimation of Distributions (Preprint)

Abstract

A class of distortions termed functional Bregman divergences is defined, which includes squared error and relative entropy. A functional Bregman divergence acts on functions or distributions, and generalizes the standard Bregman divergence for vectors and a previous pointwise Bregman divergence that was defined for functions. A recently published result showed that the mean minimizes the expected Bregman divergence. The new functional definition enables the extension of this result to the continuous case to show that the mean minimizes the expected functional Bregman divergence over a set of functions or distributions. It is shown how this theorem applies to the Bayesian estimation of distributions. Estimation of the uniform distribution from independent and identically drawn samples is used as a case study. We have defined a general Bregman divergence for functions and distributions that can provide a foundation for results in statistics, information theory and signal processing.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Jan 01, 2008
Accession Number
ADA479637

Entities

People

  • B. A. Frigyik
  • M. R. Gupta
  • Shivani Srivastava

Organizations

  • Purdue University

Tags

Communities of Interest

  • C4I

DTIC Thesaurus Topics

  • Calculus Of Variations
  • Case Studies
  • Discriminant Analysis
  • Distortion
  • Electrical Engineering
  • Electronic Mail
  • Gaussian Distributions
  • Information Science
  • Information Theory
  • Integrals
  • Probability
  • Probability Distributions
  • Random Variables
  • Signal Processing
  • Standards
  • Theorems
  • Vector Spaces

Readers

  • Adaptive Control and Estimation with Uncertainty in Dynamic Systems.
  • Aerial Unmanned Vehicle Swarm Micro Periodontal Dentistry.
  • Mathematical Modeling and Probability Theory.

Technology Areas

  • AI & ML
  • AI & ML - Bayesian Inference
  • AI & ML - Machine Learning Algorithms