Information Theoretic Regression Methods.

Abstract

Since the publication of the seminal note, Kuilback and Leiber (1951), there has been continual endeavor in statistics and related fields to explicate the existing statistical methods and to develop new methods based on the logarithmic information of Shannon (1948). During the last four decades numerous information theoretic regression methods have been developed. Kullback and Rosenblatt (1957) pioneered the information theoretic approach to regression by explicating the usual regression qyantities such as sums of squares and F ratios in terms of information functions. We have now information theoretic methods for model and predictive density derivation, parameter estimation and testing, model selection, collinearity analysis, and influential observation detection which can be used in sampling theory and Bayesian regression analyses. The purpose of this paper is to integrate the existing entropy-based methods in a single work, to explore their interrelationships, to elaborate on information theoretic interpretations of the existing entropy-based diagnostics and to present information theoretic interpretations for traditional diagnostics.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Apr 01, 1996
Accession Number
ADA313896

Entities

People

  • Ehsan Soofi

Organizations

  • George Mason University

Tags

Communities of Interest

  • C4I

DTIC Thesaurus Topics

  • Bayesian Inference
  • Bayesian Networks
  • Business Administration
  • Data Mining
  • Data Science
  • Databases
  • Estimators
  • Information Processing
  • Information Science
  • Information Theory
  • Knowledge Management
  • Network Science
  • Probability
  • Probability Distributions
  • Random Variables
  • Regression Analysis
  • Statistical Algorithms

Readers

  • Regression Analysis.
  • Systems Analysis and Design
  • Theoretical Analysis.

Technology Areas

  • AI & ML
  • AI & ML - Bayesian Inference
  • AI & ML - Machine Learning Algorithms