A New Interpretation of the Shannon Entropy Measure

Abstract

Although more than sixty years have elapsed since Shannon's seminal information entropy paper the literature reveals that there are divergent opinions of what it actually measures. From its similarity to Boltzman entropy in statistical mechanics, the most common view is that it measures information disorder and uncertainty. Based on an inductive derivation of the expression we propose a new interpretation relating to the use of symbols to uniquely identify probabilistic messages. Applying this new meaning to Shannon's English language experiment we extract a new interpretation of those results. Moreover, the new understanding of Shannon entropy also has the potential to improve the effectiveness of intelligence analysis applications.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Jan 01, 2015
Accession Number
ADA620109

Entities

People

  • Lewis Warren

Organizations

  • Defence Science and Technology Group

Tags

Communities of Interest

  • C4I
  • Energy and Power Technologies

DTIC Thesaurus Topics

  • Australia
  • Coding
  • Diseases And Disorders
  • Fuzzy Sets
  • Information Theory
  • Language
  • Literature
  • Mathematics
  • Mechanics
  • National Security
  • Numbers
  • Probability
  • Probability Distributions
  • Random Variables
  • Security
  • Set Theory
  • Uncertainty

Readers

  • Adaptive Control and Estimation with Uncertainty in Dynamic Systems.
  • Computational Linguistics
  • Systems Analysis and Design