Information-Theoretic Properties of Languages and Their Grammars.

Abstract

This document describes means for computing a number of information-theoretic properties of languages and their grammars. For example, the entropy of a system of symbols is widely recognized as a measure of that system's complexity and organization. It is shown how the entropy of a language can be computed in a simple way from a grammar annotated with production probabilities. The author then develops means for statistically estimating these production probabilities from measureable properties of strings in the language. He also considers the computation of other information theoretic properties of language and grammars, such as the average information born by a symbol in a language and the average information used by the productions of a grammar. (Author)

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Aug 01, 1984
Accession Number
ADA146688

Entities

People

  • B. J. Maclennan

Organizations

  • Naval Postgraduate School

Tags

Communities of Interest

  • Energy and Power Technologies

DTIC Thesaurus Topics

  • Alphabets
  • Bits
  • Calculus
  • Computations
  • Computer Science
  • Computers
  • Context Free Grammars
  • Demographic Cohorts
  • Equations
  • Formal Languages
  • Grammars
  • Information Theory
  • Language
  • Military Research
  • Power Series
  • Probability
  • Probability Distributions

Fields of Study

  • Computer science

Readers

  • Computer Science.
  • Library and Information Science
  • Statistical inference.