Informational and Causal Architecture of Discrete-Time Renewal Processes

Abstract

Renewal processes are broadly used to model stochastic behavior consisting of isolated events separated by periods of quiescence, whose durations are specified by a given probability law. Here, we identify the minimal sufficient statistic for their prediction (the set of causal states), calculate the historical memory capacity required to store those states (statistical complexity), delineate what information is predictable (excess entropy), and decompose the entropy of a single measurement into that shared with the past, future, or both.The causal state equivalence relation defines a new subclass of renewal processes with a finite number of causal states despite having an unbounded interevent count distribution. We use the resulting formulae to analyze the output of the parametrized Simple Nonunifilar Source, generated by a simple two-state hidden Markov model, but with an infinite-state-machine presentation. All in all, the results lay the groundwork for analyzing more complex processes with infinite statistical complexity and infinite excess entropy.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Jul 13, 2015
Accession Number
AD1053655

Entities

People

  • James P. Crutchfield
  • Sarah E. Marzen

Organizations

  • University of California, Berkeley

Tags

Communities of Interest

  • Energy and Power Technologies

DTIC Thesaurus Topics

  • Computational Science
  • Equations
  • Ergodic Processes
  • Excess Entropy
  • Generative Models
  • Hidden Markov Models
  • Information Theory
  • Markov Models
  • Markov Processes
  • Predictive Modeling
  • Probabilistic Models
  • Probability
  • Probability Distributions
  • Random Variables
  • Statistics
  • Stochastic Processes
  • Theorems

Readers

  • Computational Modeling and Simulation
  • Mathematical Modeling and Probability Theory.
  • Systems Analysis and Design