Critical Language Modelling with Trace-Density Representations
Abstract
Statistical language modelling, whose aim is to capture the joint probability distribution of sequences of words, has applications to problems of interest to the Army including information retrieval, speech recognition, artificial intelligence, man-machine interfaces, translation, and natural language problems that involve incomplete information. Current state-of-the art long-short term memory recurrent neural networks have made impressive gains over previous hidden Markov models, but still fail to match the higher order statistics of language, which exhibits a kind of critical behavior common to biological systems . A new trace-density model for language is proposed. It is inspired by quantum statistical physics which contain many examples of solvable models that exhibit the same kind of criticality that biological systems do. The model involves a novel cubic constraint which reduces the parameters of the model to topologically compact moduli space, making it feasible to find a model using a maximal entropy computer-training algorithm.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Oct 11, 2018
- Source ID
- W911NF1710393
Entities
People
- John Terilla
Organizations
- Army Contracting Command
- Queens College
- United States Army