Implicitly Defined Neural Networks for Sequence Labeling

Abstract

In this paper, we propose an implicit neural network architecture and show that it can be computed in a reasonably efficient manner. Our architecture relaxes the causality assumption in formulating recurrent neural networks, so that the hidden states of the network are coupled together, in order to improve performance on complex, long-range dependencies in either direction of a sequence. We contrast our architecture with a bidirectional RNN, and show that our proposed architecture the bidirectional network matches its performance on one task, while providing an ensembling benefit greater than ensembling multiple bidirectional networks.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Jul 31, 2017
Accession Number
AD1032195

Entities

People

  • Brian J. Thompson
  • Michaeel M. Kazi

Organizations

  • Massachusetts Institute of Technology

Tags

DTIC Thesaurus Topics

  • Air Force
  • Air Force Research Laboratories
  • Algorithms
  • Ambiguity
  • Artificial Neural Networks
  • Computational Linguistics
  • Computational Science
  • Computations
  • Computing System Architectures
  • Eigenvalues
  • Equations
  • Grammars
  • Health Care
  • Hidden Markov Models
  • Linguistics
  • Machine Translation
  • Markov Models
  • Military Research
  • Models
  • Monte Carlo Method
  • Natural Language Processing
  • Network Architecture
  • Neural Networks
  • Probabilistic Models
  • Recurrent Neural Networks
  • Sampling
  • Sequences
  • Signal Processing
  • Training
  • Vocabulary

Fields of Study

  • Computer science

Readers

  • Computational Fluid Dynamics (CFD)
  • Neural Network Machine Learning.

Technology Areas

  • AI & ML
  • AI & ML - Neural Networks