Implicitly-Defined Neural Networks for Sequence Labeling

Abstract

We relax the causality assumption in formulating recurrent neural networks, so that the hidden states of the network are all coupled together. This goes beyond bidirectional RNN, which consists of two explicit recurrent networks concatenated together. The motivation behind doing this is to improve performance on long-range dependencies, and to improve stability (solution drift) in NLP tasks. We choose an implicit neural network architecture, show that it can be computed reasonably efficiently, and demonstrate it proof of-concept on the task of part-of-speech tagging.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Sep 09, 2016
Accession Number
AD1033419

Entities

People

  • Michaeel Kazi

Organizations

  • Massachusetts Institute of Technology

Tags

Communities of Interest

  • Energy and Power Technologies

DTIC Thesaurus Topics

  • Air Force
  • Air Force Research Laboratories
  • Artificial Intelligence Software
  • Boundary Value Problems
  • Computational Linguistics
  • Computational Science
  • Computer Languages
  • Computing System Architectures
  • Deep Learning
  • Equations
  • Language
  • Linguistics
  • Machine Translation
  • Natural Language Processing
  • Neural Networks
  • Recurrent Neural Networks
  • Signal Processing

Fields of Study

  • Computer science

Readers

  • Computational Fluid Dynamics (CFD)
  • Computational Linguistics
  • Neural Network Machine Learning.

Technology Areas

  • AI & ML
  • AI & ML - Machine Learning Algorithms
  • AI & ML - Machine Translation
  • AI & ML - Neural Networks