Automated Metrics for Speech Translation

Abstract

In this paper, we describe automated measures used to evaluate machine translation quality in the Defense Advanced Research Projects Agency's Spoken Language Communication and Translation System for Tactical Use program, which is developing speech translation systems for dialogue between English and Iraqi Arabic speakers in military contexts. Limitations of the automated measures are illustrated along with variants of the measures that seek to overcome those limitations. Both the dialogue structure of the data and the Iraqi Arabic language challenge these measures, and the paper presents some solutions adopted by MITRE and NIST to improve confidence in the scores.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Jan 01, 2010
Accession Number
AD1108264

Entities

People

  • Beatrice Oshika
  • Christy Doran
  • Dan Parvaz
  • Greg Sanders
  • John Aberdeen
  • Karine Megerdoomian
  • Mark Arehart
  • Sherri Condon

Organizations

  • MITRE Corporation
  • National Institute of Standards and Technology

Tags

Communities of Interest

  • Biomedical

DTIC Thesaurus Topics

  • Accuracy
  • Arabic Language
  • Artificial Intelligence
  • Artificial Intelligence Software
  • Automated Speech Recognition
  • Automatic
  • Civil Affairs
  • Computational Science
  • Computer Languages
  • Corporations
  • Frequency
  • Human Behavior
  • Information Retrieval
  • Language
  • Machine Translation
  • Military Personnel
  • Natural Language Computing
  • Natural Language Processing
  • Natural Languages
  • Recognition
  • Standards
  • Stemming
  • Test And Evaluation
  • Test Sets
  • Translations

Readers

  • Library and Information Science
  • Software Engineering.
  • Speech Processing/Speech Recognition.

Technology Areas

  • AI & ML
  • AI & ML - DoD AI Strategy
  • AI & ML - Machine Translation