Measuring Human Readability of Machine Generated Text: Three Case Studies in Speech Recognition and Machine Translation
Abstract
We present highlights from three experiments that test the readability of current state-of-the art system output from (1) an automated English speech-to-text system (2) a textbased Arabic-to-English machine translation system and (3) an audiobased Arabic-to-English MT process. We measure readability in terms of reaction time and passage comprehension in each case, applying standard psycholinguistic testing procedures and a modified version of the standard Defense Language Proficiency Test for Arabic called the DLPT*. We learned that: (1) subjects are slowed down about 25% when reading system STT output, (2) textbased MT systems enable an English speaker to pass Arabic Level 2 on the DLPT* and (3) audiobased MT systems do not enable English speakers to pass Arabic Level 2. We intend for these generic measures of readability to predict performance of more application specific tasks.
Document Details
- Document Type
- Technical Report
- Publication Date
- Mar 01, 2005
- Accession Number
- ADA510892
Entities
People
- Clifford Weinstein
- Douglas Jones
- Douglas Reynolds
- Edward Gibson
- Martha Herzog
- Neil Granoien
- Wade Shen
Organizations
- Massachusetts Institute of Technology