Good Applications for Crummy Machine Translation
Abstract
We have recently begun work in machine translation and felt that it would probably make sense to start by surveying the literature on evaluation. As we read more and more on evaluation, we found that the success of an evaluation often depends very strongly on the selection of an appropriate application. If the application is well-chosen, then it often becomes fairly clear how the system should be evaluated. Moreover, the evaluation is likely to make the system look good. Conversely, if the application is not clearly identified (or worse, if the application is poorly-chosen), then it is often very difficult to find a satisfying evaluation paradigm. We begin our discussion with a brief review of some evaluation metrics that have been tried in the past and conclude that it is difficult to identify a satisfying evaluation paradigm that will make sense over all possible applications. It is probably wise to identify the application first, and then we will be in a much better position to address evaluation questions. The discussion will then turn to the main point, a discussion of how to pick a good niche application for state-of-the-art (crummy) machine translation. Machine translation, Evaluation, Niche application, ALPAC Report, Multilingual computational linguistics.
Document Details
- Document Type
- Technical Report
- Publication Date
- Jul 01, 1991
- Accession Number
- ADA269768
Entities
People
- Eduard H. Hovy
- Kenneth W. Church
Organizations
- University of Southern California