An Evaluation Methodology for Natural Language Processing Systems

Abstract

The Neal-Montgomery NLP Evaluation Methodology was developed under the 'Benchmark Investigation/Identification' project as a means of determining the linguistic competence of Natural Language Processing (NLP) systems. Embodied in an evaluation tool based on a detailed classification of linguistic phenomena with, currently, over 350 test items, the methodology produces descriptive profiles of NLP system linguistic capabilities, and can be applied without regard to the current system application, domain, or specific language processing function. To date, it has been applied to three NL data base query systems and three text processing systems. Researchers are invited to apply the evaluation methodology (included within this report) to their systems and report their critique and recommendations towards the evolution of standard evaluation procedures for NLP systems.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Dec 01, 1992
Accession Number
ADA263301

Entities

People

  • Christine A . Montgomery
  • Douglas J. Funke
  • Elissa L. Feit
  • Jeannette G. Neal

Organizations

  • Calspan

Tags

Communities of Interest

  • C4I
  • Counter IED
  • Human Systems
  • Weapons Technologies

DTIC Thesaurus Topics

  • Artificial Intelligence
  • Cognition
  • Command And Control
  • Computational Science
  • Computer Programming
  • Computer Programs
  • Computer Science
  • Computers
  • Data Analysis
  • Databases
  • Employment
  • Information Processing
  • Language
  • Linguistics
  • Natural Language Processing
  • Natural Languages
  • Personnel Management

Readers

  • Computational Linguistics
  • Software Engineering.

Technology Areas

  • AI & ML
  • AI & ML - Machine Translation