MUC-3 Test Results and Analysis
Abstract
Table 1 summarizes LSI 's scores for Phase II of MUC-3 . In evaluating these scores, which place LSI about two/thirds of the way down in a results-ranked list of MUC-3 participants, it should be noted that our MUC- 3 system reflects a major redevelopment of the key components of the DBG message understanding system, which is currently in process . Specifically, innovative development of a parser based on government-binding principle s is under way, 2 with associated revisions of the lexicon, functional parse (recovery of the predicate/argument functions representing the underlying semantic structure of the sentence), and DBG template 3 generation and frame hierarchy components (the areas indicated by the heavy lines in the system flow chart shown in Figure 1) . This innovative development is described more fully in the system summary paper. For the purposes of this site report, it is obvious that the "under construction" status of the DBG system had considerable impact upon our ability to achieve a respectable score. Had we chosen instead to go with the fairly robust previous version of the DBG system (described in [2] and [3], recently evaluated for Rome Laboratory by KSC, Inc .), our MUC-3 scores would certainly have been substantially better, because all components of the DBG system would have been fully functional (see the discussion on functionality of the DBG version currently under development below). However, we felt strongly that the time had come to replace our chart parser with weighted rules by a more powerful and generic model that would provide a better foundation for current work, including automated translation and the integration of speech processing with the DBG system, as well as for the complex MUC-3 messages.
Document Details
- Document Type
- Technical Report
- Publication Date
- Jan 01, 1991
- Accession Number
- ADA458882
Entities
People
- Bonnie G. Stalls
- Christine A . Montgomery
- Robert E. Stumberger
- Robert S. Belvin