The TACITUS System: The MUC-3 Experience
Abstract
SRI International has been engaged in research on text understanding for a number of years. The Naval Ocean Systems Center (NOSC) has sponsored three workshops in recent years for evaluating text understanding systems. SRI participated in the first Message Understanding Conference (MUC-1) in June 1987 as an observer, and subsequently as a participant. Our system was evaluated in the second and third workshops, MUC-2 and MUC-3. For MUC-2, the task that the systems had to perform was to extract information for database entries saying who did what to whom, when, where, and with what result. The application domain for MUC-3 was news articles on terrorist activities in Latin America. The task was similar to that in MUC-2, though somewhat more information had to be extracted. The principal measures in the MUC-3 evaluation were recall and precision. Recall is the number of answers the system got right divided by the number of possible right answers. It measures how comprehensive the system is in its extraction of relevant information. Precision is the number of answers the system got right divided by the number of answers the system gave. It measures the system's accuracy. The system SRI used for these evaluations is called TACITUS. TACITUS is a system for interpreting natural language texts that has been under development since 1985. It has a preprocessor and postprocessor currently tailored to the MUC-3 application. It performs a syntactic analysis of the sentences in the text, using a fairly complete grammar of English, producing a logical form in first-order predicate calculus. Pragmatics problems are solved by abductive inference in a pragmatics, or interpretation, component.
Document Details
- Document Type
- Technical Report
- Publication Date
- Oct 01, 1991
- Accession Number
- ADA461049
Entities
People
- David Magerman
- Douglas E. Appelt
- Jerry R. Hobbs
- John S. Bear
- Mabry Tyson
Organizations
- SRI International