Robustness Versus Fidelity in Natural Language Understanding
Abstract
A number of issues arise when trying to scale-up natural language understanding (NLU) tools designed for relatively simple domains (e.g. flight information) to domains such as medical advising or tutoring where deep understanding of user utterances is necessary. Because the subject matter is richer, the range of vocabulary and grammatical structures is larger meaning NLU tools are more likely to encounter out-of-vocabulary words or extra-grammatical utterances. This is especially true in medical advising and tutoring where users may not know the correct vocabulary and use common sense terms or descriptions instead. Techniques designed to improve robustness (e.g., skipping unknown words, relaxing grammatical constraints, mapping unknown words to known words) are effective at increasing the number of utterances for which an NLU sub-system can produce a semantic interpretation. However, such techniques introduce additional ambiguity and can lead to a loss of fidelity (i.e., a mismatch between the semantic interpretation and what the language producer meant). To control this trade-off, we propose semantic interpretation confidence scores akin to speech recognition confidence scores and describe our initial attempt to compute such a score in a modularized NLU sub-system.
Document Details
- Document Type
- Technical Report
- Publication Date
- Jan 01, 2004
- Accession Number
- ADA460978
Entities
People
- Johanna D. Moore
- Mark G. Core
Organizations
- University of Edinburgh