Approaches Toward Developing Attention-Driven Models of Referring Expression Generation
Abstract
Future autonomous agents will need to achieve common ground through direct agent-to-agent communication in the field. One important form of linguistic expressions are known as referring expressions (REs), which direct an audiences attention (e.g., ``look at the group of vehicles west of the factory building). The process of producing REs is known as referring expression generation (REG). In this report, we document progress made toward improving our fundamental understanding of REG, during the course of a one-year Karles Fellowship awarded to the author. In particular, we focus on a particular case of REG task, quantified REG, where prior approaches grounded in the assumption of perfect, symbolic knowledge fail to generate human-like REs. We present two novel approaches to account for perceptual costs during REG. Both involve using an incomplete symbolic scene representation as a basis to perform REG. The first approach uses a model of perceptual cost to remove symbolic facts that are likely not encoded due to prohibitive time costs. The second approach uses NRLs ARCADIA computational framework to model the process of incremental and strategic perception.
Document Details
- Document Type
- Technical Report
- Publication Date
- Feb 06, 2023
- Accession Number
- AD1193088
Entities
People
- Gordon Briggs
Organizations
- United States Naval Research Laboratory