SUBTLE: Situation Understanding Bot through Language and Environment
Abstract
While highly constrained language can be used for robot control, robots that can operate as fully autonomous subordinate agents communicating via rich language remain an open challenge. Toward this end, the central goal of the SUBTLE MURI project was to develop an autonomous system that supports natural, continuous interaction with the operator through language before, during, and after mission execution. The operator communicates instructions to the system through natural language and is given feedback on how each instruction was understood as the system constructs a logical representation of its orders using linear temporal logic. While the plan is executed, the operator is updated on relevant progress via language and images and can change the robots orders. Unlike many other integrated systems of this type, the language interface is built using robust, general purpose parsing and semantics systems that do not rely on domain-specific grammars. The natural language system uses domain-general components that can easily be adapted to cover the vocabulary of new applications. We demonstrate the robustness of the natural language understanding system through a user study where participants interacted with a simulated robot in a search and rescue scenario.
Document Details
- Document Type
- Technical Report
- Publication Date
- Jan 06, 2016
- Accession Number
- AD1006698
Entities
People
- Cameron Finucane
- Constantine Lignos
- Daniel J. Brooks
- Hadas Kress-Gazit
- Holly Ann Yanco
- Ian Perera
- Kenton Lee
- Mikhail S. Medvedev
- Mitchell Marcus
- Vasumathi Raman
Organizations
- University of Pennsylvania