Learning Through Interaction for Language Understanding and Generation
Abstract
Understanding and generation of natural language are both core problems of interest in the study of autonomous reasoning and machine learning. The most common approach to language learning is using supervised learning. There are a number of drawbacks to the supervised paradigm that are especially pronounced in interactive systems. First, it performs poorly because it assumes an identical data distribution between training and use. This misses the dependence between model and human behaviors in interactive systems, where human participants are likely to change their behavior over time in response to system behavior. Second, it requires costly data collection and annotation. This is especially challenging in interactive grounded scenarios, where data collection requires complex interaction setups to mimic intended system behavior, and consistent annotation of long interactions is both costly and challenging. Finally, the supervised paradigm misses an important learning opportunity that is a natural byproduct of interactive scenarios. As humans interact with systems, they constantly give them feedback and correct them. However, in current research, such cues are largely ignored, missing an important opportunity for low-cost, scalable learning. More important, ignoring such learning cues misses an important aspect of language learning: it is a continuous process where each interlocutor balances between adapting their own behavior and signaling the other side to modify their behavior. This proposal studies algorithms and representations for continuous language learning in interactive systems, where interaction with users or acting in an environment is core to the system function. We develop methods for language learning from environment and user interaction. Instead of relying on dedicated training and data annotation, we aim to continuously learn while collaborating with users, experimenting in the environment, and completing tasks. The proposed work is organized along two thrusts: learning to understand natural language instruction and learning to generate instructions. In both thrusts, our focus is learning from human feedback or by observing human behavior. We study a a collaborative scenario, where humans and autonomous agents work side-by-side completing tasks and interacting using natural language. Our contributions are focused on defining a collaborative language-focused interaction scenario, a new class of efficient learning methods, a deep understanding of the type of learning signal human interaction provides, and linguistically-driven analysis of language learning within task-oriented collaborations.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jun 25, 2021
- Source ID
- W911NF2110106
Entities
People
- Yoav Artzi
Organizations
- Army Contracting Command
- Cornell University
- United States Army