Generating visual explanations with natural language
Abstract
We generate natural language explanations for a fine‐grained visual recognition task. Our explanations fulfill two criteria. First, explanations are class discriminative, meaning they mention attributes in an image which are important to identify a class. Second, explanations are image relevant, meaning they reflect the actual content of an image. Our system, composed of an explanation sampler and phrase‐critic model, generates class discriminative and image relevant explanations. In addition, we demonstrate that our explanations can help humans decide whether to accept or reject an AI decision.
Document Details
- Document Type
- Pub Defense Publication
- Publication Date
- Dec 01, 2021
- Source ID
- 10.1002/ail2.55
Entities
People
- Anna Rohrbach
- Bernt Schiele
- Lisa Anne Hendricks
- Trevor Darrell
- Zeynep Akata
Organizations
- Defense Advanced Research Projects Agency
- Max Planck Institute for Informatics
- University of California, Berkeley
- University of Tübingen