Visual Reasoning via Knowledge-Based Vision
Abstract
Visual Reasoning via Knowledge-Based Vision Fei-Fei Li, Computer Science Department, Stanford University Abstract Hundreds of billions of photographs and videos are generated by many users and sources each year. Whether the challenge is to recognize trillions of images, to search for more hours of videos than humans can possibly watch, or to task personal robots to work with humans, advanced visual intelligence lies at the core of these technologies. This poses an interesting challenge for the computer vision field, whose ultimate goal is to develop systems with rich visual reasoning capabilities. Great progress has been made in computer vision in recent years, especially assigning a single class label to an image when there is a foreground object. While the current paradigm of memorizing gigantic datasets of exemplars can largely solve problems like single-label classification, more complex reasoning requires far richer structured knowledge. We hypothesize that the final solution of vision requires learning and reasoning with naturalistic data as well as rich knowledge of the visual world: visual, physical and semantic attributes, as well as relationships among them. Our research is geared towards enhancing computer vision systems with structured knowledge with the goal of richer visual reasoning. We call this new breed of computer vision algorithms knowledge-based vision. Toward this end, the focus of this proposal is to leverage structured knowledge for rich visual reasoning on images. In other words, our research bridges computer vision and knowledge representation and reasoning for tackling high-level visual tasks. In particular, we propose to conduct research consisting of 3 thrusts: 1) Build a large-scale dataset of structured visual information annotated with rich knowledge meta-data; 2) Develop new representations and learning algorithms able to exploit structured knowledge for better image understanding; 3) Apply knowledge-based vision algorithms to several high-level visual tasks.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Aug 12, 2016
- Source ID
- N000141512813
Entities
People
- Fei-Fei Li
Organizations
- Office of Naval Research
- Stanford University
- United States Navy