Visual Turing test for computer vision systems
Abstract
In computer vision, as in other fields of artificial intelligence, the methods of evaluation largely define the scientific effort. Most current evaluations measure detection accuracy, emphasizing the classification of regions according to objects from a predefined library. But detection is not the same as understanding. We present here a different evaluation system, in which a query engine prepares a written test (“visual Turing test”) that uses binary questions to probe a system’s ability to identify attributes and relationships in addition to recognizing objects.
Document Details
- Document Type
- Pub Defense Publication
- Publication Date
- Mar 09, 2015
- Source ID
- 10.1073/pnas.1422953112
Entities
People
- Donald Geman
- Laurent Younes
- Neil Hallonquist
- Stuart Geman
Organizations
- Brown University
- Johns Hopkins University
- National Science Foundation
- Office of Naval Research