Computational Material Perception
Abstract
We propose a three-year research program aimed to realize computational material perception, i.e., the recognition of materials and their properties from holistic reasoning in images. The goal of computational material perception is to enable computers to reason about an image, for instance by identifying that a surface is made of ceramic, it is shaped like a cup, and the place appears to be a kitchen, to be certain that it is a ceramic cup. Research towards realizing computational material perception, in turn, will provide deep insights and testable hypotheses on how our brain achieves such visual information integration to perceive our complex but structured visual world.Our research program consists of three research thrusts that tackle deeply related three fundamental specific aims. The underlying central theme of our proposed research program is the marriage of the three ~R~s, the integration of recognition with reconstruction for reflectance recovery in arbitrary images (Specific Aim 1), the integration of recognition with reorganization for accurate material segmentation (Specific Aim 2), and the use of reconstruction and reorganization for recognition for understanding human visual perception (Specific Aim 3). The fundamental technical thread that realizes these interweaving of fundamental research thrusts is the computational realization of bottom-up top-down (material) information integration.Specific Aim 1 integrates generative and discriminative approaches to material recognition. We achieve this byleveraging object detection to detect objects in an image and apply discriminative material segmentation in thedetected window to devise statistical priors on the geometry and reflectance tailored to the class of objects.Specific Aim 2 focuses on the derivation of a deep neural network (DNN) architecture that can integrate scene context for accurate local material recognition. The key idea is to avoid the necessity of an intractable amount of training data for this. Our research focus will be on the optimal design for the bottom-up and top-down information integration and the derivation of algorithms for end-to-end training.Specific Aim 3 explores the derivation of computational models that extends computational material perception and can be applied to the study of human perception. We will focus our research on the derivation of computational models that address three fundamental questions that may serve as testable hypotheses of human visual perception: how can radiometric properties of a scene be leveraged in object recognition, how can local material recognition benefit object recognition, and can we design a deep neural network that embodies perceptual visual object attributes.The three research thrusts go hand-in-hand based on the central theme of the integration of the three fundamental computer vision researches (reconstruction, recognition, and reorganization), which also relates to the fundamental problem of bottom-up top-down information integration. The results will also likely provide deep insights and actionable models for human perception studies and a platform to foster fruitful multidisciplinary research, further extending the PI~s current efforts to bring together computer and human vision researchers to advance respective studies through material perception research.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- May 05, 2017
- Source ID
- N000141712406
Entities
People
- Ko Nishino
Organizations
- Drexel University
- Office of Naval Research
- United States Navy