Probing Materials from a Distance
Abstract
Today, computer vision technologies are being used in many aspects of our everyday lives. The data used by most computer vision systems are typically images and videos captured by conventional RGB cameras, and more recently, depth (RGBD) cameras. These cameras, along with modern advances in machine learning and deep learning techniques, have enabled vision systems to perform a wide range of tasks robustly, including object detection, segmentation and face identification. We believe an open and important problem in holistic scene understanding that complements these tasks is the recognition of materials. Our goal in this project is to design techniquesand systems that are capable of producing RGBDM images (M for material), thereby recovering material properties at an individual pixel level. If we know that a pixel corresponds to wood, metal, concrete, paper, plant or skin, this information would greatly empower virtually all vision algorithms including segmentation, detection and recognition. To this end, we propose using time-resolved scene measurements captured by high-speed single-photon cameras. These cameras record individual photons, and precisely measure their time-of-arrival. Not only do such a camera have extremely high sensitivity, but the captured data has an additional time-dimension, arich source of scene information that is inaccessible to conventional cameras. We will develop theory and techniques for modeling the time-resolved measurements resulting from various material attributes, including depth, translucency, fluorescence, and surface roughness. By combining these time-resolved measurements with polarization and spectrum cues, we will develop a comprehensive theory,practical computational techniques, and hardware prototypes for recovering a wide range of material properties from a distance, using compact material probe devices. Finally, we will create large-scale material property datasets for several real-world materials, and design hybrid physics- and learning-based techniques for recovering material properties in-the-wild from material probe measurements. If successful, the proposed research will result in vision and perception systems with capabilities that are significantly stronger than what is currently possible. These systems will not just spur wide-spread adoption of existing technologies, but also enable novel emerging applications such as autonomous navigation.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Mar 15, 2024
- Source ID
- N000142412155
Entities
People
- Mohit Gupta
Organizations
- Office of Naval Research
- United States Navy
- University of Wisconsin System