Machine vision for real-world physical structure: 3D compositional networks based on biological mechanisms
Abstract
RESEARCH PROBLEM: Even after the advent of deep convolutional networks (DCNs), machine vision remains fragile (prone to catastrophic"" failure under difficult visual conditions) and limited to coarse, semantic information (typically, about categorical object identit""y). To be useful for future applications like autonomous guidance and physical interaction with the world, machine vision will need"" to achieve the performance of biological vision, in two senses. First, it must be robust (consistently accurate) across the entire"" range of real world visual cues and obstacles, including specular reflection, transparent refraction, shape from motion, camouflage"", and partial occlusion. Second, it must generate deep knowledge about physical reality: detailed, complete physical information abo""ut the 3D world, including precise 3D shape and spatial configuration of objects, object parts, surfaces, and other scene elements."" These two requirements are closely related, since deep knowledge of physical reality by definition goes beyond the variability of i"mage cues. OBJECTIVES: We plan to leverage (i) neuroscientific discoveries about 3D shape processing in high-level primate visual c"ortex, combined with (ii) virtual reality (VR) for experimental control of 3D physical parameters in realistic scenes, to jumpstart" a next generation of machine vision systems with the robustness and deep knowledge of 3D physical reality that characterize biologi"cal vision. TECHNICAL APPROACHES: (i) We will use parametric 3D VR stimuli, genetic algorithm-based exploration of 3D object and sc""ene stimulus space, and microelectrode recording to measure 3D shape coding functions of individual neurons in multiple stages in th"e visual pathway devoted to object/scene vision. We will use DCN methods to train subnetworks to replicate the coding functions of i"ndividual neurons, and combine subnetworks into large populations of virtual neurons that extract the same information from physical" scenes that visual cortex does. (ii) We will use 2-photon imaging and microwire bundle recording to study local visual processing algorithms at multiple levels in the object/scene pathway. We will use the results to guide design of novel network architectures. (iii) We will use the subnetwork populations and architectural innovations to build DCN vision systems and demonstrate their robustness to adverse visual conditions and their ability to extract 3D shape information from realistic VR scenes. (iv) We will also train n"etworks from scratch using tasks that require 3D shape knowledge. We will use genetic algorithm testing, analogous to the neuroscien""tific experiments, to characterize the information processing roles of individual units in these networks, in order to compare solut"ions found in biology to solutions found with DCN learning.ANTICIPATED OUTCOME: We anticipate that this work will initiate a new ph"ase in machine vision research, directly utilizing neuroscientific discoveries about algorithmic processing and representation of th""e physical world, to achieve robust recognition across visual conditions combined with deep knowledge about 3D physical reality.IMP""ACT ON DoD CAPABILITIES: Progress toward robust machine vision systems with deep knowledge about the 3D physical world, which will b"e essential for autonomous guidance and control of physical interactivity.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Feb 20, 2018
- Source ID
- N000141812119
Entities
People
- Charles Connor
Organizations
- Johns Hopkins University
- Office of Naval Research
- United States Navy