ACTIVE LEARNING SENSOR-OBJECT MODELS IN HIGHLY VARIABLE ENVIRONMENTS
Abstract
The proposed work will develop methods for robots to actively collect data to build up general-purpose machine learning models of se nsor-object relationships. Using neural networks and auto-encoders as a machine learning setting, we will synthesize motion that wi ll maximize the learning utility of each measurement, enabling an agent to both generalize and improve models for future use in sear ch and identification. The goal is to enable an autonomous system to use sensors with unknown physics to identify objects of unknown geometry and composition in challenging environments under data collection time constraints. Moreover, these techniques will enabl e autonomy to actively maintain and improve its learning representations as operational conditions change. The goal of this work is to enable robots to quickly collect data under severe time constraints in unforgiving environments to improve learning models creat ed in benign environments with unlimited time and compute. This work will generate active learning algorithms to automate data coll ection to improve classification of previously unknown objects using arbitrary sensors in environments with unspecified dynamics. F or instance, electrosense sensors can use electromagnetic fields to detect and identify objects under water, but for an object of un known geometry and material properties, in water that may have fluid dynamics and nonhomogeneous composition, the sensor-object rela tionship will generally be unknown and not amenable to first-principle analysis. This work will enable novel sensing modalities such as touch and electrosense, traditional sensors such as vision, and the fusion of multiple sensors, capitalizing on model-based cont rol of the autonomous agent that moves the sensor through the environment. Our technical approach will rely on our current advances in model-predictive control (MPC) for closing the loop on conditional variational autoencoders (CVAEs), a class of neural-network a rchitecture that first encodes a signal into a low dimensional latent space and then decodes the signal for prediction and performan ce evaluation. We have developed real-time algorithmic methods that optimize information content in the latent space through variat ional synthesis of the conditional decision variables (e.g., position in space for multi-view cameras that can move, input torque fo r mechanical contact using tactile sensing). The proposed work will generalize these methods to operation in three dimensional space , including orientation, for spatially-driven sensing problems such as machine vision. Moreover, the work will include analysis of l earning convergence and safety during data collection. Lastly, in addition to creating these sensory models, we will use learned se nsor-object models for search and identification. Experimental validation will include both mid-field sensing using vision and near- field sensing using tactile sensors. Experimental goals will focus on using offline representations of sensor-object models based o n data obtained in benign circumstances as seeds for real-time updating and creation of sensor-object models based on physical acces s in severe environments. We will mimic those environments by introducing lighting irregularities, occlusion, static distortion, vir tual disturbances (such as waveforms that might be found in water) and other types of distortion to vision. Lastly, we will experim entally search for objects using these classifiers, including multimodal sensing applications using both vision and tactile sensors simultaneously.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Aug 20, 2021
- Source ID
- N000142112706
Entities
People
- Todd D Murphey
Organizations
- Northwestern University
- Office of Naval Research
- United States Navy