Artificial Intelligence Driven Adaptive Sensors for Autonomous Object Recognition
Abstract
Our research team spanning two Universities, SUNY Buffalo and SUNY Poly, proposes to establish scientific and technological foundations of an Autonomous Object Recognition (AOR) system that comprises of Adaptable detectors/Sensors (AS) driven by an Artificial Intelligence (AI) agent. The distinct characteristic of the system is in the ability of AI to communicate with AS to get additional information on demand for AOR; while the AS would respond with an adapted/modified signal thus including the sensor into the refined AI loop thus exploiting the new level of system flexibility. The proposed AS will be integrated into the AI system, which will perform a variety of object recognition and sensor control tasks. The primary goal of the AI system is to perform object recognition, and it will be trained to utilize the greater diversity of image data obtained by the proposed sensor arrays. Based on object recognition results, the AI system will be able to derive the sensor array operational parameters optimized for current environmental conditions, and use the newly obtained imagery to refine the object recognition results. In contrast to existing systems, this feedback loop will include the selection of sensorÕs narrow spectral bands within IR spectrum, better accounting for image artifacts caused by environmental factors such as lighting level, rain, snow, fog, smoke, dust, chemicals, or camouflage. Given the multi-wavelength IR imagery produced by the adaptive sensor arrays, the AI system will also be able to produce additional attribute characteristics of recognized objects, such as the thermal signatures or chemical discharge profiles. The proposed research focus is two-fold: (a) To develop and demonstrate technology for a novel IR wide band adaptable detector based on asymmetrically doped double-quantum-well structure for 3-12 ?m spectral range. The team plans to numerically design, grow heterostructures using molecular beam epitaxy, process and test single pixels and small matrices of dual band (MWIR and LWIR) sensor with spectral sensitivity controlled with the external bias. (b) To design and demonstrate the improvement of object recognition fidelity using IR wavelength-specific calls from AI subsystem to an adaptable sensor. This requires novel algorithms for adaptive IR frequency selection for object recognition from multi-wavelength IR images. The team plans to train deep learning based AOR system on existing visible and IR spectrum data sets and implement multispectral fusion module. While the dedicated AS system is developed in (a), the team will initiate image acquisition, labelling and classification using a custom-made multi-camera setup based on commercial imagers. The setup will generate both indoors and outdoors multispectral object identification data sets which will be further used for the development of AI algorithms for adaptive control of image acquisition. The results from these two tasks will be further converged to outline the design of, and develop basic technologies for, an integrated AI system with online sensor adaptation capability.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Oct 07, 2021
- Source ID
- W911NF2110251
Entities
People
- Vladimir Mitin
Organizations
- Army Contracting Command
- United States Army
- University at Buffalo