Ground and Airborne Synthetic Multi-Modal Imagery Data Generation for Training of Deep Learning Classifiers
Abstract
It is proposed to undertake a research effort to develop, test, and evaluate algorithms and techniques enabling automatic generation of synthetic multimodality imagery datasets (SMID) appropriate for training of Deep Learning Classifiers (DLCs) employed for remote sensing in maritime surveillance applications. With the promising fruition of high performance software and hardware for deep learning technologies over the past decade, it has become feasible to develop and train an appropriate model of deep learning classifier to achieve dependable near human-level classification capability based on multimodality imagery training datasets. However, at present the required coherent multimodality imagery datasets for maritime target objects of interest (TOIs) training is rarely available. Furthermore, generation of such large-scale datasets in the physical environment is costly, time consuming, and arduous to realize. This project has four specific objectives. The first research objective is to construct a high fidelity virtual environment (VE) simulation model for synthetic multimodality high frequency synthetic image rendering of realistic maritime TOIs (e.g., small boats, ships, vessels, vehicles) in their operating environment. This goal will be achieved by developing physics-based models of environment objects with appropriate radar cross section and geometrical attributes, as well as physics-based model of multimodality imaging cameras including: (1) passive Infrared (IR), (2) Light Detector and Ranging (LiDAR) and (3) Synthetic Aperture Radar (SAR) for the synthetic generation of multimodality imagery of TOIs. To accommodate these models, an optimized Ray Tracing technique will be developed for efficient 3D objects imaging in the proposed VE. To ensure realistic compatibility of the synthetically generated imagery data with those from real physical counterpart imaging systems, each digitally generated range, phase, and thermal imagery data will be conditioned by an appropriate additive noise compatible to its inherent sensing modality and its sensor optics and resolution. The second research objective is to develop a coherent scheme for systematic generation of synthetic multimodality imaging datasets of maritime TOIs in their common operating environments. The proposed scheme will establish ground-truths related to each imaging process. Such information will include description of test TOI descriptions and their physical parameters, imaging system spectral imaging parameters, and atmospheric condition parameters pertaining to each test scenario. The third research objective is to explore and exploit different deep learning classifier architectures and develop a suitable novel deep learning classifier as a proof-of-concept of the proposed approach – particularly, pre-training of DLC based on the newly generated SMID. The last objective of this project is bi-fold. One goal is to enhance and strengthen the research capability of Tennessee State University as a HBCU in this specific scientific and engineering discipline critical to the mission of the Department of Navy (DoN). The other goal is to rigorously recruit, involve, and train qualified STEM undergraduate and graduate underrepresented minority students with all technical aspects of this project. The latter objective will prepare the participating students as potential future engineering workforce for the Department of Defense (DoD) as well as DoN. This project will have significant impact towards DoN’s maritime security applications. Particularly, it produces large volume of realistic SMID for pre-training of deep learning classifiers employed for maritime surveillance applications while developing a unique deep learning classifier as a proof of concept. Furthermore, it develops future potential engineering workforce for employment by DoD.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jul 27, 2018
- Source ID
- N000141812738
Entities
People
- Amir Shirkhodaie
Organizations
- Office of Naval Research
- Tennessee State University
- United States Navy