Cyber-Resilient High-Dimensional Data Analytics with Analytical Guarantees
Abstract
This project aims to extract useful information from large amounts of networked data obtained by the Air Force. It develops a framework of computationally efficient and cyber-resilient data acquisition, data recovery, and data classification methods from high-dimensional measurements. In years 1 and 2, this project has (i) developed data recovery methods from low-bit measurements, missing data, and errors using low-dimensional models, and (ii) developed a theoretical foundation of computationally efficient neural network learning with provable guarantees. In year 3, the main research objective is to develop efficient and reliable learners with theoretical guarantees in neural network learning, given the limited resources and training samples.In this report, we mainly report the progress in year 3, while the progress reports for year 1 and year2 are attached in the end for the record. In year 3, we develop the theoretical foundation of joint data model sparsification and mixture-of-expert (MoE) in deep learning. Joint data-model sparsification focuses on sparsifying the data and neural network model simultaneously to reduce the computational cost. On the other hand, MoE executes training of different parts of the network (i.e. subnetworks) on different features in the data which allows the reduction of training compute and required training samples. In addition, wedesign the experiments on synthetic and real data to verify our theoretical findings.
Document Details
- Document Type
- Technical Report
- Publication Date
- Aug 22, 2023
- Accession Number
- AD1230943
Entities
People
- Meng Wang
Organizations
- Rensselaer Polytechnic Institute