YIP Interactive Testing, Assessment, and Repair for Rapid Autonomy Adoption
Abstract
This project aims to accelerate the adoption of autonomous systems by developing advanced methods for automatic testing, assessment, verification, and robust design. Autonomous systems have the potential to revolutionize naval operations, but their adoption is hindered by the lack of effective testing, evaluation, verification, and validation (T&E and V&V) methods. Current techniques strugglewith high complexity, nonfunctional requirements, high cost, limited coverage, and scarcity of failure data in autonomous systems. This project aims to develop a suite of advanced methods to address these challenges and accelerate the adoption of safe and trustworthy autonomous systems.Our unique technical approach is centered around four key thrusts:Objective 1: Automatic test-case generation of diverse failures for nonfunctional requirements, leveraging Bayesian inference and density evolution to achieve high coverage of system behaviors to test mission and operational performance at a low cost.Objective 2: Enable a fast and deep understanding of the failures. We will develop methods that perform root-cause analysis to infer, based on data collected during a particular set of failure events, what went wrong, i.e., what changes in the system and environment were associated with the observed failures. We also want to give an optimal strategy to allocate tests in simulation and on hardware using the failures we have observed.Objective 3: Interactive assurance case construction using natural language and large language models (LLMs), combining the strengths of LLMs with T&E methods.Objective 4: Integration of the developed methods into a unified framework and extensive experimental evaluation on various autonomous system platforms.If successful, this research will yield powerful tools and techniques for T&E and V&V of autonomous systems. The anticipated outcomes include generating diverse test scenarios for corner cases, providing assurance guarantees for mission performance, diagnosing root causes of failure, improving system robustness, and effectively building assurance cases. These capabilities will significantly enhance the DoD s ability to rapidly adopt safe and reliable autonomous systems across a wide range ofnaval applications, from regular duties to combat-ready deployments and advanced manufacturing.The project leverages the PI s extensive expertise at the intersection of formal methods, machine learning, and autonomous systems. This unique skill set enables the PIto identify and solve fundamental research problems, advancing the scientific foundations of T&E and V&V for autonomous systems. Ultimately, this project will empower the DoD to harness the immense potential of autonomy while ensuring the highest standards of safety and reliability.Approved for Public Release.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jan 13, 2025
- Source ID
- N000142512080
Entities
People
- Chuchu Fan
Organizations
- Massachusetts Institute of Technology
- Office of Naval Research
- United States Navy