Enhanced Testing of Autonomous Systems using Formal Methods
Abstract
Formal methods uses mathematical techniques to exhaustively explore models of systems for specification violations. Although much progress is being made the field, including work by the PIs, the scalability of formal approaches has not kept pace with the exponential increases in system and software complexity over time, which is observed even in safety critical flight code. As a result, we acknowledge that testing will drive the verification and validation process for autonomous systems. However, designers of autonomous systems, given finite time and budget constraints, cannot explore through all possible scenarios and uncertainties that an autonomous system might encounter. For example, the RAND corporation has estimated that, with statistical testing methods, “autonomous vehicles would have to be driven hundreds of millions of miles and sometimes hundreds of billions of miles to demonstrate their reliability in terms of fatalities and injuries.” Existing approaches, whether purely formal or purely test based, are thus inadequate for evaluating and improving the reliability of an autonomous system. For this reason, we propose a combined approach we call Enhanced Testing. At a high level, Enhanced Testing combines concrete system tests with formal methods approaches to discover branching behaviors in software and system models that are close to, but not explored by the existing tests.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jan 14, 2022
- Source ID
- FA95501910288
Entities
People
- Parasara Sridhar Duggirala
Organizations
- Air Force Office of Scientific Research
- United States Air Force
- University of North Carolina at Chapel Hill