Autonomy and Artificial Intelligence Test
Abstract
The AAIT Project continued test technology development supporting the challenges identified in the 2013–2038 DoD Unmanned Systems Integrated Roadmap, such as, integrating DoD unmanned systems within the National Airspace and safely operating unmanned aerial systems within the Major Range and Test Facility Bases (MRTFB). The AAIT project collaborated with the Autonomy Community of Interest (COI) Test and Evaluation, Verification and Validation (TEVV) Working Group to ensure that the AAIT project is investing in technologies relevant to the future of autonomous systems. The AAIT Project seeks solutions for legacy topics (test planning, test execution, safety, and performance assessment) but has also expanded our interest to ensure solutions for Artificial Intelligence and Machine Learning systems, topics identified by the intelligence community, and any other topics that are priority for TRMC and OUSD(R&E). The AAIT project initiated the Assured Development and Operation of Autonomous Systems (ADAS) effort. ADAS addresses the unique challenges of Autonomy test & evaluation to provide enterprise solutions in support of future programs. ADAS address autonomy test and evaluation verification and validation (TEV&V) needs across the full life cycle of an autonomy program from warfighter need identification to concept development and deployment. ADAS is exploring opportunities that support the overarching Autonomy T&E vision & strategy with actionable activity in requirements elicitation and formal modeling, linking formal models to implementation, composition for assurance, human autonomy interaction, development security operations, and simulation based test, live virtual constructive test, and integrated autonomous systems test. The AAIT Project explored technologies required for T&E of emerging UAS architectures, functional components, and interfaces. The AAIT project emphasized autonomy test technologies that can be integrated for use in a Test and Training Enabling Architecture (TENA) environment within the MRTFB. The AAIT Project continued investments in robustness testing technology to detect and predict safety-related vulnerabilities and failures within UAS software. The AAIT project is risk reducing Autonomy, Integration, and Teaming (AIT) test capability development, by providing autonomy test tools to be demonstrated on the Airborne Collision Avoidance System (ACAS-Xu) on Triton, and to test the Guardian Ground Based Detect and Avoid software, which will allow it to achieve certification for use during live test (DO-278A/NAVAIR Cert). The same technologies are risk reducing Autonomous Systems Test Capability (ASTC) development. The AAIT project used DARPA Collaborative Operations in Denied Environments (CODE) as a test case for this robustness technology, identifying and reporting on safety vulnerabilities found deep within the software, further identifying the conditions required to trigger the safety defects. The AAIT Project completed development of technology to improve test planning for ground and air autonomy using optimization algorithms to rapidly generate salient test scenarios. Expansion to the ground domain continued with the integration of AAIT technology into the Autonomous Ground Resupply (AGR) autonomy within the Autonomous Navigation Virtual Environment Laboratory (ANVEL) simulation. The integrated autonomy simulation will be used to validate AAIT technologies in the ground domain. New architecture and state-space designs better support multiple domains of autonomy testing. Unmanned Ground Vehicle and Undersea Vehicle domains test technology development will risk reduce CTEIP autonomous test capability development efforts. The AAIT Project is initiated development of technology to create machine-learned, behavioral copies of autonomy software. This technology creates faster-than-real-time versions of a given autonomy that can then be tested in an accelerated timeline in a simulated environment, and can also be cloned to be tested in parallel-processing fashion. This technology will provide faster, better, and more statistically significant testing data for testers.
Document Details
- Document Type
- Accomplishment
- Publication Date
- Oct 01, 2022
- Source ID
- 840415ef95d648487b1d6591def2c694