Workshop on Metacognitive Prediction of AI Behavior
Abstract
Conference Proposal: Workshop on Metacognitive Prediction of AI Behavior PI: Paulo Shakarian, Arizona State University In Response to W911NF-23-S-0001, Attn: Robert St. Amant Project Description. As artificial intelligence become more prevalent in military systems, improved characterization of such systems will, in-turn, become important to ensure that such systems are safe and reliable in supporting the warfighter. However, while AI systems, often using supervised machine learning or reinforcement learning, have provided excellent results for a variety of applications, the reasons behind their failure modes or anomalous behavior are generally not well understood. The idea of metacognition, reasoning about an AI system itself, is a key avenue to understanding the behavior and performance of machine learning systems. Recently, a variety of methodologies have been explored in the literature, which including stress testing of robotic systems [1], model introspection [2], model certification [3], and performance prediction [4]. Moreover, researchers across multiple disciplines including computer science, control theory, mechanical engineering, human factors, and business schools have explored these problems from different angles. The objectives of the workshop are as follows: ? Create a taxonomy of various approaches to metacognition of AI systems ? Understand the requirements for various metacognitive approaches ? Summarize recent results obtained in the study of AI metacognition ? Enumerate current applications for which AI metacognitive techniques have been applied ? Understand the relationship between AI metacognition and human operators Specific topics to be covered include, but are not limited to: ? Explainable performance prediction of black-box AI systems ? Stress testing of reinforcement learning systems ? How can metacognition be used to increase trust in AI systems by the operator ? Applications of AI metacognition to robotic and vision systems Workshop Location. We propose to hold the workshop at Arizona State University?s Skysong facility in Scottsdale, Arizona https://corporate.asu.edu/skysong Workshop dates (tentative): Oct. 9-11, 2023 Military Relevance. This workshop is directly relevant to ARO?s ?Knowledge Systems? topic (ARL-BAA-0033) specifically ?Engineering AI Systems.? The lack of modularity, performance guarantees, and general knowledge of AI system behavior has precluded wide-scale deployment of such technology within the military ? especially for technology such as deep neural networks where behavior in certain situations can become unpredictable. Metacognitive prediction of AI behavior represents a key step in ensuring such technology is usable for mission critical DoD applications where reliability and robustness are important. Example Army use cases include autonomous ground vehicles, ISR, intelligence analysis, and cyber security. We are also in discussions with Cambridge University Press for the creation of an edited volume associated with the event which will capture the technical details of the presentations in a volume that can be used as a reference in the creation of future research programs.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Sep 01, 2023
- Source ID
- W911NF2310345
Entities
People
- Paulo Shakarian
Organizations
- Arizona State University
- Army Contracting Command
- United States Army