Engendering & Leveraging Trust in LongitudinalHuman-AI Interactions
Abstract
Decision-making in the modern navy is often characterized by scenarios where the decision-makers need to leverage complex AI systems to make important decisions in the presence of significant uncertainty and limited time. In these scenarios, the effectiveness of the decision-makers is in many ways increasingly limited by their ability to correctly assess when they can trust and rely on the assistance of their AI-powered tools. The problem of building AI systems that can engender an appropriate level of trust and reliance in the end-user is thus an important open challenge that needs to be tackled for us to meaningfully deploy AI systems in complex andmission-critical scenarios. In this project, we propose to deal with this exact problem while grounding it in the context of sequential decision-making that are ubiquitous in the Navy decision-making and mission planning scenarios. We will achieve this by building on our significant expertise in modeling the mental states of the users ofAI systems and the development of behavioral and communication techniques to influence the user#s mental models. In particular, through this project, we will formalize and operationalize amental model based theory of trust that has several desirable characteristics to make it an ideal candidate for use in longitudinalhuman-AI interaction. This framework formalizes human#s trust in a given AI system as a function of the human#s expectation of the system#s abilities. Thus any AI system that aims to take into account human trust in its decision-making should be capable of meeting this expectation and, as required updating them, to better reflect the agent#s true capabilities. We will also develop decision making frameworks that can leverage our mental model based theory of trust to guide longitudinal interactions in mission planning scenarios. Throughout this project, we will employ human studies to both evaluate the di#erent parts of the trust model we propose and also to evaluate theapplications we develop based of of this model. The work proposed here builds on our previous work with the Science of Autonomy program which lead to a rich framework for mental-model based explainable decision making.Approved for Public Release
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- May 15, 2023
- Source ID
- N000142312409
Entities
People
- Subbarao Kambhampati
Organizations
- Arizona State University
- Office of Naval Research
- United States Navy