Knowledge Extraction from Human Interaction with AI
Abstract
Summary: Former Secretary of Defense James N. Mattis stated that artificial intelligence (AI) may change the "fundamental nature of,war". Soon humans will routinely collaborate with and "teach" AI domain knowledge to complement and extend AI s models. Focused on t,he common pattern searching tasks (e.g., mining intelligence from geospatial data), this proposal investigates methods to extract kn,owledge from human interaction with AI, e.g., cost-effectively annotating domain-specific concepts that lead to the classification o,f a pattern and learning how humans search for such p,riven statistical models that function as "black boxes", making it almost impossible for non-computing domain experts to comprehend,or control how AI automates a task, thus limiting how AI can assist humans in real-world settings.Prior Art: Considering human-AI co,llaboration as a bidirectional communication problem, prior work (e.g., DARPA s XAI program) mainly focused on making AI comprehensi,ble (AI to human), while making relatively less progress on the other direction (human to AI): humans often provide only a single la,bel for the training data and rely on AI to "reverse-engineer" the underlying human knowledge, at times causing biases and errors wh,en such knowledge is misrepresented.Objective & Technical Approaches: Focused on a common task for knowledge workersfinding certain, patterns from a large amount of data, I propose methods that extract two types of knowledge from human interaction with AI to compl, to cost-effectively annotate domain-specific concepts that serve as knowledge for identifying a pattern and further for AI to disco,ver new concepts using building blocks from a library of pre-defined concepts in similar domains;(ii) The "how knowledge (how to f,ind such a pattern): I will develop a computational representation of humans pattern searching task, formulate "perceived importanc,e"a metric to capture the underlying knowledgethat is computed via a log analysis of user input, investigate methods that restruct,ure an existing task to elicit humans expression of knowledge, and compare different approaches to incorporate such knowledge into,an AI s model.Future Navy Relevance: AI will play an increasingly important role in the future success of the US Navy and enabling h,umans to collaborate with AI is the key to operationalize the ever advances of AI. This project s outcome will contribute to human-A,I collaboration in pattern searching tasks with a direct implication on the Navyfor example, as geospatial intelligence becomes inc,reasing critical in various Naval operations, AI that can learn from human analysts can leverage their strategy ("how" knowledge) to, automate portions of their tasks, alleviate their workload, and produce more timely and comprehensive intelligence.PI Qualification,: With over a decade of experience in Human-Computer Interaction (HCI) research (40+ publications, four award-winning papers), I am,able to employ a suite of HCI methods as the right "ingredients" needed to solve the proposed research problems. I have been success,ful in managing analogous projects including an NSF CAREER Award, an NSF CRII award, and a Hellman Fellowship. I am at an ideal plac,e to carry out this project, having been collaborating with physicians in UCLA Health and already collected multiple data sets ready, to be used. Further, I have been able to convert my research efforts to STEM education, e.g., developing the new curriculum on huma,n-AI interaction, hosting NSF REU sites for community college transfer students, and introducing non-engineering undergraduates to H
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jul 08, 2022
- Source ID
- N000142212188
Entities
People
- Xiang Anthony Chen
Organizations
- Office of Naval Research
- United States Navy
- University of California, Los Angeles