HARMONIC: Human-AI Robotic Team Member Operating with Natural Intelligence and Communication
Abstract
This project will develop HARMONIC, a Human-AI Robotic Team Member Operating with Natural Intelligence and Communication, that willa) perform as a member a human-robotic team (HRs) performing shipboard maintenance and repair tasks, b) earn the trust of its humanteammates due to its ability to explain decisions and actions of self and others; c) learn new skills on the job from teammates by cognitively interpreting results of multimodal perception from language and demonstration inputs; and d) teach teammates by explaining concepts and demonstrating skills. HARMONIC will us a novel dual robotic/cognitive architecture to combine tactical robotic perception and action with strategic cognitive agency capabilities of understanding and decision-making. A key novelty in the project is an unprecedented scope of situational and informational content HARMONIC will leverage in support of its operation. This content will include the robot s computational cognitive models of the world (ontology), of language, of general human activities and Navy-specific activities and of general and Navy-specific human interactions as well as metacognition-supporting models of self and teammates. Special attention will be given to robotic behavior in the face of disturbances, thus addressing a critical issue of robotic safety under dynamic conditions.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Nov 08, 2024
- Source ID
- N000142412679
Entities
People
- Sergei Nirenburg
Organizations
- Office of Naval Research
- Rensselaer Polytechnic Institute
- United States Navy