Close-Range Collaboration in Diver-Agent Teams Using Diver Nonverbal Behavior and Physiological Signals
Abstract
Divers operate in teams for a number of reasons. Two humans can cover more territory when exploring, jointly complete tasks, and manipulate objects that would be unwieldy for a single person. Most importantly, a dive partner provides an important safety margin#when a human diver is impaired, their dive partner can diagnose impairment through behavior and interactive communications, leverage mutual trust to direct the diver to the surface, and assist as needed until physical and cognitive capacity is restored. These are fundamental challenges for human-agent collaboration which require dynamically shifting roles, assured and trusted autonomy, and reliable communications in contested and perceptually challenged environments. In this proposal, we build on previous work by the PIs on XR and haptic-assisted navigation for visually impaired users and previous research on human-human and human-animal guide pairings, to inform an investigation on how underwater autonomous vehicles (UAV)s can be equipped with the skills to function as effective partners for divers who may experience sensory and/or cognitive impairment in the course of the dive. This knowledge can be applied to current and future iterations of UAV diving guides. The challenges we address include: 1. The need for the UAV to recognize human divers in varied conditions and react appropriately 2.The need for a UAV partner to monitor individual human divers for sensory and/or cognitive impairments that might arise in the course of the dive and assess the need for an intervention.3.The need for divers to easily gather information from UAV sensors in environments that may provide audio and visual challenges, without these additional sources of information impairing their ability to observe and act in their immediate environment4.The need to establish reliable trust and communication between divers and UAVs so that humans can cede the leadership role when appropriate and be safely guided by UAV when impaired. To address these challenges, we will work in both simulated and real-life environments, and build on existing work on enhanced perception in challenging environments, comparing the usability and efficacy of visual, auditory, and haptic (tactile) enhancements. This project will develop new perception algorithms for inferring physiological and emotional states by observing divers# nonverbal behavior integrated with biometric data from wearable sensors and devices, such as gesture recognition gloves and motion trackers. It will add to existing cognitive assessments appropriate for ongoing monitoring of diver behavior. It will develop new communication methods between divers and UAVs that can promote quick and efficient information exchange under circumstances where hand signals, audio cues and text communication are too challenging. The fundamental research in this proposal will aid in the development of communication, sensing, and actuation tools for the UAV-guide to help cognitively/physically impaired divers, navigate safely through confined spaces and ascend to the surface according to individualized safety stops. This complements current ongoing efforts to aid divers, including work on identifying safe timing for return to the surface, dive suits to help divers work in extreme conditions, an augmented reality helmet for human-agent communication, and UAV-assisted navigation, in which the UAV is remote. Approved for public release.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- May 15, 2023
- Source ID
- N000142312469
Entities
People
- Andrea Won
Organizations
- Cornell University
- Office of Naval Research
- United States Navy