DURIP Extended Reality Simulation and Control of Aerospace Vehicles with Brain Activity Monitoring
Abstract
Next-generation aircraft and rotorcraft configurations have become more complex. Future Vertical Lift (FVL) and 6th generationfighter jets feature high levels of control augmentation, control redundancy, and performance that exceed the current piloting capabilities of the human body. These features drive the need for novel human-machine interfaces that ease the pilot workload and cue only the necessary information to maximize pilot-vehicle performance while ensuring the safety of flight operations. Additionally, because of the piloting complexity and operational cost of future-generation aerospace vehicles, novel training solutions must be investigated to replace potentially unsafe and expensive live simulations and training. The proposed experimental equipment will enable the long-term investigation of Extended Reality (XR) simulations of aerospace vehicles with the broad aim of maximizing the pilot-vehicle performance through optimal human-machine interaction. These simulations will make use of Virtual and Augmented Reality (VR/AR), full-body haptic feedback and biometry tracking suits to enable XR, 6-DoF motion platforms for acceleration cueing, and wearable systems for brain activity monitoring. Investigation to be enabled by the proposed equipment start from the observation that current perception models for shared human-machine piloting of vehicles are based on the dominant visual (sight) and vestibular (equilibrium) cues, but neglect less dominant perception cues, such as somatosensory cues (haptics) or auditory cues (3-D audio), and interactions between primary and secondary sensory channels. As such, there is limited understanding on shared human-machine perception solutions for vehicle piloting when the symbiotic system operates with denied/impaired sensory channels (degraded visual environment), denied/impaired interactions between sensory channels, or with primary sensory channels augmented with traditionally secondary perception cues. Denialor impairment of perceptual channels may arise environment conditions (weather), factors limiting machine abilities (sensor failure), or human pilot abilities (temporary or permanent disabilities like fatigue). Within the context of manned piloting of aircraft, modeling of secondary sensory cues could help develop cueing strategies to enable emergent pilot vehicle system (PVS) performance, or to supplement for pilot spatial disorientation which may be caused by temporary or permanent loss/malfunction oftheir vestibular and/or visual systems. As such, we aim to (i) characterize the role of secondary cues for piloting when primary cues are disrupted, (ii) characterize interaction between primary and secondary cues, (iii) explore how primary or primary-secondary cues may be replaced or augmented by different combinations of sensory input, and (iv) leverage these findings to enable real-time cross-diagnostics of human/AI pilot performance. These findings will be validated in XR piloted simulation studies involving manned fixed-wing aircraft and rotorcraft operations. The broader impact of the research enabled by the proposed equipment, within the context of the Navy s specific mission, is articulated into three categories: (i) time and cost abatement for the creation of Launchand Recovery Envelopes (LREs) aboard naval vessels, (ii) increase of the safety in shipboard approach and landing operations for both conventional helicopter configurations and tiltrotor aircraft, and (iii) enabling disruptive human-machine interfaces with applications to shipboard approach and landing, air-to-air combat in and out of line of sight, and formation flying operations like aerialrefueling. With regards to shipboard approach and landing, the proposed investigation is complementary to the PI#s 2022 ONR YIP award.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Mar 08, 2024
- Source ID
- N000142412167
Entities
People
- Umberto Saetti
Organizations
- Office of Naval Research
- United States Navy
- University of Maryland