Learning to Mine a Soundscape
Abstract
Approved for Public ReleaseSUMMARY ONR Announcement # N00014-21-S-F003 - Topic 18: Learning from HearingResearch Topic Chiefs:Dr. Ky,le M. Becker, ONR 322, 703-696-6832, kyle.becker1@navy.milDr. Harold Hawkins, ONR 341, 703-696-4323, harold.hawkins@navy.milLEARNING, TO MINE A SOUNDSCAPE PI: Mounya Elhilali (Johns Hopkins University)Problem statement and project goals: As humans, animals and mach,ines navigate and interact with their environments, they can infer a multitude of information from the acoustic signals that they re,ceive and integrate it with other modalities to make sense of the changing surroundings and adapt their behaviors. Acoustic signalsn,ot only convey information about sound events, but also inform of the physical structure of the environment as well as context. This, inference of objects and events from sounds is informed by priors or learned representations that guide perception and behavior. Th,e current project aims to identify the nature and role of these priors in guiding perception and behavior in humans, animals and mod,els. There are 3 critical factors to consider when exploring these priors. Origins and Structure: Priors are informed by physical at,tributes of sources, constraints of the environment and semantic mappings of events in their context. Learning: They are learned ove,r short-term (scene context within seconds and minutes) and long-term (over hours, days and lifetimes). Engagement: Priors are adapt,ed by task demands, modulated by behavioral outcomes, and reshaped by cognitive constraints (attention, memory). The project examine,s these multiple facets in 3 interleaved research thrusts that shed light on: (1) the space of acoustic taxonomies that give rise to, these priors, (2) behavioral and neural systems that reflect acquisition and deployment of these priors, and (3) models and algorit,hms to emulate this function in mathematical models (see diagram below). The proposal explores these questions by bringing together, an interdisciplinary team with expertise in sensors, physical acoustics, physics-based simulations, creation of auditory virtual re,ality, animal behavior and neurophysiology, cognitive science, learning theory, signal processing and artificial intelligence. Poten,tial impact on DoD capabilities: The approach laid out in this project will have a direct impact on building robust systems for inte,lligent audio analytics. By shedding light on behavioral and neural underpinnings of building priors from soundscapes and exploiting, them, we envision translation of theories to models for surveillance in urban environments, mine-warfare in underwater settings, as, well as audio object analysis and recognition and anomaly detection. Progress in this area has many potential long-term benefits, i,ncluding improved communication in natural and chaotic situations; as well next-generation auditory displays and assistive listening, devices. In parallel, hardware and augmented reality systems developed in this project have direct applications for mission trainin,g and readiness-enhancing technologies. Also, physical acoustics models can directly inform beamforming and aperture arrays of relev,ance to DoD capability.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Dec 06, 2022
- Source ID
- N000142312086
Entities
People
- Mounya Elhilali
Organizations
- Johns Hopkins University
- Office of Naval Research
- United States Navy