Situational Awareness, Scene Understanding, and Context Inference With Everyday RF Signals
Abstract
Situational awareness, scene understanding, and context inference through occlusions is an important enabling capability for many applications. Assessing the situation through walls, however, is a considerably challenging and unsolved problem. In this research work, we propose a new foundation that can enable everyday Radio Frequency (RF) signals, such as WiFi, to build a complete scene understanding and context inference through walls. Our approach is multi-disciplinary spanning wireless communications, vision, and signal processing.More specifically, our proposed foundation has four underlying components. In Objective 1, we propose a completely different way of representing and imaging the scene, by exploiting edge interactions with the incoming waves, using Geometrical Theory of Diffraction (GTD) and the corresponding Keller cones. This enables capturing the scene in a compressive yet informative manner, thus addressing the high dimensionality of the RF-based imaging problem of interest. In Objective 2, we propose a methodology for generating a large synthetic RF dataset pertaining to many different scenes, objects, and activities, using available vision datasets, and further infer meaningful statistical metrics on the spatial dependencies of the scenes, to be used in conjunction with our proposed compressive RF imaging pipeline. In Objective 3, we extensively study the synthetic RF dataset that we shall generate under Objective 2, and mathematically characterize a key set of features that are relevant to each scenario. To achieve this, we utilize a combination of full frequency-domain high-energy epoch analysis and short-time transient analysis, in conjunction with the proposed compressive edge representation methodology of Objective 1. This analysis also allows us to train deep-learning pipelines that are generalizable. We further propose to exploit the interesting relationship that exists between the context of a scene and the location of the person in the scene, via utilizing our past work on tracking and localization. Finally, in Objective 4, we run several experiments to validate the proposed foundation and understand its fundamental capabilities and limitations as a function of the given resources and complexity of the scene.Situational awareness is central to many applications as it is key to disaster relief, search and rescue, and security. In particular, the ability to establish a situation through walls can be a considerably useful asset. The goal of the proposed work is to enable complete situational awareness through walls, which can significantly impact the aforementioned applications. Furthermore, by bringing a foundational understanding to the capabilities and limitations of RF-based situational awareness, this work can provide a clear guideline for designing an RF-based system that can achieve the needed level of performance. It will also have significant implications for smart homes/buildings, and personalized services. Overall, this research effort can enablethe successful deployment of an RF-based scene understanding and context inference system, which can impact many applications. (Approved for Public Release)
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jul 24, 2023
- Source ID
- N000142312715
Entities
People
- Yasamin Mostofi
Organizations
- Office of Naval Research
- United States Navy
- University of California, Santa Barbara