Anomaly Detection with Visual Question Answering

Abstract

Anomaly detection is critical for many different use-cases, such as identifying safety hazards to potentially prevent disasters.Developing the capability for a human-robot team to ask targeted questions would be critical to quickly identify a violation of protocol and then quickly take action to rectify the situation. In this report, we experiment with how visual question answering algorithms can be used with a set of carefully constructed questions to detect anomalies in a virtual makerspace and a real-world alleyway. Our exploratory results show improvement over a random baseline and we discuss challenges for future work.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Oct 30, 2023
Accession Number
AD1214022

Entities

People

  • Rahul Sharma
  • Stephanie M. Lukin

Organizations

  • United States Army Research Laboratory
  • University of Maryland

Tags

Fields of Study

  • Computer science

Readers

  • Agent-Based Social Robotics and Mobile-Assisted Learning in Virtual Environments.
  • Systems Analysis and Design

Technology Areas

  • AI & ML
  • AI & ML - DoD AI Strategy
  • AI & ML - Information Retrieval
  • Autonomy