Common Sense for Superhuman Perception- From Vibration to Vision to Verbal Multiscale Multimodal Intelligence
Abstract
The PI introduces a framework that learns common sense knowledge about the world through agents embodied in simulated or real world scenes, that move and interact with them. The agents learn perceptual representations of objects and their dynamics from their self-generated visuomotor experiences, as well as from experiences of human teachers that similarly interact with real and simulated scenes and additionally narrate their experiences in natural language. This permits the agents to associate sensory representations with language symbols.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Mar 07, 2023
- Source ID
- FA95502110092
Entities
People
- Katerina Fragkiadaki
Organizations
- Air Force Office of Scientific Research
- Carnegie Mellon University
- United States Air Force