Video Analysis for Perception and Action
Abstract
(Approved for public release.)The great vision scientist, James J. Gibson, famously said #We see in order to move and we move in order to see#. For an animal, be it a fly, a fish, an eagle, a tiger, or a human, vision is a dynamic process - images are acquired asthe animal moves about # which means that the fundamental input is a video stream. So it should be in computer vision. But in our field#s sixty year history, emphasis has largely been on analyzing static images, because even these were challenging from a computational point of view. We believe that the time has come to fully exploit video data, which is necessary to take us to the next stage of visual understanding. However, computing with video data is extremely challenging because of the sheer volume of data involved, and also due to the streaming nature of the video signal, which makes it very correlated in time. The purpose of this DURIP grant is to request funding to help build a computing cluster at UC Berkeley which will allow research on video data within academic researchenvironment. Particular areas of investigation, that will be directly enabled by this grant include: 1) learning from large-scale (over 3000 hours), egocentric data encompassing a wide range of visual activities, 2) using videos to better understand the three-dimensional structure of our visual world, 3) providing visual analysis at large temporal scales (tens of minutes instead of just a couple of seconds) # something that is not feasible with current computing setups, 4) discovering spatio-temporal patterns within large-scale video data, 5) analyzing videos using analysis-by-synthesis techniques, 6) using the knowledge learned from large-scale videodata and apply it to modeling locomotion and manipulation of robotic agents in the real world. Furthermore, the proposed equipment will also impact several of our ongoing DoD supported research projects.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Mar 03, 2023
- Source ID
- N000142312287
Entities
People
- Jitendra Malik
Organizations
- Office of Naval Research
- United States Navy
- University of California Regents