THIS IS A CONTINUATION OF N00014-13-1-0761 Structural learning methods for modeling complex human activities from video
Abstract
Project Summary In this project we propose a new framework for learning models of visual concepts and using these models for enabling a system to recognize and interpret the visual content of images and videos. In this proposal we focus on recognizing visual concepts that describe complex human activities. In order to gain full understanding of the activity that takes place in a scene, not only must a visual system recognize the existence of the key objects and their pose in isolation, but also recognize the underlying complex structure of interactions at the appropriate level of semantic and spatial- temporal resolution. Unfortunately, no state-of-the-art computer vision method can do this. Most of the methods for activity recognition handle activities in isolation performed by a single actor and do not account for the interaction of individuals with themselves or the surrounding objects. Moreover, most of the methods characterize videos of activities with just single class labels (e.g, walking vs running), without attempting to interpret complex structures of activities. Our proposed work seeks to fill this gap and introduces a novel framework for modeling complex structured activities. We will leverage and introduce novel machine learning algorithms based on deep learning in order to: i) learn models automatically with different degree of supervision; ii) make our algorithms scalable to diverse environmental settings and flexible to interpret various classes of activities. We seek to use these models to recognize unknown activities from videos and characterize these videos with a rich list of class labels that describe the activity at different levels of spatial, temporal and semantic resolution
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jun 03, 2016
- Source ID
- N000141612138
Entities
People
- Silvio Savarese
Organizations
- Office of Naval Research
- Stanford University
- United States Navy