THIS IS A CONTINUATION OF N00014-13-1-0762 Structural learning methods for modeling complex human activities from vide
Abstract
Short Work Statement:Develop novel machine learning approaches for learning models of complex human activities from video for visualscene interpretation.Objective:The goal is to develop novel structured machine learning approaches for learning models of complex activities from video for visual scene interpretation.Approach:This is joint research by PIs Honglak Lee (U. Michigan) and Silvio Savarese (Stanford). The PIs will develop a new framework for learning models of visual concepts and using these models for enabling a system to recognize and interpret the visual content of images and videos. They will focus on recognizing visual concepts that describe complex human activities. To fully understand the activities that take place in a scene, not only must a visual system recognize the existence of the key objects and their pose in isolation, but also recognize the underlying complex structure ofinteractions at the appropriate level of semantic and spatial- temporal resolution. Most current research for activity recognition are focused on recognizing activities in isolation performed by a single actor and do not account for the interaction of individuals with themselves or nearby objects. Moreover, methods characterize videos of activities with just single class labels for actions (e.g, walking, running, reaching), without attempting to interpret complex structuresof activities. The PIs will introduce a novel framework for modeling complex activities based on structured deeplearning in order to: (a) learn models automatically with different degree of supervision; and (b) make the algorithms scalable to diverse environmental settings and flexible to interpret various classes of activities. These models will be used to recognize unknown activities from videos and characterize these videos with a rich list of class labels that describe the activity at different levels of spatial, temporal and semantic resolution. Among other directions, the PIs willinvestigate and develop attribute-based representation and recognition for objects, actions, and interactions.Algorithms will be evaluated in the contexts of robot perception and understanding video surveillance.Overall Merits and ONR RelevanceThis work is expected to substantially contribute to the methods for learning complex concepts, in particular complex activities from video. This work is highly relevant to enabling Navy s Autonomy and Information Dominance. Automated methods for recognizing complex activities are of critical importance to naval missions that include perception for autonomousagents and understanding surveillance imagery.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Sep 30, 2016
- Source ID
- N000141612928
Entities
People
- Honglak Lee
Organizations
- Board of Regents of the University of Michigan
- Office of Naval Research
- United States Navy