Hierarchical Generative Models
Abstract
Statement of Work:The Pis, Alan Yuille (UCLA) and Stuart Geman (Brown Univ.), will extend the generative hierarchical compositional approaches to parse 2D images into 3D scenes. Yuille will develop generative compositional models of objects concentrating on the hierarchical structure, the spatial relations between parts and subparts, the three-dimensional models, and the inference and learning algorithms for the object models. He will use the generative appearance models supplied by Geman as input features (likelihood ratios), contrast their performance with alternatives such as Deep Convolutional Neural Networks (DCNNs), and develop novel generative models which exploit the newappearance cues. This task division will be repeated in all three years of the grant. In year 1, the task will be to develop hierarchical generative models of objects concentrating on the shape and appearance of significant parts of the objects such as joints. In year 2, the task will be to extend these models to include object parts and enable semantic segmentation. In year 3, the task will extend the models to include three-dimensional structure. In all cases, the Pis retain the same division of tasks between modeling appearance (Geman) and modeling structure (Yuille).Objective:Develop a principled method, based on generative hierarchical compositional approaches, for modeling objects and parsing images into 3D scenes and objects, with particular emphasis on 3D parsing of humans and inferring their activities.Approach:The Pis will address two important aspects in recognition, namely, appearance and structure (shape). They will extend their promising approach based on generative hierarchical compositional models for visual recognition. They will investigate and develop the following: (a) Learning richer generative models. Develop a theoretical framework to integrate generative appearance models into compositional framework of objects. Recursive construction of visual dictionaries using the curriculum learning strategy. Continue to extend benchmarking datasets and use them to trainand evaluate the new algorithms. (b) Richer representations of objects. Extend the theoretical framework to include generative appearance models of object parts enabling semantic segmentation. Extend datasets to contain labels for occluded parts and develop, and test, algorithms to parse in complex scenes with significant occlusion. ~ 3D objectmodels. Extend the approach to learn generative appearance models of 3D objects. Develop datasets for testing and evaluating these algorithms.ONR mission: This research addresses Information Dominance and Autonomy focus areas. This work is expected todevelop reliable methods for scene understanding from images.Overall merit: This research is expected to develop a principled method for scene understanding, based on learning generative-hierarchical-compositional models of objects, that will have the capability to parse 2D images into 3D scenes.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Sep 23, 2016
- Source ID
- N000141613168
Entities
People
- Alan Yuille
Organizations
- Johns Hopkins University
- Office of Naval Research
- United States Navy