2017 ROPO Project: Generating and combining descriptors based on deep learning
Abstract
This project will address the fundamental research problems pertaining to combiningmultimodal information sources for improved semantic analysis, thereby producing accuratedescription of the captured video scene from saliency and human perception point-of-view.In particular, the study will undertake an investigation on feature extractions and theircombining schemes using various deep learning models. Five information sources (sound,speech, background scene, objects, and optical characters) in a short clip of video can beseparately streamed and tagged with appropriate event descriptor. Each description will bethen combined with each other to produce the overall event description sufficient enough toprovide the segmented metadata to an overall summarization. Some key elements of theinvestigation includes: visual and acoustic signal based detection and classification usingdeep learning; multisensory visual and acoustic based combing rule; image enhancement forbetter scene analysis; and detection and localization of sound sources. The program officerwill collaborate with graduate students to pursue various research issues in their technicalapproaches, solutions, and related publications of the topical areas.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Mar 03, 2017
- Source ID
- N000141712349
Entities
People
- Hanseok Ko
Organizations
- Korea University
- Office of Naval Research
- United States Navy