Large-Scale Video Translation by Deep Learning and Knowledge Graph
Abstract
Translating video into natural language has wide range of Navy-relevant applications including surveillance, warfighter assistance,"" human-computer interface, diagnosis, assessment. The increasing ubiquitousness of multimedia information in today~s world has posit""ioned video as a favored information vehicle, and given rise to an astonishing generation of social media and surveillance footage."" With cell-phones now featuring video recording capability along with broadband connectivity, video material can be recorded and dis"tributed across the world just as easily as textcould just a couple of years ago. This raises a series of technological demands for" automatic video understanding and content summarization, which has motivated the research community to guideits steps towards a be""tter attainment of such capabilities. In the meanwhile, it presents the big challenge of semantic understanding of video contents an"d automatic translating them into human language. The underlying basic goal of the proposal is to enhance DoD~s capabilities of visual intelligence for automatic video translation using natural language. We propose to bridge the longstanding semantic gap between computable low-level features and semantics that theyencode. We will address fundamental research problems of representation and inv"ariant description of video data, video dynamics and relation (with scene and other objects) modeling, knowledge representation mode""ling, and natural language description generation. All these researches require a group of Graphics Processing Unit (GPU) servers th"at allow us to efficiently and quickly perform model training and simulation. The support for such equipment will significantly faci"litate our current and future research, and educate scientists and engineers in areas important to national defense, which will cont"inuously contribute to the accomplishment of the DoD~s mission.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Nov 03, 2017
- Source ID
- N000141712924
Entities
People
- Yun Fu
Organizations
- Northeastern University
- Office of Naval Research
- United States Navy