Software and hardware for deep learning of video sequences
Abstract
Software and hardware for deep learning of video sequences Eugenio Culurciello, Purdue University Robotic explorers and combat robots, unmanned aerial vehicles (UAVs), imaging security fences, defense sensor networks, and other embedded vision systems for the DoD all require low cost and high-speed vision systems capable of recognizing and categorizing objects in a scene. In this proposal we propose to deliver an ultra-compact low-power vision system that can implement all the state-of-the-art vision algorithms in real-time, that can be easily programmable from a variety of languages and can operate autonomously or be easily interfaced to existing DoD systems. The ideal synthetic vision system is meant to replace human operators in a variety of tasks. Here we report a few examples: (1) it can be used in cars and vehicles, as cheap widgets able to detect pedestrians, obstacles, street signs, and dangerous situations – a prelude to autonomous navigation (2) in autonomous mobile robots, both wheeled or winged (UAVs), that are guided by vision for military and civilian applications (e.g. ground maintenance). (3) surveillance cameras for autonomous fences that automatically detect events of interest, and only alerts human personnel when specified targets and events are present or detected, video summarization, and attention These examples are of extremely high-importance to the DoD because they would provide reliable artificial eyes that can monitor a video scene and report its content to users, or act upon it. In the context of this proposal we plan to: 1) Study and design algorithms for the recognition of actions in video sequences, and for video to text conversion 2) Study and design algorithm for visual attention coupled to video sequences 3) Advance the algorithms for feature extraction (deep neural networks or DNN) and also temporal analysis (recursive neural networks or RNN) 4) Design hardware components to accelerate the execution of DNN and RNN with performance equal or above current large GPUs, and with 1000x less power consumption. 5) Study and evaluate the migration of hardware from FPGA prototypes to full systems ona- chip (SoC) implementations in current deep submicron circuit fabrication processes.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Aug 12, 2016
- Source ID
- N000141512791
Entities
People
- Eugenio Culurciello
Organizations
- Office of Naval Research
- United States Navy
- University of Virginia