Higher-level perception and control with a fully spiking neural network autopilot for autonomous drone flight
Abstract
There is a need in the Department of Defense (DoD) for advanced compute capabilities on platformsseverely limited in Size, Weight, and Power (SWaP). For example, drones hold a large potential forvarious types of missions, if they are able to perform these missions in an autonomous fashion.However, current solutions for the artificial intelligence of robots require substantial resources interms of sensing, computing power, and memory. For the swift onboard execution of deep artificialneural networks, processors are needed with Graphical Processing Units (GPUs). A typical processoris the NVidia TX2, weighing 76 grams and consuming 7.5 Watts. This makes it suboptimal for smalldrones (< 500 grams) if they have to fly longer distances and makes it unacceptable for very lightweightdrones (<100 grams) that will be operating in swarms to perform collective missions.Spiking neural networks (SNNs) form a novel generation of neural networks that are highly promising,since they will reduce the required computing energy in the order of a factor 1000 or more1. Thisenormous reduction in energy expenditure comes from the fact that neurons in SNNs fire sparsely andasynchronously, and neurons only spend energy when firing. In the context of integrated circuittechnology reaching fundamental physical limits due to thermal and stability issues, big technologycompanies have started to design hardware specifically for spiking nets, such as Intels Loihi and IBMsTrue North. The main problem with SNNs is that their richer internal dynamics and the discrete natureof spikes make the straightforward utilization of traditional learning mechanisms such asbackpropagation impossible. As a consequence of the challenging nature of learning in SNNs, mostSNNs have been applied to relatively simple tasks, and only in recent years there has been progresstowards more complex tasks such as visual object recognition. Moreover, for the same reasons thereis a very limited body of work on SNNs on robotic platforms, and even more so onboard of drones.The goal of this proposal is to create the first fully spiking neural network autopilot for autonomousdrone flight. In particular, in the three years of the project we aim for the SNN autopilot to performall essential low-level tasks such as attitude and velocity control (based on proprioceptive and eventbasedvision inputs) and medium-level tasks such as obstacle avoidance (also based on event-basedvision). A fully SNN autopilot would hold the potential to even allow tiny, light-weight drones to flycompletely autonomously. Besides this technological promise, the scientific interest of the projectalso lies in: (1) Developing new learning techniques both for control and perception, with a focus incontrast to many current SNN learning methods on online learning, i.e., learning that can beexecuted locally at the neuron level as data comes in, enhancing the chance of implementing it onSNN hardware, and (2) analyzing how trained SNNs will perform low-level tasks such as attitude andvelocity control, comparing the SNN solution to traditional state estimation and control techniquesand analyzing the solutions optimality in terms of accuracy and energy expenditure.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Apr 06, 2021
- Source ID
- N629092112014
Entities
People
- Guido de Croon
Organizations
- Delft University of Technology
- Office of Naval Research
- United States Navy