Neuromorphic Engineering Needs Closed-Loop Benchmarks
Abstract
Neuromorphic engineering aims to build (autonomous) systems by mimicking biological systems. It is motivated by the observation that biological organisms—from algae to primates—excel in sensing their environment, reacting promptly to their perils and opportunities. Furthermore, they do so more resiliently than our most advanced machines, at a fraction of the power consumption. It follows that the performance of neuromorphic systems should be evaluated in terms of real-time operation, power consumption, and resiliency to real-world perturbations and noise using task-relevant evaluation metrics. Yet, following in the footsteps of conventional machine learning, most neuromorphic benchmarks rely on recorded datasets that foster sensing accuracy as the primary measure for performance. Sensing accuracy is but an arbitrary proxy for the actual system's goal—taking a good decision in a timely manner. Moreover, static datasets hinder our ability to study and compare closed-loop sensing and control strategies that are central to survival for biological organisms. This article makes the case for a renewed focus on closed-loop benchmarks involving real-world tasks. Such benchmarks will be crucial in developing and progressing neuromorphic Intelligence. The shift towards dynamic real-world benchmarking tasks should usher in richer, more resilient, and robust artificially intelligent systems in the future.
Document Details
- Document Type
- Pub Defense Publication
- Publication Date
- Feb 14, 2022
- Source ID
- 10.3389/fnins.2022.813555
Entities
People
- Alexandre Marcireau
- Andre van Schaik
- Bharath Ramesh
- Damien Joubert
- Gregory Cohen
- Moritz B. Milde
- Nicholas O. Ralph
- Nik Dennler
- Saeed Afshar
- Sami El Arja
- Yeshwanth Bethi
- Ying Xu
Organizations
- Air Force Office of Scientific Research